blob: 21a96a4bd5ff5d415b93436a5d30725d700b1030 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001:mod:`subprocess` --- Subprocess management
2===========================================
3
4.. module:: subprocess
5 :synopsis: Subprocess management.
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04006
Georg Brandl116aa622007-08-15 14:28:22 +00007.. moduleauthor:: Peter Åstrand <astrand@lysator.liu.se>
8.. sectionauthor:: Peter Åstrand <astrand@lysator.liu.se>
9
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040010**Source code:** :source:`Lib/subprocess.py`
11
12--------------
Georg Brandl116aa622007-08-15 14:28:22 +000013
Georg Brandl116aa622007-08-15 14:28:22 +000014The :mod:`subprocess` module allows you to spawn new processes, connect to their
15input/output/error pipes, and obtain their return codes. This module intends to
Benjamin Peterson5eea8a72014-03-12 21:41:35 -050016replace several older modules and functions::
Georg Brandl116aa622007-08-15 14:28:22 +000017
18 os.system
19 os.spawn*
Georg Brandl116aa622007-08-15 14:28:22 +000020
21Information about how the :mod:`subprocess` module can be used to replace these
22modules and functions can be found in the following sections.
23
Benjamin Peterson41181742008-07-02 20:22:54 +000024.. seealso::
25
26 :pep:`324` -- PEP proposing the subprocess module
27
Georg Brandl116aa622007-08-15 14:28:22 +000028
Ezio Melotti402f75d2012-11-08 10:07:10 +020029Using the :mod:`subprocess` Module
30----------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +000031
Gregory P. Smith6e730002015-04-14 16:14:25 -070032The recommended approach to invoking subprocesses is to use the :func:`run`
Benjamin Petersonef9ffcb2015-04-14 22:12:14 -040033function for all use cases it can handle. For more advanced use cases, the
34underlying :class:`Popen` interface can be used directly.
Nick Coghlanc29248f2011-11-08 20:49:23 +100035
Gregory P. Smith6e730002015-04-14 16:14:25 -070036The :func:`run` function was added in Python 3.5; if you need to retain
37compatibility with older versions, see the :ref:`call-function-trio` section.
Nick Coghlanc29248f2011-11-08 20:49:23 +100038
Gregory P. Smith6e730002015-04-14 16:14:25 -070039
40.. function:: run(args, *, stdin=None, input=None, stdout=None, stderr=None,\
Andriy Maletsky22d131a2018-08-09 23:01:54 +030041 capture_output=False, shell=False, cwd=None, timeout=None, \
Jakub Stasiak7432f092018-11-12 04:40:42 +010042 check=False, encoding=None, errors=None, text=None, env=None, \
Zackery Spytz46545002020-05-17 04:52:47 -060043 universal_newlines=None, **other_popen_kwargs)
Nick Coghlanc29248f2011-11-08 20:49:23 +100044
45 Run the command described by *args*. Wait for command to complete, then
Gregory P. Smith6e730002015-04-14 16:14:25 -070046 return a :class:`CompletedProcess` instance.
Nick Coghlanc29248f2011-11-08 20:49:23 +100047
48 The arguments shown above are merely the most common ones, described below
Nick Coghlan217f05b2011-11-08 22:11:21 +100049 in :ref:`frequently-used-arguments` (hence the use of keyword-only notation
50 in the abbreviated signature). The full function signature is largely the
Bo Baylesce0f33d2018-01-30 00:40:39 -060051 same as that of the :class:`Popen` constructor - most of the arguments to
52 this function are passed through to that interface. (*timeout*, *input*,
53 *check*, and *capture_output* are not.)
Nick Coghlan217f05b2011-11-08 22:11:21 +100054
Bo Baylesce0f33d2018-01-30 00:40:39 -060055 If *capture_output* is true, stdout and stderr will be captured.
56 When used, the internal :class:`Popen` object is automatically created with
57 ``stdout=PIPE`` and ``stderr=PIPE``. The *stdout* and *stderr* arguments may
Gregory P. Smithe8830912019-05-14 12:33:17 -070058 not be supplied at the same time as *capture_output*. If you wish to capture
59 and combine both streams into one, use ``stdout=PIPE`` and ``stderr=STDOUT``
60 instead of *capture_output*.
Nick Coghlanc29248f2011-11-08 20:49:23 +100061
Gregory P. Smith6e730002015-04-14 16:14:25 -070062 The *timeout* argument is passed to :meth:`Popen.communicate`. If the timeout
63 expires, the child process will be killed and waited for. The
Nick Coghlan217f05b2011-11-08 22:11:21 +100064 :exc:`TimeoutExpired` exception will be re-raised after the child process
65 has terminated.
Nick Coghlanc29248f2011-11-08 20:49:23 +100066
Serhiy Storchakafcd9f222013-04-22 20:20:54 +030067 The *input* argument is passed to :meth:`Popen.communicate` and thus to the
68 subprocess's stdin. If used it must be a byte sequence, or a string if
andyclegg7fed7bd2017-10-23 03:01:19 +010069 *encoding* or *errors* is specified or *text* is true. When
Steve Dower050acae2016-09-06 20:16:17 -070070 used, the internal :class:`Popen` object is automatically created with
71 ``stdin=PIPE``, and the *stdin* argument may not be used as well.
Serhiy Storchakafcd9f222013-04-22 20:20:54 +030072
Serhiy Storchaka4adf01c2016-10-19 18:30:05 +030073 If *check* is true, and the process exits with a non-zero exit code, a
Gregory P. Smith6e730002015-04-14 16:14:25 -070074 :exc:`CalledProcessError` exception will be raised. Attributes of that
75 exception hold the arguments, the exit code, and stdout and stderr if they
76 were captured.
77
andyclegg7fed7bd2017-10-23 03:01:19 +010078 If *encoding* or *errors* are specified, or *text* is true,
Steve Dower050acae2016-09-06 20:16:17 -070079 file objects for stdin, stdout and stderr are opened in text mode using the
80 specified *encoding* and *errors* or the :class:`io.TextIOWrapper` default.
andyclegg7fed7bd2017-10-23 03:01:19 +010081 The *universal_newlines* argument is equivalent to *text* and is provided
82 for backwards compatibility. By default, file objects are opened in binary mode.
Steve Dower050acae2016-09-06 20:16:17 -070083
Tobias Kunzeaf1ec972018-06-05 13:41:42 +020084 If *env* is not ``None``, it must be a mapping that defines the environment
85 variables for the new process; these are used instead of the default
86 behavior of inheriting the current process' environment. It is passed directly
87 to :class:`Popen`.
88
Nick Coghlanc29248f2011-11-08 20:49:23 +100089 Examples::
90
Gregory P. Smith6e730002015-04-14 16:14:25 -070091 >>> subprocess.run(["ls", "-l"]) # doesn't capture output
92 CompletedProcess(args=['ls', '-l'], returncode=0)
Nick Coghlanc29248f2011-11-08 20:49:23 +100093
Gregory P. Smith6e730002015-04-14 16:14:25 -070094 >>> subprocess.run("exit 1", shell=True, check=True)
Nick Coghlanc29248f2011-11-08 20:49:23 +100095 Traceback (most recent call last):
Gregory P. Smith6e730002015-04-14 16:14:25 -070096 ...
Nick Coghlanc29248f2011-11-08 20:49:23 +100097 subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
98
Bo Baylesce0f33d2018-01-30 00:40:39 -060099 >>> subprocess.run(["ls", "-l", "/dev/null"], capture_output=True)
Gregory P. Smith6e730002015-04-14 16:14:25 -0700100 CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
Bo Baylesce0f33d2018-01-30 00:40:39 -0600101 stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n', stderr=b'')
Nick Coghlanc29248f2011-11-08 20:49:23 +1000102
Gregory P. Smith6e730002015-04-14 16:14:25 -0700103 .. versionadded:: 3.5
Nick Coghlanc29248f2011-11-08 20:49:23 +1000104
Steve Dower050acae2016-09-06 20:16:17 -0700105 .. versionchanged:: 3.6
106
107 Added *encoding* and *errors* parameters
108
andyclegg7fed7bd2017-10-23 03:01:19 +0100109 .. versionchanged:: 3.7
110
Bo Baylesce0f33d2018-01-30 00:40:39 -0600111 Added the *text* parameter, as a more understandable alias of *universal_newlines*.
112 Added the *capture_output* parameter.
andyclegg7fed7bd2017-10-23 03:01:19 +0100113
Gregory P. Smith6e730002015-04-14 16:14:25 -0700114.. class:: CompletedProcess
Nick Coghlanc29248f2011-11-08 20:49:23 +1000115
Gregory P. Smith6e730002015-04-14 16:14:25 -0700116 The return value from :func:`run`, representing a process that has finished.
Nick Coghlanc29248f2011-11-08 20:49:23 +1000117
Gregory P. Smith6e730002015-04-14 16:14:25 -0700118 .. attribute:: args
Nick Coghlanc29248f2011-11-08 20:49:23 +1000119
Gregory P. Smith6e730002015-04-14 16:14:25 -0700120 The arguments used to launch the process. This may be a list or a string.
Nick Coghlanc29248f2011-11-08 20:49:23 +1000121
Gregory P. Smith6e730002015-04-14 16:14:25 -0700122 .. attribute:: returncode
Serhiy Storchakafcd9f222013-04-22 20:20:54 +0300123
Gregory P. Smith6e730002015-04-14 16:14:25 -0700124 Exit status of the child process. Typically, an exit status of 0 indicates
125 that it ran successfully.
Nick Coghlan217f05b2011-11-08 22:11:21 +1000126
Gregory P. Smith6e730002015-04-14 16:14:25 -0700127 A negative value ``-N`` indicates that the child was terminated by signal
128 ``N`` (POSIX only).
129
130 .. attribute:: stdout
131
132 Captured stdout from the child process. A bytes sequence, or a string if
andyclegg7fed7bd2017-10-23 03:01:19 +0100133 :func:`run` was called with an encoding, errors, or text=True.
134 ``None`` if stdout was not captured.
Gregory P. Smith6e730002015-04-14 16:14:25 -0700135
136 If you ran the process with ``stderr=subprocess.STDOUT``, stdout and
137 stderr will be combined in this attribute, and :attr:`stderr` will be
Serhiy Storchakaecf41da2016-10-19 16:29:26 +0300138 ``None``.
Gregory P. Smith6e730002015-04-14 16:14:25 -0700139
140 .. attribute:: stderr
141
142 Captured stderr from the child process. A bytes sequence, or a string if
andyclegg7fed7bd2017-10-23 03:01:19 +0100143 :func:`run` was called with an encoding, errors, or text=True.
144 ``None`` if stderr was not captured.
Gregory P. Smith6e730002015-04-14 16:14:25 -0700145
146 .. method:: check_returncode()
147
148 If :attr:`returncode` is non-zero, raise a :exc:`CalledProcessError`.
149
150 .. versionadded:: 3.5
Nick Coghlan217f05b2011-11-08 22:11:21 +1000151
152.. data:: DEVNULL
153
154 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
155 to :class:`Popen` and indicates that the special file :data:`os.devnull`
156 will be used.
157
158 .. versionadded:: 3.3
159
Nick Coghlanc29248f2011-11-08 20:49:23 +1000160
161.. data:: PIPE
162
163 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
164 to :class:`Popen` and indicates that a pipe to the standard stream should be
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700165 opened. Most useful with :meth:`Popen.communicate`.
Nick Coghlanc29248f2011-11-08 20:49:23 +1000166
167
168.. data:: STDOUT
169
170 Special value that can be used as the *stderr* argument to :class:`Popen` and
171 indicates that standard error should go into the same handle as standard
172 output.
173
174
Andrew Svetlovb4a09ab2012-08-09 15:11:45 +0300175.. exception:: SubprocessError
176
177 Base class for all other exceptions from this module.
178
179 .. versionadded:: 3.3
180
181
182.. exception:: TimeoutExpired
183
184 Subclass of :exc:`SubprocessError`, raised when a timeout expires
185 while waiting for a child process.
186
187 .. attribute:: cmd
188
189 Command that was used to spawn the child process.
190
191 .. attribute:: timeout
192
193 Timeout in seconds.
194
195 .. attribute:: output
196
Gregory P. Smith6e730002015-04-14 16:14:25 -0700197 Output of the child process if it was captured by :func:`run` or
Andrew Svetlovb4a09ab2012-08-09 15:11:45 +0300198 :func:`check_output`. Otherwise, ``None``.
199
Gregory P. Smith6e730002015-04-14 16:14:25 -0700200 .. attribute:: stdout
201
202 Alias for output, for symmetry with :attr:`stderr`.
203
204 .. attribute:: stderr
205
206 Stderr output of the child process if it was captured by :func:`run`.
207 Otherwise, ``None``.
208
Andrew Svetlovb4a09ab2012-08-09 15:11:45 +0300209 .. versionadded:: 3.3
210
Gregory P. Smith6e730002015-04-14 16:14:25 -0700211 .. versionchanged:: 3.5
212 *stdout* and *stderr* attributes added
Andrew Svetlovb4a09ab2012-08-09 15:11:45 +0300213
214.. exception:: CalledProcessError
215
216 Subclass of :exc:`SubprocessError`, raised when a process run by
217 :func:`check_call` or :func:`check_output` returns a non-zero exit status.
218
219 .. attribute:: returncode
220
Gregory P. Smith ext:(%20%5BGoogle%20Inc.%5D)583a1d62016-06-03 00:31:21 +0000221 Exit status of the child process. If the process exited due to a
222 signal, this will be the negative signal number.
Andrew Svetlovb4a09ab2012-08-09 15:11:45 +0300223
224 .. attribute:: cmd
225
226 Command that was used to spawn the child process.
227
228 .. attribute:: output
229
Gregory P. Smith6e730002015-04-14 16:14:25 -0700230 Output of the child process if it was captured by :func:`run` or
Andrew Svetlovb4a09ab2012-08-09 15:11:45 +0300231 :func:`check_output`. Otherwise, ``None``.
232
Gregory P. Smith6e730002015-04-14 16:14:25 -0700233 .. attribute:: stdout
234
235 Alias for output, for symmetry with :attr:`stderr`.
236
237 .. attribute:: stderr
238
239 Stderr output of the child process if it was captured by :func:`run`.
240 Otherwise, ``None``.
241
242 .. versionchanged:: 3.5
243 *stdout* and *stderr* attributes added
Andrew Svetlovb4a09ab2012-08-09 15:11:45 +0300244
245
Nick Coghlanc29248f2011-11-08 20:49:23 +1000246.. _frequently-used-arguments:
247
248Frequently Used Arguments
249^^^^^^^^^^^^^^^^^^^^^^^^^
250
251To support a wide variety of use cases, the :class:`Popen` constructor (and
252the convenience functions) accept a large number of optional arguments. For
253most typical use cases, many of these arguments can be safely left at their
254default values. The arguments that are most commonly needed are:
255
256 *args* is required for all calls and should be a string, or a sequence of
257 program arguments. Providing a sequence of arguments is generally
258 preferred, as it allows the module to take care of any required escaping
259 and quoting of arguments (e.g. to permit spaces in file names). If passing
260 a single string, either *shell* must be :const:`True` (see below) or else
261 the string must simply name the program to be executed without specifying
262 any arguments.
263
264 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
265 standard output and standard error file handles, respectively. Valid values
Nick Coghlan217f05b2011-11-08 22:11:21 +1000266 are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive
267 integer), an existing file object, and ``None``. :data:`PIPE` indicates
268 that a new pipe to the child should be created. :data:`DEVNULL` indicates
269 that the special file :data:`os.devnull` will be used. With the default
270 settings of ``None``, no redirection will occur; the child's file handles
271 will be inherited from the parent. Additionally, *stderr* can be
272 :data:`STDOUT`, which indicates that the stderr data from the child
273 process should be captured into the same file handle as for *stdout*.
Nick Coghlanc29248f2011-11-08 20:49:23 +1000274
R David Murray1b00f252012-08-15 10:43:58 -0400275 .. index::
276 single: universal newlines; subprocess module
277
Pablo Galindoe14c0102018-02-11 20:58:23 +0000278 If *encoding* or *errors* are specified, or *text* (also known as
279 *universal_newlines*) is true,
Steve Dower050acae2016-09-06 20:16:17 -0700280 the file objects *stdin*, *stdout* and *stderr* will be opened in text
281 mode using the *encoding* and *errors* specified in the call or the
282 defaults for :class:`io.TextIOWrapper`.
Ronald Oussoren385521c2013-07-07 09:26:45 +0200283
Steve Dower050acae2016-09-06 20:16:17 -0700284 For *stdin*, line ending characters ``'\n'`` in the input will be converted
285 to the default line separator :data:`os.linesep`. For *stdout* and *stderr*,
286 all line endings in the output will be converted to ``'\n'``. For more
287 information see the documentation of the :class:`io.TextIOWrapper` class
288 when the *newline* argument to its constructor is ``None``.
289
290 If text mode is not used, *stdin*, *stdout* and *stderr* will be opened as
291 binary streams. No encoding or line ending conversion is performed.
292
293 .. versionadded:: 3.6
294 Added *encoding* and *errors* parameters.
Nick Coghlanc29248f2011-11-08 20:49:23 +1000295
Pablo Galindoe14c0102018-02-11 20:58:23 +0000296 .. versionadded:: 3.7
297 Added the *text* parameter as an alias for *universal_newlines*.
298
Andrew Svetlov50be4522012-08-13 22:09:04 +0300299 .. note::
300
Gregory P. Smith1f8a40b2013-03-20 18:32:03 -0700301 The newlines attribute of the file objects :attr:`Popen.stdin`,
302 :attr:`Popen.stdout` and :attr:`Popen.stderr` are not updated by
303 the :meth:`Popen.communicate` method.
Andrew Svetlov50be4522012-08-13 22:09:04 +0300304
305 If *shell* is ``True``, the specified command will be executed through
Ezio Melotti186d5232012-09-15 08:34:08 +0300306 the shell. This can be useful if you are using Python primarily for the
Nick Coghlanc29248f2011-11-08 20:49:23 +1000307 enhanced control flow it offers over most system shells and still want
Ezio Melotti186d5232012-09-15 08:34:08 +0300308 convenient access to other shell features such as shell pipes, filename
309 wildcards, environment variable expansion, and expansion of ``~`` to a
310 user's home directory. However, note that Python itself offers
311 implementations of many shell-like features (in particular, :mod:`glob`,
312 :mod:`fnmatch`, :func:`os.walk`, :func:`os.path.expandvars`,
313 :func:`os.path.expanduser`, and :mod:`shutil`).
Nick Coghlanc29248f2011-11-08 20:49:23 +1000314
Andrew Svetlov4805fa82012-08-13 22:11:14 +0300315 .. versionchanged:: 3.3
316 When *universal_newlines* is ``True``, the class uses the encoding
317 :func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>`
318 instead of ``locale.getpreferredencoding()``. See the
319 :class:`io.TextIOWrapper` class for more information on this change.
320
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700321 .. note::
Nick Coghlanc29248f2011-11-08 20:49:23 +1000322
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700323 Read the `Security Considerations`_ section before using ``shell=True``.
Andrew Svetlovc2415eb2012-10-28 11:42:26 +0200324
Nick Coghlanc29248f2011-11-08 20:49:23 +1000325These options, along with all of the other options, are described in more
326detail in the :class:`Popen` constructor documentation.
327
328
Sandro Tosi1526ad12011-12-25 11:27:37 +0100329Popen Constructor
Sandro Tosi3e6c8142011-12-25 17:14:11 +0100330^^^^^^^^^^^^^^^^^
Nick Coghlanc29248f2011-11-08 20:49:23 +1000331
332The underlying process creation and management in this module is handled by
333the :class:`Popen` class. It offers a lot of flexibility so that developers
334are able to handle the less common cases not covered by the convenience
335functions.
Georg Brandl116aa622007-08-15 14:28:22 +0000336
337
Gregory P. Smitha1ed5392013-03-23 11:44:25 -0700338.. class:: Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, \
Chris Jerdonek4a4a02b2012-10-10 17:46:18 -0700339 stderr=None, preexec_fn=None, close_fds=True, shell=False, \
Jakub Stasiak7432f092018-11-12 04:40:42 +0100340 cwd=None, env=None, universal_newlines=None, \
Chris Jerdonek4a4a02b2012-10-10 17:46:18 -0700341 startupinfo=None, creationflags=0, restore_signals=True, \
Andre Delfinodcc997c2020-12-16 22:37:28 -0300342 start_new_session=False, pass_fds=(), *, group=None, \
Gregory P. Smithf3751ef2019-10-12 13:24:56 -0700343 extra_groups=None, user=None, umask=-1, \
Ruben Vorderman23c0fb82020-10-20 01:30:02 +0200344 encoding=None, errors=None, text=None, pipesize=-1)
Georg Brandl116aa622007-08-15 14:28:22 +0000345
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700346 Execute a child program in a new process. On POSIX, the class uses
Paul Moore5ab27cc2020-10-20 21:02:24 +0100347 :meth:`os.execvpe`-like behavior to execute the child program. On Windows,
Chris Jerdonek4a4a02b2012-10-10 17:46:18 -0700348 the class uses the Windows ``CreateProcess()`` function. The arguments to
349 :class:`Popen` are as follows.
Georg Brandl116aa622007-08-15 14:28:22 +0000350
Serhiy Storchaka9e3c4522019-05-28 22:49:35 +0300351 *args* should be a sequence of program arguments or else a single string
352 or :term:`path-like object`.
Serhiy Storchakabe50a7b2018-02-28 01:03:46 +0200353 By default, the program to execute is the first item in *args* if *args* is
354 a sequence. If *args* is a string, the interpretation is
355 platform-dependent and described below. See the *shell* and *executable*
356 arguments for additional differences from the default behavior. Unless
357 otherwise stated, it is recommended to pass *args* as a sequence.
Georg Brandl116aa622007-08-15 14:28:22 +0000358
Paul Moore5ab27cc2020-10-20 21:02:24 +0100359 .. warning::
360
361 For maximum reliability, use a fully-qualified path for the executable.
362 To search for an unqualified name on :envvar:`PATH`, use
363 :meth:`shutil.which`. On all platforms, passing :data:`sys.executable`
364 is the recommended way to launch the current Python interpreter again,
365 and use the ``-m`` command-line format to launch an installed module.
366
367 Resolving the path of *executable* (or the first item of *args*) is
368 platform dependent. For POSIX, see :meth:`os.execvpe`, and note that
369 when resolving or searching for the executable path, *cwd* overrides the
370 current working directory and *env* can override the ``PATH``
371 environment variable. For Windows, see the documentation of the
372 ``lpApplicationName`` and ``lpCommandLine`` parameters of WinAPI
373 ``CreateProcess``, and note that when resolving or searching for the
374 executable path with ``shell=False``, *cwd* does not override the
375 current working directory and *env* cannot override the ``PATH``
376 environment variable. Using a full path avoids all of these variations.
377
Tim D. Smith95d024d2020-02-10 14:51:01 -0800378 An example of passing some arguments to an external program
379 as a sequence is::
380
381 Popen(["/usr/bin/git", "commit", "-m", "Fixes a bug."])
382
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700383 On POSIX, if *args* is a string, the string is interpreted as the name or
Chris Jerdonek4a4a02b2012-10-10 17:46:18 -0700384 path of the program to execute. However, this can only be done if not
385 passing arguments to the program.
Georg Brandl116aa622007-08-15 14:28:22 +0000386
R. David Murray5973e4d2010-02-04 16:41:57 +0000387 .. note::
388
Tim D. Smith95d024d2020-02-10 14:51:01 -0800389 It may not be obvious how to break a shell command into a sequence of arguments,
390 especially in complex cases. :meth:`shlex.split` can illustrate how to
391 determine the correct tokenization for *args*::
R. David Murray5973e4d2010-02-04 16:41:57 +0000392
393 >>> import shlex, subprocess
R. David Murray73bc75b2010-02-05 16:25:12 +0000394 >>> command_line = input()
R. David Murray5973e4d2010-02-04 16:41:57 +0000395 /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
396 >>> args = shlex.split(command_line)
397 >>> print(args)
398 ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
399 >>> p = subprocess.Popen(args) # Success!
400
401 Note in particular that options (such as *-input*) and arguments (such
402 as *eggs.txt*) that are separated by whitespace in the shell go in separate
403 list elements, while arguments that need quoting or backslash escaping when
404 used in the shell (such as filenames containing spaces or the *echo* command
405 shown above) are single list elements.
406
Chris Jerdonek4a4a02b2012-10-10 17:46:18 -0700407 On Windows, if *args* is a sequence, it will be converted to a string in a
408 manner described in :ref:`converting-argument-sequence`. This is because
409 the underlying ``CreateProcess()`` operates on strings.
Chris Jerdonek470ee392012-10-08 23:06:57 -0700410
Serhiy Storchaka9e3c4522019-05-28 22:49:35 +0300411 .. versionchanged:: 3.6
412 *args* parameter accepts a :term:`path-like object` if *shell* is
413 ``False`` and a sequence containing path-like objects on POSIX.
414
415 .. versionchanged:: 3.8
416 *args* parameter accepts a :term:`path-like object` if *shell* is
417 ``False`` and a sequence containing bytes and path-like objects
418 on Windows.
419
Serhiy Storchakaa97cd2e2016-10-19 16:43:42 +0300420 The *shell* argument (which defaults to ``False``) specifies whether to use
421 the shell as the program to execute. If *shell* is ``True``, it is
Chris Jerdonek4a4a02b2012-10-10 17:46:18 -0700422 recommended to pass *args* as a string rather than as a sequence.
Chris Jerdonek470ee392012-10-08 23:06:57 -0700423
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700424 On POSIX with ``shell=True``, the shell defaults to :file:`/bin/sh`. If
Chris Jerdonek470ee392012-10-08 23:06:57 -0700425 *args* is a string, the string specifies the command
426 to execute through the shell. This means that the string must be
R. David Murray5973e4d2010-02-04 16:41:57 +0000427 formatted exactly as it would be when typed at the shell prompt. This
428 includes, for example, quoting or backslash escaping filenames with spaces in
429 them. If *args* is a sequence, the first item specifies the command string, and
430 any additional items will be treated as additional arguments to the shell
Chris Jerdonek470ee392012-10-08 23:06:57 -0700431 itself. That is to say, :class:`Popen` does the equivalent of::
R. David Murray5973e4d2010-02-04 16:41:57 +0000432
433 Popen(['/bin/sh', '-c', args[0], args[1], ...])
Georg Brandl116aa622007-08-15 14:28:22 +0000434
Chris Jerdonek470ee392012-10-08 23:06:57 -0700435 On Windows with ``shell=True``, the :envvar:`COMSPEC` environment variable
436 specifies the default shell. The only time you need to specify
437 ``shell=True`` on Windows is when the command you wish to execute is built
438 into the shell (e.g. :command:`dir` or :command:`copy`). You do not need
439 ``shell=True`` to run a batch file or console-based executable.
Georg Brandl116aa622007-08-15 14:28:22 +0000440
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700441 .. note::
Chris Jerdonekcc32a682012-10-10 22:52:22 -0700442
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700443 Read the `Security Considerations`_ section before using ``shell=True``.
Chris Jerdonekcc32a682012-10-10 22:52:22 -0700444
Antoine Pitrouafe8d062014-09-21 21:10:56 +0200445 *bufsize* will be supplied as the corresponding argument to the
446 :func:`open` function when creating the stdin/stdout/stderr pipe
447 file objects:
448
449 - :const:`0` means unbuffered (read and write are one
450 system call and can return short)
451 - :const:`1` means line buffered
452 (only usable if ``universal_newlines=True`` i.e., in a text mode)
453 - any other positive value means use a buffer of approximately that
454 size
455 - negative bufsize (the default) means the system default of
456 io.DEFAULT_BUFFER_SIZE will be used.
Georg Brandl116aa622007-08-15 14:28:22 +0000457
Georg Brandl37b70bb2013-11-25 08:48:37 +0100458 .. versionchanged:: 3.3.1
Gregory P. Smitha1ed5392013-03-23 11:44:25 -0700459 *bufsize* now defaults to -1 to enable buffering by default to match the
Georg Brandl37b70bb2013-11-25 08:48:37 +0100460 behavior that most code expects. In versions prior to Python 3.2.4 and
461 3.3.1 it incorrectly defaulted to :const:`0` which was unbuffered
462 and allowed short reads. This was unintentional and did not match the
463 behavior of Python 2 as most code expected.
Antoine Pitrou4b876202010-06-02 17:10:49 +0000464
Chris Jerdonek470ee392012-10-08 23:06:57 -0700465 The *executable* argument specifies a replacement program to execute. It
466 is very seldom needed. When ``shell=False``, *executable* replaces the
Chris Jerdonek4a4a02b2012-10-10 17:46:18 -0700467 program to execute specified by *args*. However, the original *args* is
468 still passed to the program. Most programs treat the program specified
469 by *args* as the command name, which can then be different from the program
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700470 actually executed. On POSIX, the *args* name
Chris Jerdonek470ee392012-10-08 23:06:57 -0700471 becomes the display name for the executable in utilities such as
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700472 :program:`ps`. If ``shell=True``, on POSIX the *executable* argument
Chris Jerdonek470ee392012-10-08 23:06:57 -0700473 specifies a replacement shell for the default :file:`/bin/sh`.
Georg Brandl116aa622007-08-15 14:28:22 +0000474
Serhiy Storchaka9e3c4522019-05-28 22:49:35 +0300475 .. versionchanged:: 3.6
476 *executable* parameter accepts a :term:`path-like object` on POSIX.
477
478 .. versionchanged:: 3.8
479 *executable* parameter accepts a bytes and :term:`path-like object`
480 on Windows.
481
Nick Coghlanc29248f2011-11-08 20:49:23 +1000482 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
Georg Brandlaf265f42008-12-07 15:06:20 +0000483 standard output and standard error file handles, respectively. Valid values
Ross Lagerwallba102ec2011-03-16 18:40:25 +0200484 are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive
485 integer), an existing :term:`file object`, and ``None``. :data:`PIPE`
486 indicates that a new pipe to the child should be created. :data:`DEVNULL`
Nick Coghlan217f05b2011-11-08 22:11:21 +1000487 indicates that the special file :data:`os.devnull` will be used. With the
488 default settings of ``None``, no redirection will occur; the child's file
489 handles will be inherited from the parent. Additionally, *stderr* can be
490 :data:`STDOUT`, which indicates that the stderr data from the applications
491 should be captured into the same file handle as for stdout.
Georg Brandl116aa622007-08-15 14:28:22 +0000492
493 If *preexec_fn* is set to a callable object, this object will be called in the
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000494 child process just before the child is executed.
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700495 (POSIX only)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000496
497 .. warning::
498
499 The *preexec_fn* parameter is not safe to use in the presence of threads
500 in your application. The child process could deadlock before exec is
501 called.
502 If you must use it, keep it trivial! Minimize the number of libraries
503 you call into.
504
505 .. note::
506
507 If you need to modify the environment for the child use the *env*
508 parameter rather than doing it in a *preexec_fn*.
509 The *start_new_session* parameter can take the place of a previously
510 common use of *preexec_fn* to call os.setsid() in the child.
Georg Brandl116aa622007-08-15 14:28:22 +0000511
Christian Heimes98d90f72019-08-27 23:36:56 +0200512 .. versionchanged:: 3.8
513
514 The *preexec_fn* parameter is no longer supported in subinterpreters.
515 The use of the parameter in a subinterpreter raises
516 :exc:`RuntimeError`. The new restriction may affect applications that
517 are deployed in mod_wsgi, uWSGI, and other embedded environments.
518
Georg Brandl116aa622007-08-15 14:28:22 +0000519 If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
Gregory P. Smithdfb6e542018-03-25 10:27:59 -0700520 :const:`2` will be closed before the child process is executed. Otherwise
521 when *close_fds* is false, file descriptors obey their inheritable flag
522 as described in :ref:`fd_inheritance`.
523
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000524 On Windows, if *close_fds* is true then no handles will be inherited by the
Segev Finerb2a60832017-12-18 11:28:19 +0200525 child process unless explicitly passed in the ``handle_list`` element of
526 :attr:`STARTUPINFO.lpAttributeList`, or by standard handle redirection.
Georg Brandl116aa622007-08-15 14:28:22 +0000527
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000528 .. versionchanged:: 3.2
529 The default for *close_fds* was changed from :const:`False` to
530 what is described above.
531
Segev Finerb2a60832017-12-18 11:28:19 +0200532 .. versionchanged:: 3.7
533 On Windows the default for *close_fds* was changed from :const:`False` to
534 :const:`True` when redirecting the standard handles. It's now possible to
535 set *close_fds* to :const:`True` when redirecting the standard handles.
536
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000537 *pass_fds* is an optional sequence of file descriptors to keep open
538 between the parent and child. Providing any *pass_fds* forces
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700539 *close_fds* to be :const:`True`. (POSIX only)
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000540
Sergey Fedoseevf1202882018-07-06 05:01:16 +0500541 .. versionchanged:: 3.2
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000542 The *pass_fds* parameter was added.
Georg Brandl116aa622007-08-15 14:28:22 +0000543
Chris Jerdonekec3ea942012-09-30 00:10:28 -0700544 If *cwd* is not ``None``, the function changes the working directory to
Serhiy Storchaka9e3c4522019-05-28 22:49:35 +0300545 *cwd* before executing the child. *cwd* can be a string, bytes or
andrei kulakovea39f822021-04-17 05:59:18 -0400546 :term:`path-like <path-like object>` object. On POSIX, the function
Sayan Chowdhuryd5c11f72017-02-26 22:36:10 +0530547 looks for *executable* (or for the first item in *args*) relative to *cwd*
548 if the executable path is a relative path.
549
550 .. versionchanged:: 3.6
Serhiy Storchaka9e3c4522019-05-28 22:49:35 +0300551 *cwd* parameter accepts a :term:`path-like object` on POSIX.
552
553 .. versionchanged:: 3.7
554 *cwd* parameter accepts a :term:`path-like object` on Windows.
555
556 .. versionchanged:: 3.8
557 *cwd* parameter accepts a bytes object on Windows.
Georg Brandl116aa622007-08-15 14:28:22 +0000558
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200559 If *restore_signals* is true (the default) all signals that Python has set to
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000560 SIG_IGN are restored to SIG_DFL in the child process before the exec.
561 Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals.
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700562 (POSIX only)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000563
564 .. versionchanged:: 3.2
565 *restore_signals* was added.
566
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200567 If *start_new_session* is true the setsid() system call will be made in the
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700568 child process prior to the execution of the subprocess. (POSIX only)
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000569
570 .. versionchanged:: 3.2
571 *start_new_session* was added.
572
Patrick McLean2b2ead72019-09-12 10:15:44 -0700573 If *group* is not ``None``, the setregid() system call will be made in the
574 child process prior to the execution of the subprocess. If the provided
575 value is a string, it will be looked up via :func:`grp.getgrnam()` and
576 the value in ``gr_gid`` will be used. If the value is an integer, it
577 will be passed verbatim. (POSIX only)
578
579 .. availability:: POSIX
580 .. versionadded:: 3.9
581
582 If *extra_groups* is not ``None``, the setgroups() system call will be
583 made in the child process prior to the execution of the subprocess.
584 Strings provided in *extra_groups* will be looked up via
585 :func:`grp.getgrnam()` and the values in ``gr_gid`` will be used.
586 Integer values will be passed verbatim. (POSIX only)
587
588 .. availability:: POSIX
589 .. versionadded:: 3.9
590
591 If *user* is not ``None``, the setreuid() system call will be made in the
592 child process prior to the execution of the subprocess. If the provided
593 value is a string, it will be looked up via :func:`pwd.getpwnam()` and
594 the value in ``pw_uid`` will be used. If the value is an integer, it will
595 be passed verbatim. (POSIX only)
596
597 .. availability:: POSIX
598 .. versionadded:: 3.9
599
Gregory P. Smithf3751ef2019-10-12 13:24:56 -0700600 If *umask* is not negative, the umask() system call will be made in the
601 child process prior to the execution of the subprocess.
602
603 .. availability:: POSIX
604 .. versionadded:: 3.9
605
Christian Heimesa342c012008-04-20 21:01:16 +0000606 If *env* is not ``None``, it must be a mapping that defines the environment
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000607 variables for the new process; these are used instead of the default
608 behavior of inheriting the current process' environment.
Georg Brandl116aa622007-08-15 14:28:22 +0000609
R. David Murray1055e892009-04-16 18:15:32 +0000610 .. note::
R. David Murrayf4ac1492009-04-15 22:35:15 +0000611
Georg Brandl2708f3a2009-12-20 14:38:23 +0000612 If specified, *env* must provide any variables required for the program to
613 execute. On Windows, in order to run a `side-by-side assembly`_ the
614 specified *env* **must** include a valid :envvar:`SystemRoot`.
R. David Murrayf4ac1492009-04-15 22:35:15 +0000615
Georg Brandl5d941342016-02-26 19:37:12 +0100616 .. _side-by-side assembly: https://en.wikipedia.org/wiki/Side-by-Side_Assembly
R. David Murray1055e892009-04-16 18:15:32 +0000617
Pablo Galindoe14c0102018-02-11 20:58:23 +0000618 If *encoding* or *errors* are specified, or *text* is true, the file objects
619 *stdin*, *stdout* and *stderr* are opened in text mode with the specified
620 encoding and *errors*, as described above in :ref:`frequently-used-arguments`.
621 The *universal_newlines* argument is equivalent to *text* and is provided
622 for backwards compatibility. By default, file objects are opened in binary mode.
Steve Dower050acae2016-09-06 20:16:17 -0700623
624 .. versionadded:: 3.6
625 *encoding* and *errors* were added.
Georg Brandl116aa622007-08-15 14:28:22 +0000626
Pablo Galindoe14c0102018-02-11 20:58:23 +0000627 .. versionadded:: 3.7
628 *text* was added as a more readable alias for *universal_newlines*.
629
Brian Curtine6242d72011-04-29 22:17:51 -0500630 If given, *startupinfo* will be a :class:`STARTUPINFO` object, which is
631 passed to the underlying ``CreateProcess`` function.
Jamesb5d9e082017-11-08 14:18:59 +0000632 *creationflags*, if given, can be one or more of the following flags:
633
634 * :data:`CREATE_NEW_CONSOLE`
635 * :data:`CREATE_NEW_PROCESS_GROUP`
636 * :data:`ABOVE_NORMAL_PRIORITY_CLASS`
637 * :data:`BELOW_NORMAL_PRIORITY_CLASS`
638 * :data:`HIGH_PRIORITY_CLASS`
639 * :data:`IDLE_PRIORITY_CLASS`
640 * :data:`NORMAL_PRIORITY_CLASS`
641 * :data:`REALTIME_PRIORITY_CLASS`
642 * :data:`CREATE_NO_WINDOW`
643 * :data:`DETACHED_PROCESS`
644 * :data:`CREATE_DEFAULT_ERROR_MODE`
645 * :data:`CREATE_BREAKAWAY_FROM_JOB`
Georg Brandl116aa622007-08-15 14:28:22 +0000646
Ruben Vorderman23c0fb82020-10-20 01:30:02 +0200647 *pipesize* can be used to change the size of the pipe when
648 :data:`PIPE` is used for *stdin*, *stdout* or *stderr*. The size of the pipe
649 is only changed on platforms that support this (only Linux at this time of
650 writing). Other platforms will ignore this parameter.
651
652 .. versionadded:: 3.10
653 The ``pipesize`` parameter was added.
654
Gregory P. Smith6b657452011-05-11 21:42:08 -0700655 Popen objects are supported as context managers via the :keyword:`with` statement:
656 on exit, standard file descriptors are closed, and the process is waited for.
Brian Curtin79cdb662010-12-03 02:46:02 +0000657 ::
658
659 with Popen(["ifconfig"], stdout=PIPE) as proc:
660 log.write(proc.stdout.read())
661
Steve Dower44f91c32019-06-27 10:47:59 -0700662 .. audit-event:: subprocess.Popen executable,args,cwd,env subprocess.Popen
Steve Dower60419a72019-06-24 08:42:54 -0700663
664 Popen and the other functions in this module that use it raise an
665 :ref:`auditing event <auditing>` ``subprocess.Popen`` with arguments
Jules Lasne (jlasne)f25875a2019-11-19 13:14:53 +0100666 ``executable``, ``args``, ``cwd``, and ``env``. The value for ``args``
Steve Dower60419a72019-06-24 08:42:54 -0700667 may be a single string or a list of strings, depending on platform.
668
Brian Curtin79cdb662010-12-03 02:46:02 +0000669 .. versionchanged:: 3.2
670 Added context manager support.
671
Victor Stinner5a48e212016-05-20 12:11:15 +0200672 .. versionchanged:: 3.6
673 Popen destructor now emits a :exc:`ResourceWarning` warning if the child
674 process is still running.
675
Victor Stinnerd7befad2019-04-25 14:30:16 +0200676 .. versionchanged:: 3.8
677 Popen can use :func:`os.posix_spawn` in some cases for better
678 performance. On Windows Subsystem for Linux and QEMU User Emulation,
679 Popen constructor using :func:`os.posix_spawn` no longer raise an
680 exception on errors like missing program, but the child process fails
681 with a non-zero :attr:`~Popen.returncode`.
682
Georg Brandl116aa622007-08-15 14:28:22 +0000683
Georg Brandl116aa622007-08-15 14:28:22 +0000684Exceptions
685^^^^^^^^^^
686
687Exceptions raised in the child process, before the new program has started to
Harmandeep Singh47a2fce2019-01-04 01:23:56 +0530688execute, will be re-raised in the parent.
Georg Brandl116aa622007-08-15 14:28:22 +0000689
690The most common exception raised is :exc:`OSError`. This occurs, for example,
691when trying to execute a non-existent file. Applications should prepare for
Miss Islington (bot)45caee22021-07-24 02:59:22 -0700692:exc:`OSError` exceptions. Note that, when ``shell=True``, :exc:`OSError`
Miss Islington (bot)d0c61752021-07-22 11:25:57 -0700693will be raised by the child only if the selected shell itself was not found.
694To determine if the shell failed to find the requested application, it is
695necessary to check the return code or output from the subprocess.
Georg Brandl116aa622007-08-15 14:28:22 +0000696
697A :exc:`ValueError` will be raised if :class:`Popen` is called with invalid
698arguments.
699
Nick Coghlanc29248f2011-11-08 20:49:23 +1000700:func:`check_call` and :func:`check_output` will raise
701:exc:`CalledProcessError` if the called process returns a non-zero return
702code.
Georg Brandl116aa622007-08-15 14:28:22 +0000703
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400704All of the functions and methods that accept a *timeout* parameter, such as
705:func:`call` and :meth:`Popen.communicate` will raise :exc:`TimeoutExpired` if
706the timeout expires before the process exits.
707
Ronald Oussorenc1577902011-03-16 10:03:10 -0400708Exceptions defined in this module all inherit from :exc:`SubprocessError`.
Gregory P. Smith54d412e2011-03-14 14:08:43 -0400709
710 .. versionadded:: 3.3
711 The :exc:`SubprocessError` base class was added.
712
Miss Islington (bot)d657da82021-08-10 00:51:06 -0700713.. _subprocess-security:
Georg Brandl116aa622007-08-15 14:28:22 +0000714
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700715Security Considerations
716-----------------------
Georg Brandl116aa622007-08-15 14:28:22 +0000717
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700718Unlike some other popen functions, this implementation will never
719implicitly call a system shell. This means that all characters,
720including shell metacharacters, can safely be passed to child processes.
721If the shell is invoked explicitly, via ``shell=True``, it is the application's
722responsibility to ensure that all whitespace and metacharacters are
723quoted appropriately to avoid
Georg Brandl5d941342016-02-26 19:37:12 +0100724`shell injection <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_
Ammar Askarf9a83862020-11-11 02:29:56 -0500725vulnerabilities. On :ref:`some platforms <shlex-quote-warning>`, it is possible
726to use :func:`shlex.quote` for this escaping.
Georg Brandl116aa622007-08-15 14:28:22 +0000727
728
729Popen Objects
730-------------
731
732Instances of the :class:`Popen` class have the following methods:
733
734
735.. method:: Popen.poll()
736
Serhiy Storchaka9e0ae532013-08-24 00:23:38 +0300737 Check if child process has terminated. Set and return
Ivan Chernoff006617f2017-08-29 17:46:24 +0300738 :attr:`~Popen.returncode` attribute. Otherwise, returns ``None``.
Georg Brandl116aa622007-08-15 14:28:22 +0000739
740
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400741.. method:: Popen.wait(timeout=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000742
Serhiy Storchaka9e0ae532013-08-24 00:23:38 +0300743 Wait for child process to terminate. Set and return
744 :attr:`~Popen.returncode` attribute.
Georg Brandl116aa622007-08-15 14:28:22 +0000745
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400746 If the process does not terminate after *timeout* seconds, raise a
747 :exc:`TimeoutExpired` exception. It is safe to catch this exception and
748 retry the wait.
749
Victor Stinner07171242014-02-24 13:18:47 +0100750 .. note::
751
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700752 This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE``
753 and the child process generates enough output to a pipe such that
754 it blocks waiting for the OS pipe buffer to accept more data.
755 Use :meth:`Popen.communicate` when using pipes to avoid that.
756
757 .. note::
758
Victor Stinner07171242014-02-24 13:18:47 +0100759 The function is implemented using a busy loop (non-blocking call and
760 short sleeps). Use the :mod:`asyncio` module for an asynchronous wait:
761 see :class:`asyncio.create_subprocess_exec`.
762
Reid Kleckner28f13032011-03-14 12:36:53 -0400763 .. versionchanged:: 3.3
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400764 *timeout* was added.
Georg Brandl116aa622007-08-15 14:28:22 +0000765
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400766.. method:: Popen.communicate(input=None, timeout=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000767
768 Interact with process: Send data to stdin. Read data from stdout and stderr,
Gareth Reesbf2e5152020-06-24 04:12:10 +0100769 until end-of-file is reached. Wait for process to terminate and set the
770 :attr:`~Popen.returncode` attribute. The optional *input* argument should be
771 data to be sent to the child process, or ``None``, if no data should be sent
772 to the child. If streams were opened in text mode, *input* must be a string.
773 Otherwise, it must be bytes.
Georg Brandl116aa622007-08-15 14:28:22 +0000774
Victor Stinner39892052014-10-14 00:52:07 +0200775 :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``.
Steve Dower050acae2016-09-06 20:16:17 -0700776 The data will be strings if streams were opened in text mode; otherwise,
777 bytes.
Georg Brandl116aa622007-08-15 14:28:22 +0000778
Guido van Rossum0d3fb8a2007-11-26 23:23:18 +0000779 Note that if you want to send data to the process's stdin, you need to create
780 the Popen object with ``stdin=PIPE``. Similarly, to get anything other than
781 ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or
782 ``stderr=PIPE`` too.
783
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400784 If the process does not terminate after *timeout* seconds, a
785 :exc:`TimeoutExpired` exception will be raised. Catching this exception and
786 retrying communication will not lose any output.
787
788 The child process is not killed if the timeout expires, so in order to
789 cleanup properly a well-behaved application should kill the child process and
790 finish communication::
791
792 proc = subprocess.Popen(...)
793 try:
794 outs, errs = proc.communicate(timeout=15)
795 except TimeoutExpired:
796 proc.kill()
797 outs, errs = proc.communicate()
798
Christian Heimes7f044312008-01-06 17:05:40 +0000799 .. note::
Georg Brandl116aa622007-08-15 14:28:22 +0000800
Christian Heimes7f044312008-01-06 17:05:40 +0000801 The data read is buffered in memory, so do not use this method if the data
802 size is large or unlimited.
803
Reid Kleckner28f13032011-03-14 12:36:53 -0400804 .. versionchanged:: 3.3
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400805 *timeout* was added.
806
Georg Brandl116aa622007-08-15 14:28:22 +0000807
Christian Heimesa342c012008-04-20 21:01:16 +0000808.. method:: Popen.send_signal(signal)
809
810 Sends the signal *signal* to the child.
811
Victor Stinnere85a3052020-01-15 17:38:55 +0100812 Do nothing if the process completed.
813
Christian Heimesa342c012008-04-20 21:01:16 +0000814 .. note::
815
Brian Curtineb24d742010-04-12 17:16:38 +0000816 On Windows, SIGTERM is an alias for :meth:`terminate`. CTRL_C_EVENT and
Senthil Kumaran916bd382010-10-15 12:55:19 +0000817 CTRL_BREAK_EVENT can be sent to processes started with a *creationflags*
Brian Curtineb24d742010-04-12 17:16:38 +0000818 parameter which includes `CREATE_NEW_PROCESS_GROUP`.
Christian Heimesa342c012008-04-20 21:01:16 +0000819
Christian Heimesa342c012008-04-20 21:01:16 +0000820
821.. method:: Popen.terminate()
822
Mathieu Dupuy65460562020-05-17 21:29:51 +0000823 Stop the child. On POSIX OSs the method sends SIGTERM to the
Georg Brandl60203b42010-10-06 10:11:56 +0000824 child. On Windows the Win32 API function :c:func:`TerminateProcess` is called
Christian Heimesa342c012008-04-20 21:01:16 +0000825 to stop the child.
826
Christian Heimesa342c012008-04-20 21:01:16 +0000827
828.. method:: Popen.kill()
829
Mathieu Dupuy65460562020-05-17 21:29:51 +0000830 Kills the child. On POSIX OSs the function sends SIGKILL to the child.
Christian Heimesa342c012008-04-20 21:01:16 +0000831 On Windows :meth:`kill` is an alias for :meth:`terminate`.
832
Christian Heimesa342c012008-04-20 21:01:16 +0000833
Georg Brandl116aa622007-08-15 14:28:22 +0000834The following attributes are also available:
835
Gregory P. Smith024c5ee2014-04-29 11:33:23 -0700836.. attribute:: Popen.args
837
838 The *args* argument as it was passed to :class:`Popen` -- a
839 sequence of program arguments or else a single string.
840
841 .. versionadded:: 3.3
Georg Brandl734e2682008-08-12 08:18:18 +0000842
Georg Brandl116aa622007-08-15 14:28:22 +0000843.. attribute:: Popen.stdin
844
Benjamin Peterson3d8814e2014-01-18 00:45:56 -0500845 If the *stdin* argument was :data:`PIPE`, this attribute is a writeable
Steve Dower050acae2016-09-06 20:16:17 -0700846 stream object as returned by :func:`open`. If the *encoding* or *errors*
847 arguments were specified or the *universal_newlines* argument was ``True``,
848 the stream is a text stream, otherwise it is a byte stream. If the *stdin*
849 argument was not :data:`PIPE`, this attribute is ``None``.
Georg Brandl116aa622007-08-15 14:28:22 +0000850
851
852.. attribute:: Popen.stdout
853
Benjamin Peterson3d8814e2014-01-18 00:45:56 -0500854 If the *stdout* argument was :data:`PIPE`, this attribute is a readable
855 stream object as returned by :func:`open`. Reading from the stream provides
Steve Dower050acae2016-09-06 20:16:17 -0700856 output from the child process. If the *encoding* or *errors* arguments were
857 specified or the *universal_newlines* argument was ``True``, the stream is a
858 text stream, otherwise it is a byte stream. If the *stdout* argument was not
859 :data:`PIPE`, this attribute is ``None``.
Benjamin Petersonaf69fe22014-01-18 00:49:04 -0500860
Georg Brandl116aa622007-08-15 14:28:22 +0000861
862.. attribute:: Popen.stderr
863
Benjamin Peterson3d8814e2014-01-18 00:45:56 -0500864 If the *stderr* argument was :data:`PIPE`, this attribute is a readable
865 stream object as returned by :func:`open`. Reading from the stream provides
Steve Dower050acae2016-09-06 20:16:17 -0700866 error output from the child process. If the *encoding* or *errors* arguments
867 were specified or the *universal_newlines* argument was ``True``, the stream
868 is a text stream, otherwise it is a byte stream. If the *stderr* argument was
869 not :data:`PIPE`, this attribute is ``None``.
Georg Brandl116aa622007-08-15 14:28:22 +0000870
Gregory P. Smith6436cba2014-05-11 13:26:21 -0700871.. warning::
872
873 Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`,
874 :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid
875 deadlocks due to any of the other OS pipe buffers filling up and blocking the
876 child process.
877
Georg Brandl116aa622007-08-15 14:28:22 +0000878
879.. attribute:: Popen.pid
880
881 The process ID of the child process.
882
Georg Brandl58bfdca2010-03-21 09:50:49 +0000883 Note that if you set the *shell* argument to ``True``, this is the process ID
884 of the spawned shell.
885
Georg Brandl116aa622007-08-15 14:28:22 +0000886
887.. attribute:: Popen.returncode
888
Christian Heimes7f044312008-01-06 17:05:40 +0000889 The child return code, set by :meth:`poll` and :meth:`wait` (and indirectly
890 by :meth:`communicate`). A ``None`` value indicates that the process
891 hasn't terminated yet.
Georg Brandl48310cd2009-01-03 21:18:54 +0000892
Christian Heimes7f044312008-01-06 17:05:40 +0000893 A negative value ``-N`` indicates that the child was terminated by signal
Gregory P. Smith8e0aa052014-05-11 13:28:35 -0700894 ``N`` (POSIX only).
Georg Brandl116aa622007-08-15 14:28:22 +0000895
896
Brian Curtine6242d72011-04-29 22:17:51 -0500897Windows Popen Helpers
898---------------------
899
900The :class:`STARTUPINFO` class and following constants are only available
901on Windows.
902
Berker Peksagf5184742017-03-01 12:51:55 +0300903.. class:: STARTUPINFO(*, dwFlags=0, hStdInput=None, hStdOutput=None, \
Segev Finerb2a60832017-12-18 11:28:19 +0200904 hStdError=None, wShowWindow=0, lpAttributeList=None)
Brian Curtin73365dd2011-04-29 22:18:33 -0500905
Brian Curtine6242d72011-04-29 22:17:51 -0500906 Partial support of the Windows
Georg Brandl5d941342016-02-26 19:37:12 +0100907 `STARTUPINFO <https://msdn.microsoft.com/en-us/library/ms686331(v=vs.85).aspx>`__
Berker Peksagf5184742017-03-01 12:51:55 +0300908 structure is used for :class:`Popen` creation. The following attributes can
909 be set by passing them as keyword-only arguments.
910
911 .. versionchanged:: 3.7
912 Keyword-only argument support was added.
Brian Curtine6242d72011-04-29 22:17:51 -0500913
914 .. attribute:: dwFlags
915
Senthil Kumarana6bac952011-07-04 11:28:30 -0700916 A bit field that determines whether certain :class:`STARTUPINFO`
917 attributes are used when the process creates a window. ::
Brian Curtine6242d72011-04-29 22:17:51 -0500918
919 si = subprocess.STARTUPINFO()
920 si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW
921
922 .. attribute:: hStdInput
923
Senthil Kumarana6bac952011-07-04 11:28:30 -0700924 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
925 is the standard input handle for the process. If
926 :data:`STARTF_USESTDHANDLES` is not specified, the default for standard
927 input is the keyboard buffer.
Brian Curtine6242d72011-04-29 22:17:51 -0500928
929 .. attribute:: hStdOutput
930
Senthil Kumarana6bac952011-07-04 11:28:30 -0700931 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
932 is the standard output handle for the process. Otherwise, this attribute
933 is ignored and the default for standard output is the console window's
Brian Curtine6242d72011-04-29 22:17:51 -0500934 buffer.
935
936 .. attribute:: hStdError
937
Senthil Kumarana6bac952011-07-04 11:28:30 -0700938 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
939 is the standard error handle for the process. Otherwise, this attribute is
Brian Curtine6242d72011-04-29 22:17:51 -0500940 ignored and the default for standard error is the console window's buffer.
941
942 .. attribute:: wShowWindow
943
Senthil Kumarana6bac952011-07-04 11:28:30 -0700944 If :attr:`dwFlags` specifies :data:`STARTF_USESHOWWINDOW`, this attribute
Brian Curtine6242d72011-04-29 22:17:51 -0500945 can be any of the values that can be specified in the ``nCmdShow``
946 parameter for the
Georg Brandl5d941342016-02-26 19:37:12 +0100947 `ShowWindow <https://msdn.microsoft.com/en-us/library/ms633548(v=vs.85).aspx>`__
Senthil Kumarana6bac952011-07-04 11:28:30 -0700948 function, except for ``SW_SHOWDEFAULT``. Otherwise, this attribute is
Brian Curtine6242d72011-04-29 22:17:51 -0500949 ignored.
Brian Curtin73365dd2011-04-29 22:18:33 -0500950
Brian Curtine6242d72011-04-29 22:17:51 -0500951 :data:`SW_HIDE` is provided for this attribute. It is used when
952 :class:`Popen` is called with ``shell=True``.
953
Segev Finerb2a60832017-12-18 11:28:19 +0200954 .. attribute:: lpAttributeList
955
956 A dictionary of additional attributes for process creation as given in
957 ``STARTUPINFOEX``, see
958 `UpdateProcThreadAttribute <https://msdn.microsoft.com/en-us/library/windows/desktop/ms686880(v=vs.85).aspx>`__.
959
960 Supported attributes:
961
962 **handle_list**
963 Sequence of handles that will be inherited. *close_fds* must be true if
964 non-empty.
965
966 The handles must be temporarily made inheritable by
967 :func:`os.set_handle_inheritable` when passed to the :class:`Popen`
968 constructor, else :class:`OSError` will be raised with Windows error
969 ``ERROR_INVALID_PARAMETER`` (87).
970
971 .. warning::
972
973 In a multithreaded process, use caution to avoid leaking handles
974 that are marked inheritable when combining this feature with
975 concurrent calls to other process creation functions that inherit
976 all handles such as :func:`os.system`. This also applies to
977 standard handle redirection, which temporarily creates inheritable
978 handles.
979
980 .. versionadded:: 3.7
Brian Curtine6242d72011-04-29 22:17:51 -0500981
Jamesb5d9e082017-11-08 14:18:59 +0000982Windows Constants
983^^^^^^^^^^^^^^^^^
Brian Curtine6242d72011-04-29 22:17:51 -0500984
985The :mod:`subprocess` module exposes the following constants.
986
987.. data:: STD_INPUT_HANDLE
988
989 The standard input device. Initially, this is the console input buffer,
990 ``CONIN$``.
991
992.. data:: STD_OUTPUT_HANDLE
993
994 The standard output device. Initially, this is the active console screen
995 buffer, ``CONOUT$``.
996
997.. data:: STD_ERROR_HANDLE
998
999 The standard error device. Initially, this is the active console screen
1000 buffer, ``CONOUT$``.
1001
1002.. data:: SW_HIDE
1003
1004 Hides the window. Another window will be activated.
1005
1006.. data:: STARTF_USESTDHANDLES
1007
1008 Specifies that the :attr:`STARTUPINFO.hStdInput`,
Senthil Kumarana6bac952011-07-04 11:28:30 -07001009 :attr:`STARTUPINFO.hStdOutput`, and :attr:`STARTUPINFO.hStdError` attributes
Brian Curtine6242d72011-04-29 22:17:51 -05001010 contain additional information.
1011
1012.. data:: STARTF_USESHOWWINDOW
1013
Senthil Kumarana6bac952011-07-04 11:28:30 -07001014 Specifies that the :attr:`STARTUPINFO.wShowWindow` attribute contains
Brian Curtine6242d72011-04-29 22:17:51 -05001015 additional information.
1016
1017.. data:: CREATE_NEW_CONSOLE
1018
1019 The new process has a new console, instead of inheriting its parent's
1020 console (the default).
Brian Curtin73365dd2011-04-29 22:18:33 -05001021
Brian Curtin30401932011-04-29 22:20:57 -05001022.. data:: CREATE_NEW_PROCESS_GROUP
1023
1024 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1025 group will be created. This flag is necessary for using :func:`os.kill`
1026 on the subprocess.
1027
1028 This flag is ignored if :data:`CREATE_NEW_CONSOLE` is specified.
1029
Jamesb5d9e082017-11-08 14:18:59 +00001030.. data:: ABOVE_NORMAL_PRIORITY_CLASS
1031
1032 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1033 will have an above average priority.
1034
1035 .. versionadded:: 3.7
1036
1037.. data:: BELOW_NORMAL_PRIORITY_CLASS
1038
1039 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1040 will have a below average priority.
1041
1042 .. versionadded:: 3.7
1043
1044.. data:: HIGH_PRIORITY_CLASS
1045
1046 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1047 will have a high priority.
1048
1049 .. versionadded:: 3.7
1050
1051.. data:: IDLE_PRIORITY_CLASS
1052
1053 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1054 will have an idle (lowest) priority.
1055
1056 .. versionadded:: 3.7
1057
1058.. data:: NORMAL_PRIORITY_CLASS
1059
1060 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1061 will have an normal priority. (default)
1062
1063 .. versionadded:: 3.7
1064
1065.. data:: REALTIME_PRIORITY_CLASS
1066
1067 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1068 will have realtime priority.
1069 You should almost never use REALTIME_PRIORITY_CLASS, because this interrupts
1070 system threads that manage mouse input, keyboard input, and background disk
1071 flushing. This class can be appropriate for applications that "talk" directly
1072 to hardware or that perform brief tasks that should have limited interruptions.
1073
1074 .. versionadded:: 3.7
1075
1076.. data:: CREATE_NO_WINDOW
1077
1078 A :class:`Popen` ``creationflags`` parameter to specify that a new process
Jules Lasne (jlasne)7e9ce4c2019-03-04 19:12:04 +01001079 will not create a window.
Jamesb5d9e082017-11-08 14:18:59 +00001080
1081 .. versionadded:: 3.7
1082
1083.. data:: DETACHED_PROCESS
1084
1085 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1086 will not inherit its parent's console.
1087 This value cannot be used with CREATE_NEW_CONSOLE.
1088
1089 .. versionadded:: 3.7
1090
1091.. data:: CREATE_DEFAULT_ERROR_MODE
1092
1093 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1094 does not inherit the error mode of the calling process. Instead, the new
1095 process gets the default error mode.
1096 This feature is particularly useful for multithreaded shell applications
1097 that run with hard errors disabled.
1098
1099 .. versionadded:: 3.7
1100
1101.. data:: CREATE_BREAKAWAY_FROM_JOB
1102
1103 A :class:`Popen` ``creationflags`` parameter to specify that a new process
1104 is not associated with the job.
1105
1106 .. versionadded:: 3.7
1107
Gregory P. Smith6e730002015-04-14 16:14:25 -07001108.. _call-function-trio:
1109
1110Older high-level API
1111--------------------
1112
1113Prior to Python 3.5, these three functions comprised the high level API to
1114subprocess. You can now use :func:`run` in many cases, but lots of existing code
1115calls these functions.
1116
Zackery Spytz46545002020-05-17 04:52:47 -06001117.. function:: call(args, *, stdin=None, stdout=None, stderr=None, \
1118 shell=False, cwd=None, timeout=None, **other_popen_kwargs)
Gregory P. Smith6e730002015-04-14 16:14:25 -07001119
1120 Run the command described by *args*. Wait for command to complete, then
Berker Peksagbf1d4b62015-07-25 14:27:07 +03001121 return the :attr:`~Popen.returncode` attribute.
Gregory P. Smith6e730002015-04-14 16:14:25 -07001122
Tim Hoffmann1a13efb2019-09-11 13:26:31 +02001123 Code needing to capture stdout or stderr should use :func:`run` instead::
Gregory P. Smith6e730002015-04-14 16:14:25 -07001124
1125 run(...).returncode
1126
Gregory P. Smith7a2e84c2019-03-23 00:40:28 -07001127 To suppress stdout or stderr, supply a value of :data:`DEVNULL`.
Gregory P. Smith6e730002015-04-14 16:14:25 -07001128
Gregory P. Smith7a2e84c2019-03-23 00:40:28 -07001129 The arguments shown above are merely some common ones.
1130 The full function signature is the
Berker Peksagbf1d4b62015-07-25 14:27:07 +03001131 same as that of the :class:`Popen` constructor - this function passes all
1132 supplied arguments other than *timeout* directly through to that interface.
1133
Gregory P. Smith6e730002015-04-14 16:14:25 -07001134 .. note::
1135
1136 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
1137 function. The child process will block if it generates enough
1138 output to a pipe to fill up the OS pipe buffer as the pipes are
1139 not being read from.
1140
1141 .. versionchanged:: 3.3
1142 *timeout* was added.
1143
Zackery Spytz46545002020-05-17 04:52:47 -06001144.. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, \
1145 shell=False, cwd=None, timeout=None, \
1146 **other_popen_kwargs)
Gregory P. Smith6e730002015-04-14 16:14:25 -07001147
1148 Run command with arguments. Wait for command to complete. If the return
1149 code was zero then return, otherwise raise :exc:`CalledProcessError`. The
1150 :exc:`CalledProcessError` object will have the return code in the
1151 :attr:`~CalledProcessError.returncode` attribute.
Miss Islington (bot)31be5442021-09-07 11:44:16 -07001152 If :func:`check_call` was unable to start the process it will propagate the exception
1153 that was raised.
Gregory P. Smith6e730002015-04-14 16:14:25 -07001154
Tim Hoffmann1a13efb2019-09-11 13:26:31 +02001155 Code needing to capture stdout or stderr should use :func:`run` instead::
Gregory P. Smith6e730002015-04-14 16:14:25 -07001156
1157 run(..., check=True)
1158
Gregory P. Smith7a2e84c2019-03-23 00:40:28 -07001159 To suppress stdout or stderr, supply a value of :data:`DEVNULL`.
Gregory P. Smith6e730002015-04-14 16:14:25 -07001160
Gregory P. Smith7a2e84c2019-03-23 00:40:28 -07001161 The arguments shown above are merely some common ones.
1162 The full function signature is the
Berker Peksagbf1d4b62015-07-25 14:27:07 +03001163 same as that of the :class:`Popen` constructor - this function passes all
1164 supplied arguments other than *timeout* directly through to that interface.
1165
Gregory P. Smith6e730002015-04-14 16:14:25 -07001166 .. note::
1167
1168 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
1169 function. The child process will block if it generates enough
1170 output to a pipe to fill up the OS pipe buffer as the pipes are
1171 not being read from.
1172
1173 .. versionchanged:: 3.3
1174 *timeout* was added.
1175
1176
Steve Dower050acae2016-09-06 20:16:17 -07001177.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, \
Alex Gaynor368cf1d2017-05-25 22:28:17 -04001178 cwd=None, encoding=None, errors=None, \
Zackery Spytz46545002020-05-17 04:52:47 -06001179 universal_newlines=None, timeout=None, text=None, \
1180 **other_popen_kwargs)
Gregory P. Smith6e730002015-04-14 16:14:25 -07001181
1182 Run command with arguments and return its output.
1183
1184 If the return code was non-zero it raises a :exc:`CalledProcessError`. The
1185 :exc:`CalledProcessError` object will have the return code in the
1186 :attr:`~CalledProcessError.returncode` attribute and any output in the
1187 :attr:`~CalledProcessError.output` attribute.
1188
1189 This is equivalent to::
1190
1191 run(..., check=True, stdout=PIPE).stdout
1192
Gregory P. Smith7a2e84c2019-03-23 00:40:28 -07001193 The arguments shown above are merely some common ones.
Berker Peksagbf1d4b62015-07-25 14:27:07 +03001194 The full function signature is largely the same as that of :func:`run` -
1195 most arguments are passed directly through to that interface.
Gregory P. Smith64abf372020-12-24 20:57:21 -08001196 One API deviation from :func:`run` behavior exists: passing ``input=None``
1197 will behave the same as ``input=b''`` (or ``input=''``, depending on other
1198 arguments) rather than using the parent's standard input file handle.
Berker Peksagbf1d4b62015-07-25 14:27:07 +03001199
Gregory P. Smith6e730002015-04-14 16:14:25 -07001200 By default, this function will return the data as encoded bytes. The actual
1201 encoding of the output data may depend on the command being invoked, so the
1202 decoding to text will often need to be handled at the application level.
1203
Gregory P. Smith7a2e84c2019-03-23 00:40:28 -07001204 This behaviour may be overridden by setting *text*, *encoding*, *errors*,
1205 or *universal_newlines* to ``True`` as described in
1206 :ref:`frequently-used-arguments` and :func:`run`.
Gregory P. Smith6e730002015-04-14 16:14:25 -07001207
1208 To also capture standard error in the result, use
1209 ``stderr=subprocess.STDOUT``::
1210
1211 >>> subprocess.check_output(
1212 ... "ls non_existent_file; exit 0",
1213 ... stderr=subprocess.STDOUT,
1214 ... shell=True)
1215 'ls: non_existent_file: No such file or directory\n'
1216
1217 .. versionadded:: 3.1
1218
1219 .. versionchanged:: 3.3
1220 *timeout* was added.
1221
1222 .. versionchanged:: 3.4
Berker Peksagbf1d4b62015-07-25 14:27:07 +03001223 Support for the *input* keyword argument was added.
Brian Curtine6242d72011-04-29 22:17:51 -05001224
Brice Grosfc1ce812018-02-07 01:46:29 +01001225 .. versionchanged:: 3.6
1226 *encoding* and *errors* were added. See :func:`run` for details.
1227
Jakub Stasiak7432f092018-11-12 04:40:42 +01001228 .. versionadded:: 3.7
1229 *text* was added as a more readable alias for *universal_newlines*.
1230
1231
Benjamin Petersondcf97b92008-07-02 17:30:14 +00001232.. _subprocess-replacements:
1233
Ezio Melotti402f75d2012-11-08 10:07:10 +02001234Replacing Older Functions with the :mod:`subprocess` Module
1235-----------------------------------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +00001236
Nick Coghlanc29248f2011-11-08 20:49:23 +10001237In this section, "a becomes b" means that b can be used as a replacement for a.
Georg Brandl116aa622007-08-15 14:28:22 +00001238
1239.. note::
1240
Nick Coghlanc29248f2011-11-08 20:49:23 +10001241 All "a" functions in this section fail (more or less) silently if the
1242 executed program cannot be found; the "b" replacements raise :exc:`OSError`
1243 instead.
Georg Brandl116aa622007-08-15 14:28:22 +00001244
Nick Coghlanc29248f2011-11-08 20:49:23 +10001245 In addition, the replacements using :func:`check_output` will fail with a
1246 :exc:`CalledProcessError` if the requested operation produces a non-zero
Serhiy Storchaka9e0ae532013-08-24 00:23:38 +03001247 return code. The output is still available as the
1248 :attr:`~CalledProcessError.output` attribute of the raised exception.
Nick Coghlanc29248f2011-11-08 20:49:23 +10001249
1250In the following examples, we assume that the relevant functions have already
Ezio Melotti402f75d2012-11-08 10:07:10 +02001251been imported from the :mod:`subprocess` module.
Georg Brandl116aa622007-08-15 14:28:22 +00001252
1253
David Jones6a617142019-07-16 15:55:19 +01001254Replacing :program:`/bin/sh` shell command substitution
1255^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001256
Martin Panter1050d2d2016-07-26 11:18:21 +02001257.. code-block:: bash
Georg Brandl116aa622007-08-15 14:28:22 +00001258
David Jones6a617142019-07-16 15:55:19 +01001259 output=$(mycmd myarg)
Georg Brandl116aa622007-08-15 14:28:22 +00001260
Martin Panter1050d2d2016-07-26 11:18:21 +02001261becomes::
1262
1263 output = check_output(["mycmd", "myarg"])
Georg Brandl116aa622007-08-15 14:28:22 +00001264
Benjamin Petersonf10a79a2008-10-11 00:49:57 +00001265Replacing shell pipeline
1266^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001267
Martin Panter1050d2d2016-07-26 11:18:21 +02001268.. code-block:: bash
Georg Brandl116aa622007-08-15 14:28:22 +00001269
David Jones6a617142019-07-16 15:55:19 +01001270 output=$(dmesg | grep hda)
Martin Panter1050d2d2016-07-26 11:18:21 +02001271
1272becomes::
1273
Georg Brandl116aa622007-08-15 14:28:22 +00001274 p1 = Popen(["dmesg"], stdout=PIPE)
1275 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Gregory P. Smithe09d2f12011-02-05 21:47:25 +00001276 p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
Georg Brandl116aa622007-08-15 14:28:22 +00001277 output = p2.communicate()[0]
1278
Tim Hoffmann1a13efb2019-09-11 13:26:31 +02001279The ``p1.stdout.close()`` call after starting the p2 is important in order for
1280p1 to receive a SIGPIPE if p2 exits before p1.
Georg Brandl116aa622007-08-15 14:28:22 +00001281
Nick Coghlanc29248f2011-11-08 20:49:23 +10001282Alternatively, for trusted input, the shell's own pipeline support may still
Martin Panter1050d2d2016-07-26 11:18:21 +02001283be used directly:
1284
1285.. code-block:: bash
Nick Coghlanc29248f2011-11-08 20:49:23 +10001286
David Jones6a617142019-07-16 15:55:19 +01001287 output=$(dmesg | grep hda)
Martin Panter1050d2d2016-07-26 11:18:21 +02001288
1289becomes::
1290
sblondonda431f72020-12-13 06:27:22 +01001291 output = check_output("dmesg | grep hda", shell=True)
Nick Coghlanc29248f2011-11-08 20:49:23 +10001292
1293
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001294Replacing :func:`os.system`
1295^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001296
1297::
1298
1299 sts = os.system("mycmd" + " myarg")
Nick Coghlanc29248f2011-11-08 20:49:23 +10001300 # becomes
Miss Islington (bot)6fc6f432021-05-11 13:55:15 -07001301 retcode = call("mycmd" + " myarg", shell=True)
Georg Brandl116aa622007-08-15 14:28:22 +00001302
1303Notes:
1304
1305* Calling the program through the shell is usually not required.
Miss Islington (bot)6fc6f432021-05-11 13:55:15 -07001306* The :func:`call` return value is encoded differently to that of
1307 :func:`os.system`.
1308
1309* The :func:`os.system` function ignores SIGINT and SIGQUIT signals while
1310 the command is running, but the caller must do this separately when
1311 using the :mod:`subprocess` module.
Georg Brandl116aa622007-08-15 14:28:22 +00001312
Georg Brandl116aa622007-08-15 14:28:22 +00001313A more realistic example would look like this::
1314
1315 try:
1316 retcode = call("mycmd" + " myarg", shell=True)
1317 if retcode < 0:
Collin Winterc79461b2007-09-01 23:34:30 +00001318 print("Child was terminated by signal", -retcode, file=sys.stderr)
Georg Brandl116aa622007-08-15 14:28:22 +00001319 else:
Collin Winterc79461b2007-09-01 23:34:30 +00001320 print("Child returned", retcode, file=sys.stderr)
Georg Brandl116aa622007-08-15 14:28:22 +00001321 except OSError as e:
Collin Winterc79461b2007-09-01 23:34:30 +00001322 print("Execution failed:", e, file=sys.stderr)
Georg Brandl116aa622007-08-15 14:28:22 +00001323
1324
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001325Replacing the :func:`os.spawn <os.spawnl>` family
1326^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001327
1328P_NOWAIT example::
1329
1330 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
1331 ==>
1332 pid = Popen(["/bin/mycmd", "myarg"]).pid
1333
1334P_WAIT example::
1335
1336 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
1337 ==>
1338 retcode = call(["/bin/mycmd", "myarg"])
1339
1340Vector example::
1341
1342 os.spawnvp(os.P_NOWAIT, path, args)
1343 ==>
1344 Popen([path] + args[1:])
1345
1346Environment example::
1347
1348 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
1349 ==>
1350 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
1351
1352
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001353
1354Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3`
1355^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001356
1357::
1358
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001359 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
Georg Brandl116aa622007-08-15 14:28:22 +00001360 ==>
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001361 p = Popen(cmd, shell=True, bufsize=bufsize,
1362 stdin=PIPE, stdout=PIPE, close_fds=True)
1363 (child_stdin, child_stdout) = (p.stdin, p.stdout)
Georg Brandl116aa622007-08-15 14:28:22 +00001364
1365::
1366
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001367 (child_stdin,
1368 child_stdout,
1369 child_stderr) = os.popen3(cmd, mode, bufsize)
Georg Brandl116aa622007-08-15 14:28:22 +00001370 ==>
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001371 p = Popen(cmd, shell=True, bufsize=bufsize,
1372 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
1373 (child_stdin,
1374 child_stdout,
1375 child_stderr) = (p.stdin, p.stdout, p.stderr)
1376
1377::
1378
1379 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
1380 ==>
1381 p = Popen(cmd, shell=True, bufsize=bufsize,
1382 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
1383 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
1384
1385Return code handling translates as follows::
1386
1387 pipe = os.popen(cmd, 'w')
1388 ...
1389 rc = pipe.close()
Stefan Krahfc9e08d2010-07-14 10:16:11 +00001390 if rc is not None and rc >> 8:
Ezio Melotti985e24d2009-09-13 07:54:02 +00001391 print("There were some errors")
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001392 ==>
R David Murray17227a72015-09-04 10:01:19 -04001393 process = Popen(cmd, stdin=PIPE)
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001394 ...
1395 process.stdin.close()
1396 if process.wait() != 0:
Ezio Melotti985e24d2009-09-13 07:54:02 +00001397 print("There were some errors")
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001398
1399
1400Replacing functions from the :mod:`popen2` module
1401^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1402
1403.. note::
1404
1405 If the cmd argument to popen2 functions is a string, the command is executed
1406 through /bin/sh. If it is a list, the command is directly executed.
1407
1408::
1409
1410 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
1411 ==>
R David Murrayae9d1932014-05-14 10:09:52 -04001412 p = Popen("somestring", shell=True, bufsize=bufsize,
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001413 stdin=PIPE, stdout=PIPE, close_fds=True)
1414 (child_stdout, child_stdin) = (p.stdout, p.stdin)
1415
1416::
1417
1418 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
1419 ==>
1420 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
1421 stdin=PIPE, stdout=PIPE, close_fds=True)
1422 (child_stdout, child_stdin) = (p.stdout, p.stdin)
1423
1424:class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as
1425:class:`subprocess.Popen`, except that:
1426
1427* :class:`Popen` raises an exception if the execution fails.
1428
Jules Lasne (jlasne)7e9ce4c2019-03-04 19:12:04 +01001429* The *capturestderr* argument is replaced with the *stderr* argument.
Benjamin Peterson87c8d872009-06-11 22:54:11 +00001430
1431* ``stdin=PIPE`` and ``stdout=PIPE`` must be specified.
1432
1433* popen2 closes all file descriptors by default, but you have to specify
Gregory P. Smithf5604852010-12-13 06:45:02 +00001434 ``close_fds=True`` with :class:`Popen` to guarantee this behavior on
1435 all platforms or past Python versions.
Eli Bendersky046a7642011-04-15 07:23:26 +03001436
Nick Coghlanc29248f2011-11-08 20:49:23 +10001437
Nick Coghlanc29248f2011-11-08 20:49:23 +10001438Legacy Shell Invocation Functions
Nick Coghlan32e4a582011-11-08 21:50:58 +10001439---------------------------------
Nick Coghlanc29248f2011-11-08 20:49:23 +10001440
1441This module also provides the following legacy functions from the 2.x
1442``commands`` module. These operations implicitly invoke the system shell and
1443none of the guarantees described above regarding security and exception
1444handling consistency are valid for these functions.
1445
1446.. function:: getstatusoutput(cmd)
1447
Gregory P. Smith738b7d92017-09-06 17:39:23 -07001448 Return ``(exitcode, output)`` of executing *cmd* in a shell.
Nick Coghlanc29248f2011-11-08 20:49:23 +10001449
Tim Golden60798142013-11-05 12:57:25 +00001450 Execute the string *cmd* in a shell with :meth:`Popen.check_output` and
Gregory P. Smith738b7d92017-09-06 17:39:23 -07001451 return a 2-tuple ``(exitcode, output)``. The locale encoding is used;
Tim Golden60798142013-11-05 12:57:25 +00001452 see the notes on :ref:`frequently-used-arguments` for more details.
Tim Golden3a2abb52013-11-03 18:24:50 +00001453
1454 A trailing newline is stripped from the output.
Gregory P. Smith738b7d92017-09-06 17:39:23 -07001455 The exit code for the command can be interpreted as the return code
1456 of subprocess. Example::
Nick Coghlanc29248f2011-11-08 20:49:23 +10001457
1458 >>> subprocess.getstatusoutput('ls /bin/ls')
1459 (0, '/bin/ls')
1460 >>> subprocess.getstatusoutput('cat /bin/junk')
Gregory P. Smith738b7d92017-09-06 17:39:23 -07001461 (1, 'cat: /bin/junk: No such file or directory')
Nick Coghlanc29248f2011-11-08 20:49:23 +10001462 >>> subprocess.getstatusoutput('/bin/junk')
Gregory P. Smith738b7d92017-09-06 17:39:23 -07001463 (127, 'sh: /bin/junk: not found')
1464 >>> subprocess.getstatusoutput('/bin/kill $$')
1465 (-15, '')
Nick Coghlanc29248f2011-11-08 20:49:23 +10001466
Cheryl Sabella2d6097d2018-10-12 10:55:20 -04001467 .. availability:: POSIX & Windows.
R David Murray95b696a2014-03-07 20:04:17 -05001468
1469 .. versionchanged:: 3.3.4
Gregory P. Smith738b7d92017-09-06 17:39:23 -07001470 Windows support was added.
1471
1472 The function now returns (exitcode, output) instead of (status, output)
Xiang Zhang7d161722018-09-22 04:18:20 +08001473 as it did in Python 3.3.3 and earlier. exitcode has the same value as
1474 :attr:`~Popen.returncode`.
Nick Coghlanc29248f2011-11-08 20:49:23 +10001475
1476
1477.. function:: getoutput(cmd)
1478
1479 Return output (stdout and stderr) of executing *cmd* in a shell.
1480
Xiang Zhang7d161722018-09-22 04:18:20 +08001481 Like :func:`getstatusoutput`, except the exit code is ignored and the return
Nick Coghlanc29248f2011-11-08 20:49:23 +10001482 value is a string containing the command's output. Example::
1483
1484 >>> subprocess.getoutput('ls /bin/ls')
1485 '/bin/ls'
1486
Cheryl Sabella2d6097d2018-10-12 10:55:20 -04001487 .. availability:: POSIX & Windows.
R David Murray95b696a2014-03-07 20:04:17 -05001488
1489 .. versionchanged:: 3.3.4
1490 Windows support added
Nick Coghlanc29248f2011-11-08 20:49:23 +10001491
Nick Coghlan32e4a582011-11-08 21:50:58 +10001492
Eli Bendersky046a7642011-04-15 07:23:26 +03001493Notes
1494-----
1495
1496.. _converting-argument-sequence:
1497
1498Converting an argument sequence to a string on Windows
1499^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1500
1501On Windows, an *args* sequence is converted to a string that can be parsed
1502using the following rules (which correspond to the rules used by the MS C
1503runtime):
1504
15051. Arguments are delimited by white space, which is either a
1506 space or a tab.
1507
15082. A string surrounded by double quotation marks is
1509 interpreted as a single argument, regardless of white space
1510 contained within. A quoted string can be embedded in an
1511 argument.
1512
15133. A double quotation mark preceded by a backslash is
1514 interpreted as a literal double quotation mark.
1515
15164. Backslashes are interpreted literally, unless they
1517 immediately precede a double quotation mark.
1518
15195. If backslashes immediately precede a double quotation mark,
1520 every pair of backslashes is interpreted as a literal
1521 backslash. If the number of backslashes is odd, the last
1522 backslash escapes the next double quotation mark as
1523 described in rule 3.
1524
Eli Benderskyd2112312011-04-15 07:26:28 +03001525
Éric Araujo9bce3112011-07-27 18:29:31 +02001526.. seealso::
1527
1528 :mod:`shlex`
1529 Module which provides function to parse and escape command lines.