blob: 1f7315922cd155391380c06f931eec3e97dbf29e [file] [log] [blame]
Georg Brandl8ec7f652007-08-15 14:28:01 +00001
2:mod:`subprocess` --- Subprocess management
3===========================================
4
5.. module:: subprocess
6 :synopsis: Subprocess management.
7.. moduleauthor:: Peter Åstrand <astrand@lysator.liu.se>
8.. sectionauthor:: Peter Åstrand <astrand@lysator.liu.se>
9
10
11.. versionadded:: 2.4
12
13The :mod:`subprocess` module allows you to spawn new processes, connect to their
14input/output/error pipes, and obtain their return codes. This module intends to
15replace several other, older modules and functions, such as::
16
17 os.system
18 os.spawn*
19 os.popen*
20 popen2.*
21 commands.*
22
23Information about how the :mod:`subprocess` module can be used to replace these
24modules and functions can be found in the following sections.
25
Georg Brandl68b4e742008-07-01 19:59:00 +000026.. seealso::
27
28 :pep:`324` -- PEP proposing the subprocess module
29
Georg Brandl8ec7f652007-08-15 14:28:01 +000030
31Using the subprocess Module
32---------------------------
33
Nick Coghlan2ed203a2011-10-26 21:05:56 +100034The recommended approach to invoking subprocesses is to use the following
35convenience functions for all use cases they can handle. For more advanced
36use cases, the underlying :class:`Popen` interface can be used directly.
Nick Coghlan86711572011-10-24 22:19:40 +100037
38
Nick Coghlan2ed203a2011-10-26 21:05:56 +100039.. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
Nick Coghlan86711572011-10-24 22:19:40 +100040
41 Run the command described by *args*. Wait for command to complete, then
42 return the :attr:`returncode` attribute.
43
44 The arguments shown above are merely the most common ones, described below
Nick Coghlan87ba6422011-10-27 17:55:13 +100045 in :ref:`frequently-used-arguments` (hence the slightly odd notation in
46 the abbreviated signature). The full function signature is the same as
47 that of the :class:`Popen` constructor - this functions passes all
48 supplied arguments directly through to that interface.
Nick Coghlan86711572011-10-24 22:19:40 +100049
50 Examples::
51
52 >>> subprocess.call(["ls", "-l"])
53 0
54
Nick Coghlan2ed203a2011-10-26 21:05:56 +100055 >>> subprocess.call("exit 1", shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +100056 1
57
58 .. warning::
59
Nick Coghlan87ba6422011-10-27 17:55:13 +100060 Invoking the system shell with ``shell=True`` can be a security hazard
61 if combined with untrusted input. See the warning under
62 :ref:`frequently-used-arguments` for details.
63
64 .. note::
65
Nick Coghlan2ed203a2011-10-26 21:05:56 +100066 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
67 the pipes are not being read in the current process, the child
68 process may block if it generates enough output to a pipe to fill up
69 the OS pipe buffer.
Nick Coghlan86711572011-10-24 22:19:40 +100070
71
Nick Coghlan87ba6422011-10-27 17:55:13 +100072.. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
Nick Coghlan86711572011-10-24 22:19:40 +100073
74 Run command with arguments. Wait for command to complete. If the return
75 code was zero then return, otherwise raise :exc:`CalledProcessError`. The
76 :exc:`CalledProcessError` object will have the return code in the
77 :attr:`returncode` attribute.
78
Nick Coghlan87ba6422011-10-27 17:55:13 +100079 The arguments shown above are merely the most common ones, described below
80 in :ref:`frequently-used-arguments` (hence the slightly odd notation in
81 the abbreviated signature). The full function signature is the same as
82 that of the :class:`Popen` constructor - this functions passes all
83 supplied arguments directly through to that interface.
84
85 Examples::
Nick Coghlan86711572011-10-24 22:19:40 +100086
87 >>> subprocess.check_call(["ls", "-l"])
88 0
89
Nick Coghlan2ed203a2011-10-26 21:05:56 +100090 >>> subprocess.check_call("exit 1", shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +100091 Traceback (most recent call last):
92 ...
Nick Coghlan2ed203a2011-10-26 21:05:56 +100093 subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
Nick Coghlan86711572011-10-24 22:19:40 +100094
95 .. versionadded:: 2.5
96
97 .. warning::
98
Nick Coghlan87ba6422011-10-27 17:55:13 +100099 Invoking the system shell with ``shell=True`` can be a security hazard
100 if combined with untrusted input. See the warning under
101 :ref:`frequently-used-arguments` for details.
102
103 .. note::
104
105 Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
106 the pipes are not being read in the current process, the child
107 process may block if it generates enough output to a pipe to fill up
108 the OS pipe buffer.
Nick Coghlan86711572011-10-24 22:19:40 +1000109
110
Nick Coghlan87ba6422011-10-27 17:55:13 +1000111.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
Nick Coghlan86711572011-10-24 22:19:40 +1000112
113 Run command with arguments and return its output as a byte string.
114
115 If the return code was non-zero it raises a :exc:`CalledProcessError`. The
116 :exc:`CalledProcessError` object will have the return code in the
117 :attr:`returncode` attribute and any output in the :attr:`output`
118 attribute.
119
Nick Coghlan87ba6422011-10-27 17:55:13 +1000120 The arguments shown above are merely the most common ones, described below
121 in :ref:`frequently-used-arguments` (hence the slightly odd notation in
122 the abbreviated signature). The full function signature is largely the
123 same as that of the :class:`Popen` constructor, except that *stdout* is
124 not permitted as it is used internally. All other supplied arguments are
125 passed directly through to the :class:`Popen` constructor.
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000126
Nick Coghlan86711572011-10-24 22:19:40 +1000127 Examples::
128
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000129 >>> subprocess.check_output(["echo", "Hello World!"])
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000130 'Hello World!\n'
131
132 >>> subprocess.check_output("exit 1", shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +1000133 Traceback (most recent call last):
134 ...
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000135 subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
Nick Coghlan86711572011-10-24 22:19:40 +1000136
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000137 To also capture standard error in the result, use
138 ``stderr=subprocess.STDOUT``::
Nick Coghlan86711572011-10-24 22:19:40 +1000139
140 >>> subprocess.check_output(
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000141 ... "ls non_existent_file; exit 0",
142 ... stderr=subprocess.STDOUT,
143 ... shell=True)
Nick Coghlan86711572011-10-24 22:19:40 +1000144 'ls: non_existent_file: No such file or directory\n'
145
146 .. versionadded:: 2.7
147
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000148 .. warning::
149
Nick Coghlan87ba6422011-10-27 17:55:13 +1000150 Invoking the system shell with ``shell=True`` can be a security hazard
151 if combined with untrusted input. See the warning under
152 :ref:`frequently-used-arguments` for details.
153
154 .. note::
155
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000156 Do not use ``stderr=PIPE`` with this function. As the pipe is not being
157 read in the current process, the child process may block if it
158 generates enough output to the pipe to fill up the OS pipe buffer.
159
Nick Coghlan86711572011-10-24 22:19:40 +1000160
161.. data:: PIPE
162
163 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
164 to :class:`Popen` and indicates that a pipe to the standard stream should be
165 opened.
166
167
168.. data:: STDOUT
169
170 Special value that can be used as the *stderr* argument to :class:`Popen` and
171 indicates that standard error should go into the same handle as standard
172 output.
173
174
Andrew Svetlov8afcec42012-08-09 15:23:49 +0300175.. exception:: CalledProcessError
176
177 Exception raised when a process run by :func:`check_call` or
178 :func:`check_output` returns a non-zero exit status.
179
180 .. attribute:: returncode
181
182 Exit status of the child process.
183
184 .. attribute:: cmd
185
186 Command that was used to spawn the child process.
187
188 .. attribute:: output
189
190 Output of the child process if this exception is raised by
191 :func:`check_output`. Otherwise, ``None``.
192
193
194
Nick Coghlan86711572011-10-24 22:19:40 +1000195.. _frequently-used-arguments:
196
197Frequently Used Arguments
198^^^^^^^^^^^^^^^^^^^^^^^^^
199
200To support a wide variety of use cases, the :class:`Popen` constructor (and
201the convenience functions) accept a large number of optional arguments. For
202most typical use cases, many of these arguments can be safely left at their
203default values. The arguments that are most commonly needed are:
204
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000205 *args* is required for all calls and should be a string, or a sequence of
206 program arguments. Providing a sequence of arguments is generally
207 preferred, as it allows the module to take care of any required escaping
208 and quoting of arguments (e.g. to permit spaces in file names). If passing
209 a single string, either *shell* must be :const:`True` (see below) or else
210 the string must simply name the program to be executed without specifying
211 any arguments.
Nick Coghlan86711572011-10-24 22:19:40 +1000212
213 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
214 standard output and standard error file handles, respectively. Valid values
215 are :data:`PIPE`, an existing file descriptor (a positive integer), an
216 existing file object, and ``None``. :data:`PIPE` indicates that a new pipe
217 to the child should be created. With the default settings of ``None``, no
218 redirection will occur; the child's file handles will be inherited from the
219 parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that
220 the stderr data from the child process should be captured into the same file
221 handle as for stdout.
222
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000223 When *stdout* or *stderr* are pipes and *universal_newlines* is
Nick Coghlan65ad31a2011-10-26 21:15:53 +1000224 :const:`True` then all line endings will be converted to ``'\n'`` as
225 described for the universal newlines `'U'`` mode argument to :func:`open`.
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000226
227 If *shell* is :const:`True`, the specified command will be executed through
228 the shell. This can be useful if you are using Python primarily for the
229 enhanced control flow it offers over most system shells and still want
230 access to other shell features such as filename wildcards, shell pipes and
231 environment variable expansion.
232
233 .. warning::
234
235 Executing shell commands that incorporate unsanitized input from an
236 untrusted source makes a program vulnerable to `shell injection
237 <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_,
238 a serious security flaw which can result in arbitrary command execution.
239 For this reason, the use of *shell=True* is **strongly discouraged** in cases
240 where the command string is constructed from external input::
241
242 >>> from subprocess import call
243 >>> filename = input("What file would you like to display?\n")
244 What file would you like to display?
245 non_existent; rm -rf / #
246 >>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
247
248 ``shell=False`` disables all shell based features, but does not suffer
249 from this vulnerability; see the Note in the :class:`Popen` constructor
250 documentation for helpful hints in getting ``shell=False`` to work.
251
Nick Coghlan86711572011-10-24 22:19:40 +1000252These options, along with all of the other options, are described in more
253detail in the :class:`Popen` constructor documentation.
254
255
Sandro Tosidbcbd102011-12-25 11:27:22 +0100256Popen Constructor
Sandro Tosi44585bd2011-12-25 17:13:10 +0100257^^^^^^^^^^^^^^^^^
Nick Coghlan86711572011-10-24 22:19:40 +1000258
259The underlying process creation and management in this module is handled by
260the :class:`Popen` class. It offers a lot of flexibility so that developers
261are able to handle the less common cases not covered by the convenience
262functions.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000263
264
265.. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)
266
267 Arguments are:
268
Benjamin Petersonfff5cf62008-07-27 15:22:14 +0000269 *args* should be a string, or a sequence of program arguments. The program
R. David Murrayfe6e7842009-05-29 19:30:27 +0000270 to execute is normally the first item in the args sequence or the string if
271 a string is given, but can be explicitly set by using the *executable*
272 argument. When *executable* is given, the first item in the args sequence
273 is still treated by most programs as the command name, which can then be
274 different from the actual executable name. On Unix, it becomes the display
275 name for the executing program in utilities such as :program:`ps`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000276
277 On Unix, with *shell=False* (default): In this case, the Popen class uses
278 :meth:`os.execvp` to execute the child program. *args* should normally be a
Nick Coghlan7dfc9e12010-02-04 12:43:58 +0000279 sequence. If a string is specified for *args*, it will be used as the name
280 or path of the program to execute; this will only work if the program is
281 being given no arguments.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000282
Nick Coghlan7dfc9e12010-02-04 12:43:58 +0000283 .. note::
284
285 :meth:`shlex.split` can be useful when determining the correct
286 tokenization for *args*, especially in complex cases::
287
288 >>> import shlex, subprocess
289 >>> command_line = raw_input()
290 /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
291 >>> args = shlex.split(command_line)
292 >>> print args
293 ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
294 >>> p = subprocess.Popen(args) # Success!
295
296 Note in particular that options (such as *-input*) and arguments (such
297 as *eggs.txt*) that are separated by whitespace in the shell go in separate
298 list elements, while arguments that need quoting or backslash escaping when
299 used in the shell (such as filenames containing spaces or the *echo* command
300 shown above) are single list elements.
301
302 On Unix, with *shell=True*: If args is a string, it specifies the command
303 string to execute through the shell. This means that the string must be
304 formatted exactly as it would be when typed at the shell prompt. This
305 includes, for example, quoting or backslash escaping filenames with spaces in
306 them. If *args* is a sequence, the first item specifies the command string, and
307 any additional items will be treated as additional arguments to the shell
308 itself. That is to say, *Popen* does the equivalent of::
309
310 Popen(['/bin/sh', '-c', args[0], args[1], ...])
Georg Brandl8ec7f652007-08-15 14:28:01 +0000311
R David Murrayfc095202012-07-16 22:31:32 -0400312 On Windows: the :class:`Popen` class uses CreateProcess() to execute the
Eli Bendersky929e2762011-04-15 07:35:06 +0300313 child program, which operates on strings. If *args* is a sequence, it will
314 be converted to a string in a manner described in
315 :ref:`converting-argument-sequence`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000316
317 *bufsize*, if given, has the same meaning as the corresponding argument to the
318 built-in open() function: :const:`0` means unbuffered, :const:`1` means line
319 buffered, any other positive value means use a buffer of (approximately) that
320 size. A negative *bufsize* means to use the system default, which usually means
321 fully buffered. The default value for *bufsize* is :const:`0` (unbuffered).
322
Antoine Pitrouc3955452010-06-02 17:08:47 +0000323 .. note::
324
325 If you experience performance issues, it is recommended that you try to
326 enable buffering by setting *bufsize* to either -1 or a large enough
327 positive value (such as 4096).
328
Georg Brandl8ec7f652007-08-15 14:28:01 +0000329 The *executable* argument specifies the program to execute. It is very seldom
330 needed: Usually, the program to execute is defined by the *args* argument. If
331 ``shell=True``, the *executable* argument specifies which shell to use. On Unix,
332 the default shell is :file:`/bin/sh`. On Windows, the default shell is
Georg Brandl0d8649a2009-06-30 16:17:28 +0000333 specified by the :envvar:`COMSPEC` environment variable. The only reason you
334 would need to specify ``shell=True`` on Windows is where the command you
335 wish to execute is actually built in to the shell, eg ``dir``, ``copy``.
336 You don't need ``shell=True`` to run a batch file, nor to run a console-based
337 executable.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000338
Nick Coghlan86711572011-10-24 22:19:40 +1000339 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
Georg Brandlf5d5a662008-12-06 11:57:12 +0000340 standard output and standard error file handles, respectively. Valid values
341 are :data:`PIPE`, an existing file descriptor (a positive integer), an
342 existing file object, and ``None``. :data:`PIPE` indicates that a new pipe
Nick Coghlan86711572011-10-24 22:19:40 +1000343 to the child should be created. With the default settings of ``None``, no
344 redirection will occur; the child's file handles will be inherited from the
345 parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that
346 the stderr data from the child process should be captured into the same file
347 handle as for stdout.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000348
349 If *preexec_fn* is set to a callable object, this object will be called in the
350 child process just before the child is executed. (Unix only)
351
352 If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
353 :const:`2` will be closed before the child process is executed. (Unix only).
354 Or, on Windows, if *close_fds* is true then no handles will be inherited by the
355 child process. Note that on Windows, you cannot set *close_fds* to true and
356 also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.
357
358 If *shell* is :const:`True`, the specified command will be executed through the
359 shell.
360
Nick Coghlan87ba6422011-10-27 17:55:13 +1000361 .. warning::
Nick Coghlan65ad31a2011-10-26 21:15:53 +1000362
363 Enabling this option can be a security hazard if combined with untrusted
364 input. See the warning under :ref:`frequently-used-arguments`
365 for details.
366
Georg Brandl8ec7f652007-08-15 14:28:01 +0000367 If *cwd* is not ``None``, the child's current directory will be changed to *cwd*
368 before it is executed. Note that this directory is not considered when
369 searching the executable, so you can't specify the program's path relative to
370 *cwd*.
371
Georg Brandlf801b0f2008-04-19 16:58:49 +0000372 If *env* is not ``None``, it must be a mapping that defines the environment
373 variables for the new process; these are used instead of inheriting the current
374 process' environment, which is the default behavior.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000375
R. David Murray72030812009-04-16 18:12:53 +0000376 .. note::
R. David Murray6076d392009-04-15 22:33:07 +0000377
R. David Murray72030812009-04-16 18:12:53 +0000378 If specified, *env* must provide any variables required
379 for the program to execute. On Windows, in order to run a
380 `side-by-side assembly`_ the specified *env* **must** include a valid
R. David Murray6076d392009-04-15 22:33:07 +0000381 :envvar:`SystemRoot`.
382
R. David Murray72030812009-04-16 18:12:53 +0000383 .. _side-by-side assembly: http://en.wikipedia.org/wiki/Side-by-Side_Assembly
384
Georg Brandl8ec7f652007-08-15 14:28:01 +0000385 If *universal_newlines* is :const:`True`, the file objects stdout and stderr are
386 opened as text files, but lines may be terminated by any of ``'\n'``, the Unix
Georg Brandl9af94982008-09-13 17:41:16 +0000387 end-of-line convention, ``'\r'``, the old Macintosh convention or ``'\r\n'``, the
Georg Brandl8ec7f652007-08-15 14:28:01 +0000388 Windows convention. All of these external representations are seen as ``'\n'``
389 by the Python program.
390
391 .. note::
392
Georg Brandl6ab5d082009-12-20 14:33:20 +0000393 This feature is only available if Python is built with universal newline
394 support (the default). Also, the newlines attribute of the file objects
395 :attr:`stdout`, :attr:`stdin` and :attr:`stderr` are not updated by the
396 communicate() method.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000397
Brian Curtinbb23bd62011-04-29 22:23:46 -0500398 If given, *startupinfo* will be a :class:`STARTUPINFO` object, which is
399 passed to the underlying ``CreateProcess`` function.
400 *creationflags*, if given, can be :data:`CREATE_NEW_CONSOLE` or
401 :data:`CREATE_NEW_PROCESS_GROUP`. (Windows only)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000402
403
Georg Brandl8ec7f652007-08-15 14:28:01 +0000404Exceptions
405^^^^^^^^^^
406
407Exceptions raised in the child process, before the new program has started to
408execute, will be re-raised in the parent. Additionally, the exception object
409will have one extra attribute called :attr:`child_traceback`, which is a string
Georg Brandl21946af2010-10-06 09:28:45 +0000410containing traceback information from the child's point of view.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000411
412The most common exception raised is :exc:`OSError`. This occurs, for example,
413when trying to execute a non-existent file. Applications should prepare for
414:exc:`OSError` exceptions.
415
416A :exc:`ValueError` will be raised if :class:`Popen` is called with invalid
417arguments.
418
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000419:func:`check_call` and :func:`check_output` will raise
420:exc:`CalledProcessError` if the called process returns a non-zero return
421code.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000422
423
424Security
425^^^^^^^^
426
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000427Unlike some other popen functions, this implementation will never call a
428system shell implicitly. This means that all characters, including shell
429metacharacters, can safely be passed to child processes. Obviously, if the
430shell is invoked explicitly, then it is the application's responsibility to
Nick Coghlan63c54e82011-10-26 21:34:26 +1000431ensure that all whitespace and metacharacters are quoted appropriately.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000432
433
434Popen Objects
435-------------
436
437Instances of the :class:`Popen` class have the following methods:
438
439
440.. method:: Popen.poll()
441
Georg Brandl2cb103f2008-01-06 16:01:26 +0000442 Check if child process has terminated. Set and return :attr:`returncode`
443 attribute.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000444
445
446.. method:: Popen.wait()
447
Georg Brandl2cb103f2008-01-06 16:01:26 +0000448 Wait for child process to terminate. Set and return :attr:`returncode`
449 attribute.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000450
Georg Brandl143de622008-08-04 06:29:36 +0000451 .. warning::
452
Philip Jenvey26275532009-12-03 02:25:54 +0000453 This will deadlock when using ``stdout=PIPE`` and/or
454 ``stderr=PIPE`` and the child process generates enough output to
455 a pipe such that it blocks waiting for the OS pipe buffer to
456 accept more data. Use :meth:`communicate` to avoid that.
Gregory P. Smith08792502008-08-04 01:03:50 +0000457
Georg Brandl8ec7f652007-08-15 14:28:01 +0000458
459.. method:: Popen.communicate(input=None)
460
461 Interact with process: Send data to stdin. Read data from stdout and stderr,
462 until end-of-file is reached. Wait for process to terminate. The optional
463 *input* argument should be a string to be sent to the child process, or
464 ``None``, if no data should be sent to the child.
465
Georg Brandl17432012008-12-04 21:28:16 +0000466 :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000467
Georg Brandl439f2502007-11-24 11:31:46 +0000468 Note that if you want to send data to the process's stdin, you need to create
469 the Popen object with ``stdin=PIPE``. Similarly, to get anything other than
470 ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or
471 ``stderr=PIPE`` too.
472
Georg Brandl2cb103f2008-01-06 16:01:26 +0000473 .. note::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000474
Georg Brandl2cb103f2008-01-06 16:01:26 +0000475 The data read is buffered in memory, so do not use this method if the data
476 size is large or unlimited.
477
Georg Brandl8ec7f652007-08-15 14:28:01 +0000478
Christian Heimese74c8f22008-04-19 02:23:57 +0000479.. method:: Popen.send_signal(signal)
480
481 Sends the signal *signal* to the child.
482
483 .. note::
484
Brian Curtine5aa8862010-04-02 23:26:06 +0000485 On Windows, SIGTERM is an alias for :meth:`terminate`. CTRL_C_EVENT and
Ezio Melotti9ccc5812010-04-05 08:16:41 +0000486 CTRL_BREAK_EVENT can be sent to processes started with a *creationflags*
Brian Curtine5aa8862010-04-02 23:26:06 +0000487 parameter which includes `CREATE_NEW_PROCESS_GROUP`.
Georg Brandl734de682008-04-19 08:23:59 +0000488
489 .. versionadded:: 2.6
Christian Heimese74c8f22008-04-19 02:23:57 +0000490
491
492.. method:: Popen.terminate()
493
494 Stop the child. On Posix OSs the method sends SIGTERM to the
Sandro Tosi98ed08f2012-01-14 16:42:02 +0100495 child. On Windows the Win32 API function :c:func:`TerminateProcess` is called
Christian Heimese74c8f22008-04-19 02:23:57 +0000496 to stop the child.
497
Georg Brandl734de682008-04-19 08:23:59 +0000498 .. versionadded:: 2.6
499
Christian Heimese74c8f22008-04-19 02:23:57 +0000500
501.. method:: Popen.kill()
502
503 Kills the child. On Posix OSs the function sends SIGKILL to the child.
Georg Brandl734de682008-04-19 08:23:59 +0000504 On Windows :meth:`kill` is an alias for :meth:`terminate`.
505
506 .. versionadded:: 2.6
Christian Heimese74c8f22008-04-19 02:23:57 +0000507
508
Georg Brandl8ec7f652007-08-15 14:28:01 +0000509The following attributes are also available:
510
Georg Brandl143de622008-08-04 06:29:36 +0000511.. warning::
512
Georg Brandl16a57f62009-04-27 15:29:09 +0000513 Use :meth:`communicate` rather than :attr:`.stdin.write <stdin>`,
514 :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` to avoid
515 deadlocks due to any of the other OS pipe buffers filling up and blocking the
516 child process.
Georg Brandl143de622008-08-04 06:29:36 +0000517
518
Georg Brandl8ec7f652007-08-15 14:28:01 +0000519.. attribute:: Popen.stdin
520
Georg Brandlf5d5a662008-12-06 11:57:12 +0000521 If the *stdin* argument was :data:`PIPE`, this attribute is a file object
522 that provides input to the child process. Otherwise, it is ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000523
524
525.. attribute:: Popen.stdout
526
Georg Brandlf5d5a662008-12-06 11:57:12 +0000527 If the *stdout* argument was :data:`PIPE`, this attribute is a file object
528 that provides output from the child process. Otherwise, it is ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000529
530
531.. attribute:: Popen.stderr
532
Georg Brandlf5d5a662008-12-06 11:57:12 +0000533 If the *stderr* argument was :data:`PIPE`, this attribute is a file object
534 that provides error output from the child process. Otherwise, it is
535 ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000536
537
538.. attribute:: Popen.pid
539
540 The process ID of the child process.
541
Georg Brandl0b56ce02010-03-21 09:28:16 +0000542 Note that if you set the *shell* argument to ``True``, this is the process ID
543 of the spawned shell.
544
Georg Brandl8ec7f652007-08-15 14:28:01 +0000545
546.. attribute:: Popen.returncode
547
Georg Brandl2cb103f2008-01-06 16:01:26 +0000548 The child return code, set by :meth:`poll` and :meth:`wait` (and indirectly
549 by :meth:`communicate`). A ``None`` value indicates that the process
550 hasn't terminated yet.
Georg Brandlc62ef8b2009-01-03 20:55:06 +0000551
Georg Brandl2cb103f2008-01-06 16:01:26 +0000552 A negative value ``-N`` indicates that the child was terminated by signal
553 ``N`` (Unix only).
Georg Brandl8ec7f652007-08-15 14:28:01 +0000554
555
Brian Curtinbb23bd62011-04-29 22:23:46 -0500556Windows Popen Helpers
557---------------------
558
559The :class:`STARTUPINFO` class and following constants are only available
560on Windows.
561
562.. class:: STARTUPINFO()
563
564 Partial support of the Windows
565 `STARTUPINFO <http://msdn.microsoft.com/en-us/library/ms686331(v=vs.85).aspx>`__
566 structure is used for :class:`Popen` creation.
567
568 .. attribute:: dwFlags
569
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700570 A bit field that determines whether certain :class:`STARTUPINFO`
571 attributes are used when the process creates a window. ::
Brian Curtinbb23bd62011-04-29 22:23:46 -0500572
573 si = subprocess.STARTUPINFO()
574 si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW
575
576 .. attribute:: hStdInput
577
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700578 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
579 is the standard input handle for the process. If
580 :data:`STARTF_USESTDHANDLES` is not specified, the default for standard
581 input is the keyboard buffer.
Brian Curtinbb23bd62011-04-29 22:23:46 -0500582
583 .. attribute:: hStdOutput
584
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700585 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
586 is the standard output handle for the process. Otherwise, this attribute
587 is ignored and the default for standard output is the console window's
Brian Curtinbb23bd62011-04-29 22:23:46 -0500588 buffer.
589
590 .. attribute:: hStdError
591
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700592 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
593 is the standard error handle for the process. Otherwise, this attribute is
Brian Curtinbb23bd62011-04-29 22:23:46 -0500594 ignored and the default for standard error is the console window's buffer.
595
596 .. attribute:: wShowWindow
597
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700598 If :attr:`dwFlags` specifies :data:`STARTF_USESHOWWINDOW`, this attribute
Brian Curtinbb23bd62011-04-29 22:23:46 -0500599 can be any of the values that can be specified in the ``nCmdShow``
600 parameter for the
601 `ShowWindow <http://msdn.microsoft.com/en-us/library/ms633548(v=vs.85).aspx>`__
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700602 function, except for ``SW_SHOWDEFAULT``. Otherwise, this attribute is
Brian Curtinbb23bd62011-04-29 22:23:46 -0500603 ignored.
604
605 :data:`SW_HIDE` is provided for this attribute. It is used when
606 :class:`Popen` is called with ``shell=True``.
607
608
609Constants
610^^^^^^^^^
611
612The :mod:`subprocess` module exposes the following constants.
613
614.. data:: STD_INPUT_HANDLE
615
616 The standard input device. Initially, this is the console input buffer,
617 ``CONIN$``.
618
619.. data:: STD_OUTPUT_HANDLE
620
621 The standard output device. Initially, this is the active console screen
622 buffer, ``CONOUT$``.
623
624.. data:: STD_ERROR_HANDLE
625
626 The standard error device. Initially, this is the active console screen
627 buffer, ``CONOUT$``.
628
629.. data:: SW_HIDE
630
631 Hides the window. Another window will be activated.
632
633.. data:: STARTF_USESTDHANDLES
634
635 Specifies that the :attr:`STARTUPINFO.hStdInput`,
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700636 :attr:`STARTUPINFO.hStdOutput`, and :attr:`STARTUPINFO.hStdError` attributes
Brian Curtinbb23bd62011-04-29 22:23:46 -0500637 contain additional information.
638
639.. data:: STARTF_USESHOWWINDOW
640
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700641 Specifies that the :attr:`STARTUPINFO.wShowWindow` attribute contains
Brian Curtinbb23bd62011-04-29 22:23:46 -0500642 additional information.
643
644.. data:: CREATE_NEW_CONSOLE
645
646 The new process has a new console, instead of inheriting its parent's
647 console (the default).
648
649 This flag is always set when :class:`Popen` is created with ``shell=True``.
650
651.. data:: CREATE_NEW_PROCESS_GROUP
652
653 A :class:`Popen` ``creationflags`` parameter to specify that a new process
654 group will be created. This flag is necessary for using :func:`os.kill`
655 on the subprocess.
656
657 This flag is ignored if :data:`CREATE_NEW_CONSOLE` is specified.
658
659
Georg Brandl0ba92b22008-06-22 09:05:29 +0000660.. _subprocess-replacements:
661
Georg Brandl8ec7f652007-08-15 14:28:01 +0000662Replacing Older Functions with the subprocess Module
663----------------------------------------------------
664
Nick Coghlan86711572011-10-24 22:19:40 +1000665In this section, "a becomes b" means that b can be used as a replacement for a.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000666
667.. note::
668
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000669 All "a" functions in this section fail (more or less) silently if the
670 executed program cannot be found; the "b" replacements raise :exc:`OSError`
671 instead.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000672
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000673 In addition, the replacements using :func:`check_output` will fail with a
674 :exc:`CalledProcessError` if the requested operation produces a non-zero
675 return code. The output is still available as the ``output`` attribute of
676 the raised exception.
677
678In the following examples, we assume that the relevant functions have already
679been imported from the subprocess module.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000680
681
682Replacing /bin/sh shell backquote
683^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
684
685::
686
687 output=`mycmd myarg`
Nick Coghlan86711572011-10-24 22:19:40 +1000688 # becomes
689 output = check_output(["mycmd", "myarg"])
Georg Brandl8ec7f652007-08-15 14:28:01 +0000690
691
Benjamin Petersoncae58482008-10-10 20:38:49 +0000692Replacing shell pipeline
693^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000694
695::
696
697 output=`dmesg | grep hda`
Nick Coghlan86711572011-10-24 22:19:40 +1000698 # becomes
Georg Brandl8ec7f652007-08-15 14:28:01 +0000699 p1 = Popen(["dmesg"], stdout=PIPE)
700 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Gregory P. Smithe3e967f2011-02-05 21:49:56 +0000701 p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000702 output = p2.communicate()[0]
703
Gregory P. Smithe3e967f2011-02-05 21:49:56 +0000704The p1.stdout.close() call after starting the p2 is important in order for p1
705to receive a SIGPIPE if p2 exits before p1.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000706
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000707Alternatively, for trusted input, the shell's own pipeline support may still
R David Murray5fc56eb2012-04-03 08:46:05 -0400708be used directly::
Nick Coghlan86711572011-10-24 22:19:40 +1000709
710 output=`dmesg | grep hda`
711 # becomes
712 output=check_output("dmesg | grep hda", shell=True)
713
714
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000715Replacing :func:`os.system`
716^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000717
718::
719
720 sts = os.system("mycmd" + " myarg")
Nick Coghlan86711572011-10-24 22:19:40 +1000721 # becomes
722 sts = call("mycmd" + " myarg", shell=True)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000723
724Notes:
725
726* Calling the program through the shell is usually not required.
727
Georg Brandl8ec7f652007-08-15 14:28:01 +0000728A more realistic example would look like this::
729
730 try:
731 retcode = call("mycmd" + " myarg", shell=True)
732 if retcode < 0:
733 print >>sys.stderr, "Child was terminated by signal", -retcode
734 else:
735 print >>sys.stderr, "Child returned", retcode
736 except OSError, e:
737 print >>sys.stderr, "Execution failed:", e
738
739
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000740Replacing the :func:`os.spawn <os.spawnl>` family
741^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000742
743P_NOWAIT example::
744
745 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
746 ==>
747 pid = Popen(["/bin/mycmd", "myarg"]).pid
748
749P_WAIT example::
750
751 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
752 ==>
753 retcode = call(["/bin/mycmd", "myarg"])
754
755Vector example::
756
757 os.spawnvp(os.P_NOWAIT, path, args)
758 ==>
759 Popen([path] + args[1:])
760
761Environment example::
762
763 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
764 ==>
765 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
766
767
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000768Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3`
769^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000770
771::
772
Philip Jenvey8b902042009-09-29 19:10:15 +0000773 pipe = os.popen("cmd", 'r', bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000774 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000775 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout
Georg Brandl8ec7f652007-08-15 14:28:01 +0000776
777::
778
Philip Jenvey8b902042009-09-29 19:10:15 +0000779 pipe = os.popen("cmd", 'w', bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000780 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000781 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin
Georg Brandl8ec7f652007-08-15 14:28:01 +0000782
783::
784
Philip Jenvey8b902042009-09-29 19:10:15 +0000785 (child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000786 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000787 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000788 stdin=PIPE, stdout=PIPE, close_fds=True)
789 (child_stdin, child_stdout) = (p.stdin, p.stdout)
790
791::
792
793 (child_stdin,
794 child_stdout,
Philip Jenvey8b902042009-09-29 19:10:15 +0000795 child_stderr) = os.popen3("cmd", mode, bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000796 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000797 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000798 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
799 (child_stdin,
800 child_stdout,
801 child_stderr) = (p.stdin, p.stdout, p.stderr)
802
803::
804
Philip Jenvey8b902042009-09-29 19:10:15 +0000805 (child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode,
806 bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000807 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000808 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000809 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
810 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
811
Philip Jenvey8b902042009-09-29 19:10:15 +0000812On Unix, os.popen2, os.popen3 and os.popen4 also accept a sequence as
813the command to execute, in which case arguments will be passed
814directly to the program without shell intervention. This usage can be
815replaced as follows::
816
817 (child_stdin, child_stdout) = os.popen2(["/bin/ls", "-l"], mode,
818 bufsize)
819 ==>
820 p = Popen(["/bin/ls", "-l"], bufsize=bufsize, stdin=PIPE, stdout=PIPE)
821 (child_stdin, child_stdout) = (p.stdin, p.stdout)
822
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000823Return code handling translates as follows::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000824
Philip Jenvey8b902042009-09-29 19:10:15 +0000825 pipe = os.popen("cmd", 'w')
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000826 ...
827 rc = pipe.close()
Stefan Kraha253dc12010-07-14 10:06:07 +0000828 if rc is not None and rc >> 8:
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000829 print "There were some errors"
830 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000831 process = Popen("cmd", 'w', shell=True, stdin=PIPE)
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000832 ...
833 process.stdin.close()
834 if process.wait() != 0:
835 print "There were some errors"
836
837
838Replacing functions from the :mod:`popen2` module
839^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000840
Georg Brandl8ec7f652007-08-15 14:28:01 +0000841::
842
843 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
844 ==>
845 p = Popen(["somestring"], shell=True, bufsize=bufsize,
846 stdin=PIPE, stdout=PIPE, close_fds=True)
847 (child_stdout, child_stdin) = (p.stdout, p.stdin)
848
Philip Jenvey8b902042009-09-29 19:10:15 +0000849On Unix, popen2 also accepts a sequence as the command to execute, in
850which case arguments will be passed directly to the program without
851shell intervention. This usage can be replaced as follows::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000852
Philip Jenvey8b902042009-09-29 19:10:15 +0000853 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize,
854 mode)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000855 ==>
856 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
857 stdin=PIPE, stdout=PIPE, close_fds=True)
858 (child_stdout, child_stdin) = (p.stdout, p.stdin)
859
Georg Brandlf5d5a662008-12-06 11:57:12 +0000860:class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as
861:class:`subprocess.Popen`, except that:
Georg Brandl8ec7f652007-08-15 14:28:01 +0000862
Georg Brandlf5d5a662008-12-06 11:57:12 +0000863* :class:`Popen` raises an exception if the execution fails.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000864
865* the *capturestderr* argument is replaced with the *stderr* argument.
866
Georg Brandlf5d5a662008-12-06 11:57:12 +0000867* ``stdin=PIPE`` and ``stdout=PIPE`` must be specified.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000868
869* popen2 closes all file descriptors by default, but you have to specify
Georg Brandlf5d5a662008-12-06 11:57:12 +0000870 ``close_fds=True`` with :class:`Popen`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000871
Nick Coghlan2ed203a2011-10-26 21:05:56 +1000872
Eli Bendersky929e2762011-04-15 07:35:06 +0300873Notes
874-----
875
876.. _converting-argument-sequence:
877
878Converting an argument sequence to a string on Windows
879^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
880
881On Windows, an *args* sequence is converted to a string that can be parsed
882using the following rules (which correspond to the rules used by the MS C
883runtime):
884
8851. Arguments are delimited by white space, which is either a
886 space or a tab.
887
8882. A string surrounded by double quotation marks is
889 interpreted as a single argument, regardless of white space
890 contained within. A quoted string can be embedded in an
891 argument.
892
8933. A double quotation mark preceded by a backslash is
894 interpreted as a literal double quotation mark.
895
8964. Backslashes are interpreted literally, unless they
897 immediately precede a double quotation mark.
898
8995. If backslashes immediately precede a double quotation mark,
900 every pair of backslashes is interpreted as a literal
901 backslash. If the number of backslashes is odd, the last
902 backslash escapes the next double quotation mark as
903 described in rule 3.
904