blob: 88eb7846a3e2ef25b0db22f7143f4681e1760bc5 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001:mod:`subprocess` --- Subprocess management
2===========================================
3
4.. module:: subprocess
5 :synopsis: Subprocess management.
6.. moduleauthor:: Peter Åstrand <astrand@lysator.liu.se>
7.. sectionauthor:: Peter Åstrand <astrand@lysator.liu.se>
8
9
Georg Brandl116aa622007-08-15 14:28:22 +000010The :mod:`subprocess` module allows you to spawn new processes, connect to their
11input/output/error pipes, and obtain their return codes. This module intends to
12replace several other, older modules and functions, such as::
13
14 os.system
15 os.spawn*
Georg Brandl116aa622007-08-15 14:28:22 +000016
17Information about how the :mod:`subprocess` module can be used to replace these
18modules and functions can be found in the following sections.
19
Benjamin Peterson41181742008-07-02 20:22:54 +000020.. seealso::
21
22 :pep:`324` -- PEP proposing the subprocess module
23
Georg Brandl116aa622007-08-15 14:28:22 +000024
25Using the subprocess Module
26---------------------------
27
28This module defines one class called :class:`Popen`:
29
30
Gregory P. Smith8edd99d2010-12-14 13:43:30 +000031.. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=())
Georg Brandl116aa622007-08-15 14:28:22 +000032
33 Arguments are:
34
Benjamin Petersond18de0e2008-07-31 20:21:46 +000035 *args* should be a string, or a sequence of program arguments. The program
Benjamin Petersonfa0d7032009-06-01 22:42:33 +000036 to execute is normally the first item in the args sequence or the string if
37 a string is given, but can be explicitly set by using the *executable*
38 argument. When *executable* is given, the first item in the args sequence
39 is still treated by most programs as the command name, which can then be
40 different from the actual executable name. On Unix, it becomes the display
41 name for the executing program in utilities such as :program:`ps`.
Georg Brandl116aa622007-08-15 14:28:22 +000042
43 On Unix, with *shell=False* (default): In this case, the Popen class uses
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +000044 :meth:`os.execvp` like behavior to execute the child program.
45 *args* should normally be a
R. David Murray5973e4d2010-02-04 16:41:57 +000046 sequence. If a string is specified for *args*, it will be used as the name
47 or path of the program to execute; this will only work if the program is
48 being given no arguments.
Georg Brandl116aa622007-08-15 14:28:22 +000049
R. David Murray5973e4d2010-02-04 16:41:57 +000050 .. note::
51
52 :meth:`shlex.split` can be useful when determining the correct
53 tokenization for *args*, especially in complex cases::
54
55 >>> import shlex, subprocess
R. David Murray73bc75b2010-02-05 16:25:12 +000056 >>> command_line = input()
R. David Murray5973e4d2010-02-04 16:41:57 +000057 /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
58 >>> args = shlex.split(command_line)
59 >>> print(args)
60 ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
61 >>> p = subprocess.Popen(args) # Success!
62
63 Note in particular that options (such as *-input*) and arguments (such
64 as *eggs.txt*) that are separated by whitespace in the shell go in separate
65 list elements, while arguments that need quoting or backslash escaping when
66 used in the shell (such as filenames containing spaces or the *echo* command
67 shown above) are single list elements.
68
69 On Unix, with *shell=True*: If args is a string, it specifies the command
70 string to execute through the shell. This means that the string must be
71 formatted exactly as it would be when typed at the shell prompt. This
72 includes, for example, quoting or backslash escaping filenames with spaces in
73 them. If *args* is a sequence, the first item specifies the command string, and
74 any additional items will be treated as additional arguments to the shell
75 itself. That is to say, *Popen* does the equivalent of::
76
77 Popen(['/bin/sh', '-c', args[0], args[1], ...])
Georg Brandl116aa622007-08-15 14:28:22 +000078
R. David Murrayc7399d02010-11-12 00:35:31 +000079 .. warning::
80
81 Executing shell commands that incorporate unsanitized input from an
82 untrusted source makes a program vulnerable to `shell injection
83 <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_,
84 a serious security flaw which can result in arbitrary command execution.
85 For this reason, the use of *shell=True* is **strongly discouraged** in cases
86 where the command string is constructed from external input::
87
88 >>> from subprocess import call
89 >>> filename = input("What file would you like to display?\n")
90 What file would you like to display?
91 non_existent; rm -rf / #
92 >>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
93
94 *shell=False* does not suffer from this vulnerability; the above Note may be
95 helpful in getting code using *shell=False* to work.
96
Georg Brandl116aa622007-08-15 14:28:22 +000097 On Windows: the :class:`Popen` class uses CreateProcess() to execute the child
98 program, which operates on strings. If *args* is a sequence, it will be
99 converted to a string using the :meth:`list2cmdline` method. Please note that
100 not all MS Windows applications interpret the command line the same way:
101 :meth:`list2cmdline` is designed for applications using the same rules as the MS
102 C runtime.
103
104 *bufsize*, if given, has the same meaning as the corresponding argument to the
105 built-in open() function: :const:`0` means unbuffered, :const:`1` means line
106 buffered, any other positive value means use a buffer of (approximately) that
107 size. A negative *bufsize* means to use the system default, which usually means
108 fully buffered. The default value for *bufsize* is :const:`0` (unbuffered).
109
Antoine Pitrou4b876202010-06-02 17:10:49 +0000110 .. note::
111
112 If you experience performance issues, it is recommended that you try to
113 enable buffering by setting *bufsize* to either -1 or a large enough
114 positive value (such as 4096).
115
Georg Brandl116aa622007-08-15 14:28:22 +0000116 The *executable* argument specifies the program to execute. It is very seldom
117 needed: Usually, the program to execute is defined by the *args* argument. If
118 ``shell=True``, the *executable* argument specifies which shell to use. On Unix,
119 the default shell is :file:`/bin/sh`. On Windows, the default shell is
Alexandre Vassalotti260484d2009-07-17 11:43:26 +0000120 specified by the :envvar:`COMSPEC` environment variable. The only reason you
121 would need to specify ``shell=True`` on Windows is where the command you
122 wish to execute is actually built in to the shell, eg ``dir``, ``copy``.
123 You don't need ``shell=True`` to run a batch file, nor to run a console-based
124 executable.
Georg Brandl116aa622007-08-15 14:28:22 +0000125
126 *stdin*, *stdout* and *stderr* specify the executed programs' standard input,
Georg Brandlaf265f42008-12-07 15:06:20 +0000127 standard output and standard error file handles, respectively. Valid values
Ross Lagerwallba102ec2011-03-16 18:40:25 +0200128 are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive
129 integer), an existing :term:`file object`, and ``None``. :data:`PIPE`
130 indicates that a new pipe to the child should be created. :data:`DEVNULL`
131 indicates that the special file :data:`os.devnull` will be used. With ``None``,
132 no redirection will occur; the child's file handles will be inherited from
133 the parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates
134 that the stderr data from the applications should be captured into the same
135 file handle as for stdout.
Georg Brandl116aa622007-08-15 14:28:22 +0000136
137 If *preexec_fn* is set to a callable object, this object will be called in the
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000138 child process just before the child is executed.
139 (Unix only)
140
141 .. warning::
142
143 The *preexec_fn* parameter is not safe to use in the presence of threads
144 in your application. The child process could deadlock before exec is
145 called.
146 If you must use it, keep it trivial! Minimize the number of libraries
147 you call into.
148
149 .. note::
150
151 If you need to modify the environment for the child use the *env*
152 parameter rather than doing it in a *preexec_fn*.
153 The *start_new_session* parameter can take the place of a previously
154 common use of *preexec_fn* to call os.setsid() in the child.
Georg Brandl116aa622007-08-15 14:28:22 +0000155
156 If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
157 :const:`2` will be closed before the child process is executed. (Unix only).
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000158 The default varies by platform: Always true on Unix. On Windows it is
159 true when *stdin*/*stdout*/*stderr* are :const:`None`, false otherwise.
Gregory P. Smithd23047b2010-12-04 09:10:44 +0000160 On Windows, if *close_fds* is true then no handles will be inherited by the
Georg Brandl116aa622007-08-15 14:28:22 +0000161 child process. Note that on Windows, you cannot set *close_fds* to true and
162 also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.
163
Gregory P. Smith8edd99d2010-12-14 13:43:30 +0000164 .. versionchanged:: 3.2
165 The default for *close_fds* was changed from :const:`False` to
166 what is described above.
167
168 *pass_fds* is an optional sequence of file descriptors to keep open
169 between the parent and child. Providing any *pass_fds* forces
170 *close_fds* to be :const:`True`. (Unix only)
171
172 .. versionadded:: 3.2
173 The *pass_fds* parameter was added.
Georg Brandl116aa622007-08-15 14:28:22 +0000174
175 If *cwd* is not ``None``, the child's current directory will be changed to *cwd*
176 before it is executed. Note that this directory is not considered when
177 searching the executable, so you can't specify the program's path relative to
178 *cwd*.
179
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000180 If *restore_signals* is True (the default) all signals that Python has set to
181 SIG_IGN are restored to SIG_DFL in the child process before the exec.
182 Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals.
183 (Unix only)
184
185 .. versionchanged:: 3.2
186 *restore_signals* was added.
187
188 If *start_new_session* is True the setsid() system call will be made in the
189 child process prior to the execution of the subprocess. (Unix only)
190
191 .. versionchanged:: 3.2
192 *start_new_session* was added.
193
Christian Heimesa342c012008-04-20 21:01:16 +0000194 If *env* is not ``None``, it must be a mapping that defines the environment
Gregory P. Smithfb94c5f2010-03-14 06:49:55 +0000195 variables for the new process; these are used instead of the default
196 behavior of inheriting the current process' environment.
Georg Brandl116aa622007-08-15 14:28:22 +0000197
R. David Murray1055e892009-04-16 18:15:32 +0000198 .. note::
R. David Murrayf4ac1492009-04-15 22:35:15 +0000199
Georg Brandl2708f3a2009-12-20 14:38:23 +0000200 If specified, *env* must provide any variables required for the program to
201 execute. On Windows, in order to run a `side-by-side assembly`_ the
202 specified *env* **must** include a valid :envvar:`SystemRoot`.
R. David Murrayf4ac1492009-04-15 22:35:15 +0000203
R. David Murray1055e892009-04-16 18:15:32 +0000204 .. _side-by-side assembly: http://en.wikipedia.org/wiki/Side-by-Side_Assembly
205
Georg Brandl116aa622007-08-15 14:28:22 +0000206 If *universal_newlines* is :const:`True`, the file objects stdout and stderr are
207 opened as text files, but lines may be terminated by any of ``'\n'``, the Unix
Georg Brandlc575c902008-09-13 17:46:05 +0000208 end-of-line convention, ``'\r'``, the old Macintosh convention or ``'\r\n'``, the
Georg Brandl116aa622007-08-15 14:28:22 +0000209 Windows convention. All of these external representations are seen as ``'\n'``
210 by the Python program.
211
212 .. note::
213
Georg Brandl7f01a132009-09-16 15:58:14 +0000214 This feature is only available if Python is built with universal newline
215 support (the default). Also, the newlines attribute of the file objects
216 :attr:`stdout`, :attr:`stdin` and :attr:`stderr` are not updated by the
217 :meth:`communicate` method.
Georg Brandl116aa622007-08-15 14:28:22 +0000218
219 The *startupinfo* and *creationflags*, if given, will be passed to the
220 underlying CreateProcess() function. They can specify things such as appearance
221 of the main window and priority for the new process. (Windows only)
222
Brian Curtin79cdb662010-12-03 02:46:02 +0000223 Popen objects are supported as context managers via the :keyword:`with` statement,
224 closing any open file descriptors on exit.
225 ::
226
227 with Popen(["ifconfig"], stdout=PIPE) as proc:
228 log.write(proc.stdout.read())
229
230 .. versionchanged:: 3.2
231 Added context manager support.
232
Georg Brandl116aa622007-08-15 14:28:22 +0000233
Ross Lagerwallba102ec2011-03-16 18:40:25 +0200234.. data:: DEVNULL
235
236 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
237 to :class:`Popen` and indicates that the special file :data:`os.devnull`
238 will be used.
239
240 .. versionadded:: 3.3
241
242
Georg Brandlaf265f42008-12-07 15:06:20 +0000243.. data:: PIPE
244
245 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
246 to :class:`Popen` and indicates that a pipe to the standard stream should be
247 opened.
248
249
250.. data:: STDOUT
251
252 Special value that can be used as the *stderr* argument to :class:`Popen` and
253 indicates that standard error should go into the same handle as standard
254 output.
Georg Brandl48310cd2009-01-03 21:18:54 +0000255
Georg Brandlaf265f42008-12-07 15:06:20 +0000256
Georg Brandl116aa622007-08-15 14:28:22 +0000257Convenience Functions
258^^^^^^^^^^^^^^^^^^^^^
259
Brett Cannona23810f2008-05-26 19:04:21 +0000260This module also defines four shortcut functions:
Georg Brandl116aa622007-08-15 14:28:22 +0000261
262
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400263.. function:: call(*popenargs, timeout=None, **kwargs)
Georg Brandl116aa622007-08-15 14:28:22 +0000264
265 Run command with arguments. Wait for command to complete, then return the
266 :attr:`returncode` attribute.
267
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400268 The arguments are the same as for the :class:`Popen` constructor, with the
269 exception of the *timeout* argument, which is given to :meth:`Popen.wait`.
270 Example::
Georg Brandl116aa622007-08-15 14:28:22 +0000271
Georg Brandl2708f3a2009-12-20 14:38:23 +0000272 >>> retcode = subprocess.call(["ls", "-l"])
Georg Brandl116aa622007-08-15 14:28:22 +0000273
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400274 If the timeout expires, the child process will be killed and then waited for
275 again. The :exc:`TimeoutExpired` exception will be re-raised after the child
276 process has terminated.
277
Philip Jenveyab7481a2009-05-22 05:46:35 +0000278 .. warning::
279
Philip Jenveyb0896842009-12-03 02:29:36 +0000280 Like :meth:`Popen.wait`, this will deadlock when using
281 ``stdout=PIPE`` and/or ``stderr=PIPE`` and the child process
282 generates enough output to a pipe such that it blocks waiting
283 for the OS pipe buffer to accept more data.
Philip Jenveyab7481a2009-05-22 05:46:35 +0000284
Reid Kleckner28f13032011-03-14 12:36:53 -0400285 .. versionchanged:: 3.3
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400286 *timeout* was added.
Georg Brandl116aa622007-08-15 14:28:22 +0000287
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400288
289.. function:: check_call(*popenargs, timeout=None, **kwargs)
Georg Brandl116aa622007-08-15 14:28:22 +0000290
291 Run command with arguments. Wait for command to complete. If the exit code was
Benjamin Petersone5384b02008-10-04 22:00:42 +0000292 zero then return, otherwise raise :exc:`CalledProcessError`. The
Georg Brandl116aa622007-08-15 14:28:22 +0000293 :exc:`CalledProcessError` object will have the return code in the
294 :attr:`returncode` attribute.
295
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400296 The arguments are the same as for the :func:`call` function. Example::
Georg Brandl116aa622007-08-15 14:28:22 +0000297
Georg Brandl2708f3a2009-12-20 14:38:23 +0000298 >>> subprocess.check_call(["ls", "-l"])
299 0
Georg Brandl116aa622007-08-15 14:28:22 +0000300
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400301 As in the :func:`call` function, if the timeout expires, the child process
302 will be killed and the wait retried. The :exc:`TimeoutExpired` exception
303 will be re-raised after the child process has terminated.
304
Philip Jenveyab7481a2009-05-22 05:46:35 +0000305 .. warning::
306
307 See the warning for :func:`call`.
308
Reid Kleckner28f13032011-03-14 12:36:53 -0400309 .. versionchanged:: 3.3
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400310 *timeout* was added.
Georg Brandl116aa622007-08-15 14:28:22 +0000311
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400312
313.. function:: check_output(*popenargs, timeout=None, **kwargs)
Georg Brandlf9734072008-12-07 15:30:06 +0000314
315 Run command with arguments and return its output as a byte string.
316
Benjamin Petersonaa069002009-01-23 03:26:36 +0000317 If the exit code was non-zero it raises a :exc:`CalledProcessError`. The
318 :exc:`CalledProcessError` object will have the return code in the
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400319 :attr:`returncode` attribute and output in the :attr:`output` attribute.
Georg Brandlf9734072008-12-07 15:30:06 +0000320
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400321 The arguments are the same as for the :func:`call` function. Example::
Georg Brandlf9734072008-12-07 15:30:06 +0000322
323 >>> subprocess.check_output(["ls", "-l", "/dev/null"])
Georg Brandl2708f3a2009-12-20 14:38:23 +0000324 b'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000325
326 The stdout argument is not allowed as it is used internally.
Benjamin Peterson25c95f12009-05-08 20:42:26 +0000327 To capture standard error in the result, use ``stderr=subprocess.STDOUT``::
Georg Brandlf9734072008-12-07 15:30:06 +0000328
329 >>> subprocess.check_output(
Georg Brandl2708f3a2009-12-20 14:38:23 +0000330 ... ["/bin/sh", "-c", "ls non_existent_file; exit 0"],
331 ... stderr=subprocess.STDOUT)
332 b'ls: non_existent_file: No such file or directory\n'
Georg Brandlf9734072008-12-07 15:30:06 +0000333
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400334 As in the :func:`call` function, if the timeout expires, the child process
335 will be killed and the wait retried. The :exc:`TimeoutExpired` exception
336 will be re-raised after the child process has terminated. The output from
337 the child process so far will be in the :attr:`output` attribute of the
338 exception.
339
Georg Brandlf9734072008-12-07 15:30:06 +0000340 .. versionadded:: 3.1
341
Reid Kleckner28f13032011-03-14 12:36:53 -0400342 .. versionchanged:: 3.3
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400343 *timeout* was added.
344
Georg Brandlf9734072008-12-07 15:30:06 +0000345
Brett Cannona23810f2008-05-26 19:04:21 +0000346.. function:: getstatusoutput(cmd)
Georg Brandl682d7e02010-10-06 10:26:05 +0000347
Brett Cannona23810f2008-05-26 19:04:21 +0000348 Return ``(status, output)`` of executing *cmd* in a shell.
349
350 Execute the string *cmd* in a shell with :func:`os.popen` and return a 2-tuple
351 ``(status, output)``. *cmd* is actually run as ``{ cmd ; } 2>&1``, so that the
352 returned output will contain output or error messages. A trailing newline is
353 stripped from the output. The exit status for the command can be interpreted
Georg Brandl60203b42010-10-06 10:11:56 +0000354 according to the rules for the C function :c:func:`wait`. Example::
Brett Cannona23810f2008-05-26 19:04:21 +0000355
Brett Cannona23810f2008-05-26 19:04:21 +0000356 >>> subprocess.getstatusoutput('ls /bin/ls')
357 (0, '/bin/ls')
358 >>> subprocess.getstatusoutput('cat /bin/junk')
359 (256, 'cat: /bin/junk: No such file or directory')
360 >>> subprocess.getstatusoutput('/bin/junk')
361 (256, 'sh: /bin/junk: not found')
362
Georg Brandl7d418902008-12-27 19:08:11 +0000363 Availability: UNIX.
364
Brett Cannona23810f2008-05-26 19:04:21 +0000365
366.. function:: getoutput(cmd)
Georg Brandl682d7e02010-10-06 10:26:05 +0000367
Georg Brandlf9734072008-12-07 15:30:06 +0000368 Return output (stdout and stderr) of executing *cmd* in a shell.
Brett Cannona23810f2008-05-26 19:04:21 +0000369
370 Like :func:`getstatusoutput`, except the exit status is ignored and the return
371 value is a string containing the command's output. Example::
372
Brett Cannona23810f2008-05-26 19:04:21 +0000373 >>> subprocess.getoutput('ls /bin/ls')
374 '/bin/ls'
375
Georg Brandl7d418902008-12-27 19:08:11 +0000376 Availability: UNIX.
377
Brett Cannona23810f2008-05-26 19:04:21 +0000378
Georg Brandl116aa622007-08-15 14:28:22 +0000379Exceptions
380^^^^^^^^^^
381
382Exceptions raised in the child process, before the new program has started to
383execute, will be re-raised in the parent. Additionally, the exception object
384will have one extra attribute called :attr:`child_traceback`, which is a string
Georg Brandl81675612010-08-26 14:30:56 +0000385containing traceback information from the child's point of view.
Georg Brandl116aa622007-08-15 14:28:22 +0000386
387The most common exception raised is :exc:`OSError`. This occurs, for example,
388when trying to execute a non-existent file. Applications should prepare for
389:exc:`OSError` exceptions.
390
391A :exc:`ValueError` will be raised if :class:`Popen` is called with invalid
392arguments.
393
394check_call() will raise :exc:`CalledProcessError`, if the called process returns
395a non-zero return code.
396
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400397All of the functions and methods that accept a *timeout* parameter, such as
398:func:`call` and :meth:`Popen.communicate` will raise :exc:`TimeoutExpired` if
399the timeout expires before the process exits.
400
Ronald Oussorenc1577902011-03-16 10:03:10 -0400401Exceptions defined in this module all inherit from :exc:`SubprocessError`.
Gregory P. Smith54d412e2011-03-14 14:08:43 -0400402
403 .. versionadded:: 3.3
404 The :exc:`SubprocessError` base class was added.
405
Georg Brandl116aa622007-08-15 14:28:22 +0000406
407Security
408^^^^^^^^
409
410Unlike some other popen functions, this implementation will never call /bin/sh
411implicitly. This means that all characters, including shell metacharacters, can
412safely be passed to child processes.
413
414
415Popen Objects
416-------------
417
418Instances of the :class:`Popen` class have the following methods:
419
420
421.. method:: Popen.poll()
422
Christian Heimes7f044312008-01-06 17:05:40 +0000423 Check if child process has terminated. Set and return :attr:`returncode`
424 attribute.
Georg Brandl116aa622007-08-15 14:28:22 +0000425
426
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400427.. method:: Popen.wait(timeout=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000428
Christian Heimes7f044312008-01-06 17:05:40 +0000429 Wait for child process to terminate. Set and return :attr:`returncode`
430 attribute.
Georg Brandl116aa622007-08-15 14:28:22 +0000431
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400432 If the process does not terminate after *timeout* seconds, raise a
433 :exc:`TimeoutExpired` exception. It is safe to catch this exception and
434 retry the wait.
435
Georg Brandl734e2682008-08-12 08:18:18 +0000436 .. warning::
437
Philip Jenveyb0896842009-12-03 02:29:36 +0000438 This will deadlock when using ``stdout=PIPE`` and/or
439 ``stderr=PIPE`` and the child process generates enough output to
440 a pipe such that it blocks waiting for the OS pipe buffer to
441 accept more data. Use :meth:`communicate` to avoid that.
Georg Brandl734e2682008-08-12 08:18:18 +0000442
Reid Kleckner28f13032011-03-14 12:36:53 -0400443 .. versionchanged:: 3.3
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400444 *timeout* was added.
Georg Brandl116aa622007-08-15 14:28:22 +0000445
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400446
447.. method:: Popen.communicate(input=None, timeout=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000448
449 Interact with process: Send data to stdin. Read data from stdout and stderr,
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400450 until end-of-file is reached. Wait for process to terminate. The optional
Georg Brandle11787a2008-07-01 19:10:52 +0000451 *input* argument should be a byte string to be sent to the child process, or
Georg Brandl116aa622007-08-15 14:28:22 +0000452 ``None``, if no data should be sent to the child.
453
Georg Brandlaf265f42008-12-07 15:06:20 +0000454 :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``.
Georg Brandl116aa622007-08-15 14:28:22 +0000455
Guido van Rossum0d3fb8a2007-11-26 23:23:18 +0000456 Note that if you want to send data to the process's stdin, you need to create
457 the Popen object with ``stdin=PIPE``. Similarly, to get anything other than
458 ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or
459 ``stderr=PIPE`` too.
460
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400461 If the process does not terminate after *timeout* seconds, a
462 :exc:`TimeoutExpired` exception will be raised. Catching this exception and
463 retrying communication will not lose any output.
464
465 The child process is not killed if the timeout expires, so in order to
466 cleanup properly a well-behaved application should kill the child process and
467 finish communication::
468
469 proc = subprocess.Popen(...)
470 try:
471 outs, errs = proc.communicate(timeout=15)
472 except TimeoutExpired:
473 proc.kill()
474 outs, errs = proc.communicate()
475
Christian Heimes7f044312008-01-06 17:05:40 +0000476 .. note::
Georg Brandl116aa622007-08-15 14:28:22 +0000477
Christian Heimes7f044312008-01-06 17:05:40 +0000478 The data read is buffered in memory, so do not use this method if the data
479 size is large or unlimited.
480
Reid Kleckner28f13032011-03-14 12:36:53 -0400481 .. versionchanged:: 3.3
Reid Kleckner31aa7dd2011-03-14 12:02:10 -0400482 *timeout* was added.
483
Georg Brandl116aa622007-08-15 14:28:22 +0000484
Christian Heimesa342c012008-04-20 21:01:16 +0000485.. method:: Popen.send_signal(signal)
486
487 Sends the signal *signal* to the child.
488
489 .. note::
490
Brian Curtineb24d742010-04-12 17:16:38 +0000491 On Windows, SIGTERM is an alias for :meth:`terminate`. CTRL_C_EVENT and
Senthil Kumaran916bd382010-10-15 12:55:19 +0000492 CTRL_BREAK_EVENT can be sent to processes started with a *creationflags*
Brian Curtineb24d742010-04-12 17:16:38 +0000493 parameter which includes `CREATE_NEW_PROCESS_GROUP`.
Christian Heimesa342c012008-04-20 21:01:16 +0000494
Christian Heimesa342c012008-04-20 21:01:16 +0000495
496.. method:: Popen.terminate()
497
498 Stop the child. On Posix OSs the method sends SIGTERM to the
Georg Brandl60203b42010-10-06 10:11:56 +0000499 child. On Windows the Win32 API function :c:func:`TerminateProcess` is called
Christian Heimesa342c012008-04-20 21:01:16 +0000500 to stop the child.
501
Christian Heimesa342c012008-04-20 21:01:16 +0000502
503.. method:: Popen.kill()
504
505 Kills the child. On Posix OSs the function sends SIGKILL to the child.
506 On Windows :meth:`kill` is an alias for :meth:`terminate`.
507
Christian Heimesa342c012008-04-20 21:01:16 +0000508
Georg Brandl116aa622007-08-15 14:28:22 +0000509The following attributes are also available:
510
Georg Brandl734e2682008-08-12 08:18:18 +0000511.. warning::
512
Georg Brandle720c0a2009-04-27 16:20:50 +0000513 Use :meth:`communicate` rather than :attr:`.stdin.write <stdin>`,
514 :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` to avoid
515 deadlocks due to any of the other OS pipe buffers filling up and blocking the
516 child process.
Georg Brandl734e2682008-08-12 08:18:18 +0000517
518
Georg Brandl116aa622007-08-15 14:28:22 +0000519.. attribute:: Popen.stdin
520
Antoine Pitrou11cb9612010-09-15 11:11:28 +0000521 If the *stdin* argument was :data:`PIPE`, this attribute is a :term:`file
522 object` that provides input to the child process. Otherwise, it is ``None``.
Georg Brandl116aa622007-08-15 14:28:22 +0000523
524
525.. attribute:: Popen.stdout
526
Antoine Pitrou11cb9612010-09-15 11:11:28 +0000527 If the *stdout* argument was :data:`PIPE`, this attribute is a :term:`file
528 object` that provides output from the child process. Otherwise, it is ``None``.
Georg Brandl116aa622007-08-15 14:28:22 +0000529
530
531.. attribute:: Popen.stderr
532
Antoine Pitrou11cb9612010-09-15 11:11:28 +0000533 If the *stderr* argument was :data:`PIPE`, this attribute is a :term:`file
534 object` that provides error output from the child process. Otherwise, it is
Georg Brandlaf265f42008-12-07 15:06:20 +0000535 ``None``.
Georg Brandl116aa622007-08-15 14:28:22 +0000536
537
538.. attribute:: Popen.pid
539
540 The process ID of the child process.
541
Georg Brandl58bfdca2010-03-21 09:50:49 +0000542 Note that if you set the *shell* argument to ``True``, this is the process ID
543 of the spawned shell.
544
Georg Brandl116aa622007-08-15 14:28:22 +0000545
546.. attribute:: Popen.returncode
547
Christian Heimes7f044312008-01-06 17:05:40 +0000548 The child return code, set by :meth:`poll` and :meth:`wait` (and indirectly
549 by :meth:`communicate`). A ``None`` value indicates that the process
550 hasn't terminated yet.
Georg Brandl48310cd2009-01-03 21:18:54 +0000551
Christian Heimes7f044312008-01-06 17:05:40 +0000552 A negative value ``-N`` indicates that the child was terminated by signal
553 ``N`` (Unix only).
Georg Brandl116aa622007-08-15 14:28:22 +0000554
555
Benjamin Petersondcf97b92008-07-02 17:30:14 +0000556.. _subprocess-replacements:
557
Georg Brandl116aa622007-08-15 14:28:22 +0000558Replacing Older Functions with the subprocess Module
559----------------------------------------------------
560
561In this section, "a ==> b" means that b can be used as a replacement for a.
562
563.. note::
564
565 All functions in this section fail (more or less) silently if the executed
566 program cannot be found; this module raises an :exc:`OSError` exception.
567
568In the following examples, we assume that the subprocess module is imported with
569"from subprocess import \*".
570
571
572Replacing /bin/sh shell backquote
573^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
574
575::
576
577 output=`mycmd myarg`
578 ==>
579 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
580
581
Benjamin Petersonf10a79a2008-10-11 00:49:57 +0000582Replacing shell pipeline
583^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +0000584
585::
586
587 output=`dmesg | grep hda`
588 ==>
589 p1 = Popen(["dmesg"], stdout=PIPE)
590 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Gregory P. Smithe09d2f12011-02-05 21:47:25 +0000591 p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
Georg Brandl116aa622007-08-15 14:28:22 +0000592 output = p2.communicate()[0]
593
Gregory P. Smithe09d2f12011-02-05 21:47:25 +0000594The p1.stdout.close() call after starting the p2 is important in order for p1
595to receive a SIGPIPE if p2 exits before p1.
Georg Brandl116aa622007-08-15 14:28:22 +0000596
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000597Replacing :func:`os.system`
598^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +0000599
600::
601
602 sts = os.system("mycmd" + " myarg")
603 ==>
604 p = Popen("mycmd" + " myarg", shell=True)
Alexandre Vassalottie52e3782009-07-17 09:18:18 +0000605 sts = os.waitpid(p.pid, 0)[1]
Georg Brandl116aa622007-08-15 14:28:22 +0000606
607Notes:
608
609* Calling the program through the shell is usually not required.
610
611* It's easier to look at the :attr:`returncode` attribute than the exit status.
612
613A more realistic example would look like this::
614
615 try:
616 retcode = call("mycmd" + " myarg", shell=True)
617 if retcode < 0:
Collin Winterc79461b2007-09-01 23:34:30 +0000618 print("Child was terminated by signal", -retcode, file=sys.stderr)
Georg Brandl116aa622007-08-15 14:28:22 +0000619 else:
Collin Winterc79461b2007-09-01 23:34:30 +0000620 print("Child returned", retcode, file=sys.stderr)
Georg Brandl116aa622007-08-15 14:28:22 +0000621 except OSError as e:
Collin Winterc79461b2007-09-01 23:34:30 +0000622 print("Execution failed:", e, file=sys.stderr)
Georg Brandl116aa622007-08-15 14:28:22 +0000623
624
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000625Replacing the :func:`os.spawn <os.spawnl>` family
626^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +0000627
628P_NOWAIT example::
629
630 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
631 ==>
632 pid = Popen(["/bin/mycmd", "myarg"]).pid
633
634P_WAIT example::
635
636 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
637 ==>
638 retcode = call(["/bin/mycmd", "myarg"])
639
640Vector example::
641
642 os.spawnvp(os.P_NOWAIT, path, args)
643 ==>
644 Popen([path] + args[1:])
645
646Environment example::
647
648 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
649 ==>
650 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
651
652
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000653
654Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3`
655^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +0000656
657::
658
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000659 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
Georg Brandl116aa622007-08-15 14:28:22 +0000660 ==>
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000661 p = Popen(cmd, shell=True, bufsize=bufsize,
662 stdin=PIPE, stdout=PIPE, close_fds=True)
663 (child_stdin, child_stdout) = (p.stdin, p.stdout)
Georg Brandl116aa622007-08-15 14:28:22 +0000664
665::
666
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000667 (child_stdin,
668 child_stdout,
669 child_stderr) = os.popen3(cmd, mode, bufsize)
Georg Brandl116aa622007-08-15 14:28:22 +0000670 ==>
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000671 p = Popen(cmd, shell=True, bufsize=bufsize,
672 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
673 (child_stdin,
674 child_stdout,
675 child_stderr) = (p.stdin, p.stdout, p.stderr)
676
677::
678
679 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
680 ==>
681 p = Popen(cmd, shell=True, bufsize=bufsize,
682 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
683 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
684
685Return code handling translates as follows::
686
687 pipe = os.popen(cmd, 'w')
688 ...
689 rc = pipe.close()
Stefan Krahfc9e08d2010-07-14 10:16:11 +0000690 if rc is not None and rc >> 8:
Ezio Melotti985e24d2009-09-13 07:54:02 +0000691 print("There were some errors")
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000692 ==>
693 process = Popen(cmd, 'w', stdin=PIPE)
694 ...
695 process.stdin.close()
696 if process.wait() != 0:
Ezio Melotti985e24d2009-09-13 07:54:02 +0000697 print("There were some errors")
Benjamin Peterson87c8d872009-06-11 22:54:11 +0000698
699
700Replacing functions from the :mod:`popen2` module
701^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
702
703.. note::
704
705 If the cmd argument to popen2 functions is a string, the command is executed
706 through /bin/sh. If it is a list, the command is directly executed.
707
708::
709
710 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
711 ==>
712 p = Popen(["somestring"], shell=True, bufsize=bufsize,
713 stdin=PIPE, stdout=PIPE, close_fds=True)
714 (child_stdout, child_stdin) = (p.stdout, p.stdin)
715
716::
717
718 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
719 ==>
720 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
721 stdin=PIPE, stdout=PIPE, close_fds=True)
722 (child_stdout, child_stdin) = (p.stdout, p.stdin)
723
724:class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as
725:class:`subprocess.Popen`, except that:
726
727* :class:`Popen` raises an exception if the execution fails.
728
729* the *capturestderr* argument is replaced with the *stderr* argument.
730
731* ``stdin=PIPE`` and ``stdout=PIPE`` must be specified.
732
733* popen2 closes all file descriptors by default, but you have to specify
Gregory P. Smithf5604852010-12-13 06:45:02 +0000734 ``close_fds=True`` with :class:`Popen` to guarantee this behavior on
735 all platforms or past Python versions.