blob: 02ff1c9d1ee5ca1bb56343b537c52d14e47bdbe1 [file] [log] [blame]
Georg Brandl8ec7f652007-08-15 14:28:01 +00001
2:mod:`subprocess` --- Subprocess management
3===========================================
4
5.. module:: subprocess
6 :synopsis: Subprocess management.
7.. moduleauthor:: Peter Åstrand <astrand@lysator.liu.se>
8.. sectionauthor:: Peter Åstrand <astrand@lysator.liu.se>
9
10
11.. versionadded:: 2.4
12
13The :mod:`subprocess` module allows you to spawn new processes, connect to their
14input/output/error pipes, and obtain their return codes. This module intends to
15replace several other, older modules and functions, such as::
16
17 os.system
18 os.spawn*
19 os.popen*
20 popen2.*
21 commands.*
22
23Information about how the :mod:`subprocess` module can be used to replace these
24modules and functions can be found in the following sections.
25
Georg Brandl68b4e742008-07-01 19:59:00 +000026.. seealso::
27
28 :pep:`324` -- PEP proposing the subprocess module
29
Georg Brandl8ec7f652007-08-15 14:28:01 +000030
31Using the subprocess Module
32---------------------------
33
Nick Coghlan86711572011-10-24 22:19:40 +100034The recommended interface to this module is to use the following convenience
35functions for all use cases they can handle. For more advanced use cases, the
36underlying :class:`Popen` interface can be used directly.
37
38
39.. function:: call(args, *, stdin=None, stdout=None, stderr=None)
40
41 Run the command described by *args*. Wait for command to complete, then
42 return the :attr:`returncode` attribute.
43
44 The arguments shown above are merely the most common ones, described below
45 in :ref:`frequently-used-arguments`. The full function signature is the
46 same as that of the :class:`Popen` constructor - the convenience functions
47 pass all supplied arguments directly through to that interface.
48
49 Examples::
50
51 >>> subprocess.call(["ls", "-l"])
52 0
53
54 >>> subprocess.call(["python", "-c", "import sys; sys.exit(1)"])
55 1
56
57 .. warning::
58
59 Like :meth:`Popen.wait`, this will deadlock when using
60 ``stdout=PIPE`` and/or ``stderr=PIPE`` and the child process
61 generates enough output to a pipe such that it blocks waiting
62 for the OS pipe buffer to accept more data.
63
64
65.. function:: check_call(*callargs, **kwargs)
66
67 Run command with arguments. Wait for command to complete. If the return
68 code was zero then return, otherwise raise :exc:`CalledProcessError`. The
69 :exc:`CalledProcessError` object will have the return code in the
70 :attr:`returncode` attribute.
71
72 The arguments are the same as for :func:`call`. Examples::
73
74 >>> subprocess.check_call(["ls", "-l"])
75 0
76
77 >>> subprocess.check_call(["python", "-c", "import sys; sys.exit(1)"])
78 Traceback (most recent call last):
79 ...
80 subprocess.CalledProcessError: Command '['python', '-c', 'import sys; sys.exit(1)']' returned non-zero exit status 1
81
82 .. versionadded:: 2.5
83
84 .. warning::
85
86 See the warning for :func:`call`.
87
88
89.. function:: check_output(*callargs, **kwargs)
90
91 Run command with arguments and return its output as a byte string.
92
93 If the return code was non-zero it raises a :exc:`CalledProcessError`. The
94 :exc:`CalledProcessError` object will have the return code in the
95 :attr:`returncode` attribute and any output in the :attr:`output`
96 attribute.
97
98 Examples::
99
100 >>> subprocess.check_output(["ls", "-l", "/dev/null"])
101 'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
102
103 >>> subprocess.check_output(["python", "-c", "import sys; sys.exit(1)"])
104 Traceback (most recent call last):
105 ...
106 subprocess.CalledProcessError: Command '['python', '-c', 'import sys; sys.exit(1)']' returned non-zero exit status 1
107
108 The arguments are the same as for :func:`call`, except that *stdout* is
109 not allowed as it is used internally. To also capture standard error in
110 the result, use ``stderr=subprocess.STDOUT``::
111
112 >>> subprocess.check_output(
113 ... ["/bin/sh", "-c", "ls non_existent_file; exit 0"],
114 ... stderr=subprocess.STDOUT)
115 'ls: non_existent_file: No such file or directory\n'
116
117 .. versionadded:: 2.7
118
119
120.. data:: PIPE
121
122 Special value that can be used as the *stdin*, *stdout* or *stderr* argument
123 to :class:`Popen` and indicates that a pipe to the standard stream should be
124 opened.
125
126
127.. data:: STDOUT
128
129 Special value that can be used as the *stderr* argument to :class:`Popen` and
130 indicates that standard error should go into the same handle as standard
131 output.
132
133
134.. _frequently-used-arguments:
135
136Frequently Used Arguments
137^^^^^^^^^^^^^^^^^^^^^^^^^
138
139To support a wide variety of use cases, the :class:`Popen` constructor (and
140the convenience functions) accept a large number of optional arguments. For
141most typical use cases, many of these arguments can be safely left at their
142default values. The arguments that are most commonly needed are:
143
144 *args* should be a string, or a sequence of program arguments. Providing
145 a sequence of arguments is generally preferred, as it allows the module to
146 take care of any required escaping and quoting of arguments (e.g. to permit
147 spaces in file names)
148
149 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
150 standard output and standard error file handles, respectively. Valid values
151 are :data:`PIPE`, an existing file descriptor (a positive integer), an
152 existing file object, and ``None``. :data:`PIPE` indicates that a new pipe
153 to the child should be created. With the default settings of ``None``, no
154 redirection will occur; the child's file handles will be inherited from the
155 parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that
156 the stderr data from the child process should be captured into the same file
157 handle as for stdout.
158
159These options, along with all of the other options, are described in more
160detail in the :class:`Popen` constructor documentation.
161
162
163Popen Constuctor
164^^^^^^^^^^^^^^^^
165
166The underlying process creation and management in this module is handled by
167the :class:`Popen` class. It offers a lot of flexibility so that developers
168are able to handle the less common cases not covered by the convenience
169functions.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000170
171
172.. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)
173
174 Arguments are:
175
Benjamin Petersonfff5cf62008-07-27 15:22:14 +0000176 *args* should be a string, or a sequence of program arguments. The program
R. David Murrayfe6e7842009-05-29 19:30:27 +0000177 to execute is normally the first item in the args sequence or the string if
178 a string is given, but can be explicitly set by using the *executable*
179 argument. When *executable* is given, the first item in the args sequence
180 is still treated by most programs as the command name, which can then be
181 different from the actual executable name. On Unix, it becomes the display
182 name for the executing program in utilities such as :program:`ps`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000183
184 On Unix, with *shell=False* (default): In this case, the Popen class uses
185 :meth:`os.execvp` to execute the child program. *args* should normally be a
Nick Coghlan7dfc9e12010-02-04 12:43:58 +0000186 sequence. If a string is specified for *args*, it will be used as the name
187 or path of the program to execute; this will only work if the program is
188 being given no arguments.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000189
Nick Coghlan7dfc9e12010-02-04 12:43:58 +0000190 .. note::
191
192 :meth:`shlex.split` can be useful when determining the correct
193 tokenization for *args*, especially in complex cases::
194
195 >>> import shlex, subprocess
196 >>> command_line = raw_input()
197 /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
198 >>> args = shlex.split(command_line)
199 >>> print args
200 ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
201 >>> p = subprocess.Popen(args) # Success!
202
203 Note in particular that options (such as *-input*) and arguments (such
204 as *eggs.txt*) that are separated by whitespace in the shell go in separate
205 list elements, while arguments that need quoting or backslash escaping when
206 used in the shell (such as filenames containing spaces or the *echo* command
207 shown above) are single list elements.
208
209 On Unix, with *shell=True*: If args is a string, it specifies the command
210 string to execute through the shell. This means that the string must be
211 formatted exactly as it would be when typed at the shell prompt. This
212 includes, for example, quoting or backslash escaping filenames with spaces in
213 them. If *args* is a sequence, the first item specifies the command string, and
214 any additional items will be treated as additional arguments to the shell
215 itself. That is to say, *Popen* does the equivalent of::
216
217 Popen(['/bin/sh', '-c', args[0], args[1], ...])
Georg Brandl8ec7f652007-08-15 14:28:01 +0000218
R. David Murray6e4300c2010-11-12 00:39:09 +0000219 .. warning::
220
221 Executing shell commands that incorporate unsanitized input from an
222 untrusted source makes a program vulnerable to `shell injection
223 <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_,
224 a serious security flaw which can result in arbitrary command execution.
225 For this reason, the use of *shell=True* is **strongly discouraged** in cases
226 where the command string is constructed from external input::
227
228 >>> from subprocess import call
229 >>> filename = input("What file would you like to display?\n")
230 What file would you like to display?
231 non_existent; rm -rf / #
232 >>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
233
234 *shell=False* does not suffer from this vulnerability; the above Note may be
235 helpful in getting code using *shell=False* to work.
236
Georg Brandl8ec7f652007-08-15 14:28:01 +0000237 On Windows: the :class:`Popen` class uses CreateProcess() to execute the child
Eli Bendersky929e2762011-04-15 07:35:06 +0300238 child program, which operates on strings. If *args* is a sequence, it will
239 be converted to a string in a manner described in
240 :ref:`converting-argument-sequence`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000241
242 *bufsize*, if given, has the same meaning as the corresponding argument to the
243 built-in open() function: :const:`0` means unbuffered, :const:`1` means line
244 buffered, any other positive value means use a buffer of (approximately) that
245 size. A negative *bufsize* means to use the system default, which usually means
246 fully buffered. The default value for *bufsize* is :const:`0` (unbuffered).
247
Antoine Pitrouc3955452010-06-02 17:08:47 +0000248 .. note::
249
250 If you experience performance issues, it is recommended that you try to
251 enable buffering by setting *bufsize* to either -1 or a large enough
252 positive value (such as 4096).
253
Georg Brandl8ec7f652007-08-15 14:28:01 +0000254 The *executable* argument specifies the program to execute. It is very seldom
255 needed: Usually, the program to execute is defined by the *args* argument. If
256 ``shell=True``, the *executable* argument specifies which shell to use. On Unix,
257 the default shell is :file:`/bin/sh`. On Windows, the default shell is
Georg Brandl0d8649a2009-06-30 16:17:28 +0000258 specified by the :envvar:`COMSPEC` environment variable. The only reason you
259 would need to specify ``shell=True`` on Windows is where the command you
260 wish to execute is actually built in to the shell, eg ``dir``, ``copy``.
261 You don't need ``shell=True`` to run a batch file, nor to run a console-based
262 executable.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000263
Nick Coghlan86711572011-10-24 22:19:40 +1000264 *stdin*, *stdout* and *stderr* specify the executed program's standard input,
Georg Brandlf5d5a662008-12-06 11:57:12 +0000265 standard output and standard error file handles, respectively. Valid values
266 are :data:`PIPE`, an existing file descriptor (a positive integer), an
267 existing file object, and ``None``. :data:`PIPE` indicates that a new pipe
Nick Coghlan86711572011-10-24 22:19:40 +1000268 to the child should be created. With the default settings of ``None``, no
269 redirection will occur; the child's file handles will be inherited from the
270 parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that
271 the stderr data from the child process should be captured into the same file
272 handle as for stdout.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000273
274 If *preexec_fn* is set to a callable object, this object will be called in the
275 child process just before the child is executed. (Unix only)
276
277 If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
278 :const:`2` will be closed before the child process is executed. (Unix only).
279 Or, on Windows, if *close_fds* is true then no handles will be inherited by the
280 child process. Note that on Windows, you cannot set *close_fds* to true and
281 also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.
282
283 If *shell* is :const:`True`, the specified command will be executed through the
284 shell.
285
286 If *cwd* is not ``None``, the child's current directory will be changed to *cwd*
287 before it is executed. Note that this directory is not considered when
288 searching the executable, so you can't specify the program's path relative to
289 *cwd*.
290
Georg Brandlf801b0f2008-04-19 16:58:49 +0000291 If *env* is not ``None``, it must be a mapping that defines the environment
292 variables for the new process; these are used instead of inheriting the current
293 process' environment, which is the default behavior.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000294
R. David Murray72030812009-04-16 18:12:53 +0000295 .. note::
R. David Murray6076d392009-04-15 22:33:07 +0000296
R. David Murray72030812009-04-16 18:12:53 +0000297 If specified, *env* must provide any variables required
298 for the program to execute. On Windows, in order to run a
299 `side-by-side assembly`_ the specified *env* **must** include a valid
R. David Murray6076d392009-04-15 22:33:07 +0000300 :envvar:`SystemRoot`.
301
R. David Murray72030812009-04-16 18:12:53 +0000302 .. _side-by-side assembly: http://en.wikipedia.org/wiki/Side-by-Side_Assembly
303
Georg Brandl8ec7f652007-08-15 14:28:01 +0000304 If *universal_newlines* is :const:`True`, the file objects stdout and stderr are
305 opened as text files, but lines may be terminated by any of ``'\n'``, the Unix
Georg Brandl9af94982008-09-13 17:41:16 +0000306 end-of-line convention, ``'\r'``, the old Macintosh convention or ``'\r\n'``, the
Georg Brandl8ec7f652007-08-15 14:28:01 +0000307 Windows convention. All of these external representations are seen as ``'\n'``
308 by the Python program.
309
310 .. note::
311
Georg Brandl6ab5d082009-12-20 14:33:20 +0000312 This feature is only available if Python is built with universal newline
313 support (the default). Also, the newlines attribute of the file objects
314 :attr:`stdout`, :attr:`stdin` and :attr:`stderr` are not updated by the
315 communicate() method.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000316
Brian Curtinbb23bd62011-04-29 22:23:46 -0500317 If given, *startupinfo* will be a :class:`STARTUPINFO` object, which is
318 passed to the underlying ``CreateProcess`` function.
319 *creationflags*, if given, can be :data:`CREATE_NEW_CONSOLE` or
320 :data:`CREATE_NEW_PROCESS_GROUP`. (Windows only)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000321
322
Georg Brandl8ec7f652007-08-15 14:28:01 +0000323Exceptions
324^^^^^^^^^^
325
326Exceptions raised in the child process, before the new program has started to
327execute, will be re-raised in the parent. Additionally, the exception object
328will have one extra attribute called :attr:`child_traceback`, which is a string
Georg Brandl21946af2010-10-06 09:28:45 +0000329containing traceback information from the child's point of view.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000330
331The most common exception raised is :exc:`OSError`. This occurs, for example,
332when trying to execute a non-existent file. Applications should prepare for
333:exc:`OSError` exceptions.
334
335A :exc:`ValueError` will be raised if :class:`Popen` is called with invalid
336arguments.
337
338check_call() will raise :exc:`CalledProcessError`, if the called process returns
339a non-zero return code.
340
341
342Security
343^^^^^^^^
344
345Unlike some other popen functions, this implementation will never call /bin/sh
346implicitly. This means that all characters, including shell metacharacters, can
347safely be passed to child processes.
348
349
350Popen Objects
351-------------
352
353Instances of the :class:`Popen` class have the following methods:
354
355
356.. method:: Popen.poll()
357
Georg Brandl2cb103f2008-01-06 16:01:26 +0000358 Check if child process has terminated. Set and return :attr:`returncode`
359 attribute.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000360
361
362.. method:: Popen.wait()
363
Georg Brandl2cb103f2008-01-06 16:01:26 +0000364 Wait for child process to terminate. Set and return :attr:`returncode`
365 attribute.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000366
Georg Brandl143de622008-08-04 06:29:36 +0000367 .. warning::
368
Philip Jenvey26275532009-12-03 02:25:54 +0000369 This will deadlock when using ``stdout=PIPE`` and/or
370 ``stderr=PIPE`` and the child process generates enough output to
371 a pipe such that it blocks waiting for the OS pipe buffer to
372 accept more data. Use :meth:`communicate` to avoid that.
Gregory P. Smith08792502008-08-04 01:03:50 +0000373
Georg Brandl8ec7f652007-08-15 14:28:01 +0000374
375.. method:: Popen.communicate(input=None)
376
377 Interact with process: Send data to stdin. Read data from stdout and stderr,
378 until end-of-file is reached. Wait for process to terminate. The optional
379 *input* argument should be a string to be sent to the child process, or
380 ``None``, if no data should be sent to the child.
381
Georg Brandl17432012008-12-04 21:28:16 +0000382 :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000383
Georg Brandl439f2502007-11-24 11:31:46 +0000384 Note that if you want to send data to the process's stdin, you need to create
385 the Popen object with ``stdin=PIPE``. Similarly, to get anything other than
386 ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or
387 ``stderr=PIPE`` too.
388
Georg Brandl2cb103f2008-01-06 16:01:26 +0000389 .. note::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000390
Georg Brandl2cb103f2008-01-06 16:01:26 +0000391 The data read is buffered in memory, so do not use this method if the data
392 size is large or unlimited.
393
Georg Brandl8ec7f652007-08-15 14:28:01 +0000394
Christian Heimese74c8f22008-04-19 02:23:57 +0000395.. method:: Popen.send_signal(signal)
396
397 Sends the signal *signal* to the child.
398
399 .. note::
400
Brian Curtine5aa8862010-04-02 23:26:06 +0000401 On Windows, SIGTERM is an alias for :meth:`terminate`. CTRL_C_EVENT and
Ezio Melotti9ccc5812010-04-05 08:16:41 +0000402 CTRL_BREAK_EVENT can be sent to processes started with a *creationflags*
Brian Curtine5aa8862010-04-02 23:26:06 +0000403 parameter which includes `CREATE_NEW_PROCESS_GROUP`.
Georg Brandl734de682008-04-19 08:23:59 +0000404
405 .. versionadded:: 2.6
Christian Heimese74c8f22008-04-19 02:23:57 +0000406
407
408.. method:: Popen.terminate()
409
410 Stop the child. On Posix OSs the method sends SIGTERM to the
Andrew M. Kuchling64c6a0e2008-04-21 02:08:00 +0000411 child. On Windows the Win32 API function :cfunc:`TerminateProcess` is called
Christian Heimese74c8f22008-04-19 02:23:57 +0000412 to stop the child.
413
Georg Brandl734de682008-04-19 08:23:59 +0000414 .. versionadded:: 2.6
415
Christian Heimese74c8f22008-04-19 02:23:57 +0000416
417.. method:: Popen.kill()
418
419 Kills the child. On Posix OSs the function sends SIGKILL to the child.
Georg Brandl734de682008-04-19 08:23:59 +0000420 On Windows :meth:`kill` is an alias for :meth:`terminate`.
421
422 .. versionadded:: 2.6
Christian Heimese74c8f22008-04-19 02:23:57 +0000423
424
Georg Brandl8ec7f652007-08-15 14:28:01 +0000425The following attributes are also available:
426
Georg Brandl143de622008-08-04 06:29:36 +0000427.. warning::
428
Georg Brandl16a57f62009-04-27 15:29:09 +0000429 Use :meth:`communicate` rather than :attr:`.stdin.write <stdin>`,
430 :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` to avoid
431 deadlocks due to any of the other OS pipe buffers filling up and blocking the
432 child process.
Georg Brandl143de622008-08-04 06:29:36 +0000433
434
Georg Brandl8ec7f652007-08-15 14:28:01 +0000435.. attribute:: Popen.stdin
436
Georg Brandlf5d5a662008-12-06 11:57:12 +0000437 If the *stdin* argument was :data:`PIPE`, this attribute is a file object
438 that provides input to the child process. Otherwise, it is ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000439
440
441.. attribute:: Popen.stdout
442
Georg Brandlf5d5a662008-12-06 11:57:12 +0000443 If the *stdout* argument was :data:`PIPE`, this attribute is a file object
444 that provides output from the child process. Otherwise, it is ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000445
446
447.. attribute:: Popen.stderr
448
Georg Brandlf5d5a662008-12-06 11:57:12 +0000449 If the *stderr* argument was :data:`PIPE`, this attribute is a file object
450 that provides error output from the child process. Otherwise, it is
451 ``None``.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000452
453
454.. attribute:: Popen.pid
455
456 The process ID of the child process.
457
Georg Brandl0b56ce02010-03-21 09:28:16 +0000458 Note that if you set the *shell* argument to ``True``, this is the process ID
459 of the spawned shell.
460
Georg Brandl8ec7f652007-08-15 14:28:01 +0000461
462.. attribute:: Popen.returncode
463
Georg Brandl2cb103f2008-01-06 16:01:26 +0000464 The child return code, set by :meth:`poll` and :meth:`wait` (and indirectly
465 by :meth:`communicate`). A ``None`` value indicates that the process
466 hasn't terminated yet.
Georg Brandlc62ef8b2009-01-03 20:55:06 +0000467
Georg Brandl2cb103f2008-01-06 16:01:26 +0000468 A negative value ``-N`` indicates that the child was terminated by signal
469 ``N`` (Unix only).
Georg Brandl8ec7f652007-08-15 14:28:01 +0000470
471
Brian Curtinbb23bd62011-04-29 22:23:46 -0500472Windows Popen Helpers
473---------------------
474
475The :class:`STARTUPINFO` class and following constants are only available
476on Windows.
477
478.. class:: STARTUPINFO()
479
480 Partial support of the Windows
481 `STARTUPINFO <http://msdn.microsoft.com/en-us/library/ms686331(v=vs.85).aspx>`__
482 structure is used for :class:`Popen` creation.
483
484 .. attribute:: dwFlags
485
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700486 A bit field that determines whether certain :class:`STARTUPINFO`
487 attributes are used when the process creates a window. ::
Brian Curtinbb23bd62011-04-29 22:23:46 -0500488
489 si = subprocess.STARTUPINFO()
490 si.dwFlags = subprocess.STARTF_USESTDHANDLES | subprocess.STARTF_USESHOWWINDOW
491
492 .. attribute:: hStdInput
493
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700494 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
495 is the standard input handle for the process. If
496 :data:`STARTF_USESTDHANDLES` is not specified, the default for standard
497 input is the keyboard buffer.
Brian Curtinbb23bd62011-04-29 22:23:46 -0500498
499 .. attribute:: hStdOutput
500
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700501 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
502 is the standard output handle for the process. Otherwise, this attribute
503 is ignored and the default for standard output is the console window's
Brian Curtinbb23bd62011-04-29 22:23:46 -0500504 buffer.
505
506 .. attribute:: hStdError
507
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700508 If :attr:`dwFlags` specifies :data:`STARTF_USESTDHANDLES`, this attribute
509 is the standard error handle for the process. Otherwise, this attribute is
Brian Curtinbb23bd62011-04-29 22:23:46 -0500510 ignored and the default for standard error is the console window's buffer.
511
512 .. attribute:: wShowWindow
513
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700514 If :attr:`dwFlags` specifies :data:`STARTF_USESHOWWINDOW`, this attribute
Brian Curtinbb23bd62011-04-29 22:23:46 -0500515 can be any of the values that can be specified in the ``nCmdShow``
516 parameter for the
517 `ShowWindow <http://msdn.microsoft.com/en-us/library/ms633548(v=vs.85).aspx>`__
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700518 function, except for ``SW_SHOWDEFAULT``. Otherwise, this attribute is
Brian Curtinbb23bd62011-04-29 22:23:46 -0500519 ignored.
520
521 :data:`SW_HIDE` is provided for this attribute. It is used when
522 :class:`Popen` is called with ``shell=True``.
523
524
525Constants
526^^^^^^^^^
527
528The :mod:`subprocess` module exposes the following constants.
529
530.. data:: STD_INPUT_HANDLE
531
532 The standard input device. Initially, this is the console input buffer,
533 ``CONIN$``.
534
535.. data:: STD_OUTPUT_HANDLE
536
537 The standard output device. Initially, this is the active console screen
538 buffer, ``CONOUT$``.
539
540.. data:: STD_ERROR_HANDLE
541
542 The standard error device. Initially, this is the active console screen
543 buffer, ``CONOUT$``.
544
545.. data:: SW_HIDE
546
547 Hides the window. Another window will be activated.
548
549.. data:: STARTF_USESTDHANDLES
550
551 Specifies that the :attr:`STARTUPINFO.hStdInput`,
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700552 :attr:`STARTUPINFO.hStdOutput`, and :attr:`STARTUPINFO.hStdError` attributes
Brian Curtinbb23bd62011-04-29 22:23:46 -0500553 contain additional information.
554
555.. data:: STARTF_USESHOWWINDOW
556
Senthil Kumaran6f18b982011-07-04 12:50:02 -0700557 Specifies that the :attr:`STARTUPINFO.wShowWindow` attribute contains
Brian Curtinbb23bd62011-04-29 22:23:46 -0500558 additional information.
559
560.. data:: CREATE_NEW_CONSOLE
561
562 The new process has a new console, instead of inheriting its parent's
563 console (the default).
564
565 This flag is always set when :class:`Popen` is created with ``shell=True``.
566
567.. data:: CREATE_NEW_PROCESS_GROUP
568
569 A :class:`Popen` ``creationflags`` parameter to specify that a new process
570 group will be created. This flag is necessary for using :func:`os.kill`
571 on the subprocess.
572
573 This flag is ignored if :data:`CREATE_NEW_CONSOLE` is specified.
574
575
Georg Brandl0ba92b22008-06-22 09:05:29 +0000576.. _subprocess-replacements:
577
Georg Brandl8ec7f652007-08-15 14:28:01 +0000578Replacing Older Functions with the subprocess Module
579----------------------------------------------------
580
Nick Coghlan86711572011-10-24 22:19:40 +1000581In this section, "a becomes b" means that b can be used as a replacement for a.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000582
583.. note::
584
585 All functions in this section fail (more or less) silently if the executed
Nick Coghlan86711572011-10-24 22:19:40 +1000586 program cannot be found; this module raises an :exc:`OSError` exception. In
587 addition, the replacements using :func:`check_output` will fail with a
588 :exc:`CalledProcessError` if the requested operation produces a non-zero
589 return code.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000590
591In the following examples, we assume that the subprocess module is imported with
592"from subprocess import \*".
593
594
595Replacing /bin/sh shell backquote
596^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
597
598::
599
600 output=`mycmd myarg`
Nick Coghlan86711572011-10-24 22:19:40 +1000601 # becomes
602 output = check_output(["mycmd", "myarg"])
Georg Brandl8ec7f652007-08-15 14:28:01 +0000603
604
Benjamin Petersoncae58482008-10-10 20:38:49 +0000605Replacing shell pipeline
606^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000607
608::
609
610 output=`dmesg | grep hda`
Nick Coghlan86711572011-10-24 22:19:40 +1000611 # becomes
Georg Brandl8ec7f652007-08-15 14:28:01 +0000612 p1 = Popen(["dmesg"], stdout=PIPE)
613 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
Gregory P. Smithe3e967f2011-02-05 21:49:56 +0000614 p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000615 output = p2.communicate()[0]
616
Gregory P. Smithe3e967f2011-02-05 21:49:56 +0000617The p1.stdout.close() call after starting the p2 is important in order for p1
618to receive a SIGPIPE if p2 exits before p1.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000619
Nick Coghlan86711572011-10-24 22:19:40 +1000620Alternatively, for trusted input, the shell's pipeline may still be used
621directly:
622
623 output=`dmesg | grep hda`
624 # becomes
625 output=check_output("dmesg | grep hda", shell=True)
626
627
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000628Replacing :func:`os.system`
629^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000630
631::
632
633 sts = os.system("mycmd" + " myarg")
Nick Coghlan86711572011-10-24 22:19:40 +1000634 # becomes
635 sts = call("mycmd" + " myarg", shell=True)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000636
637Notes:
638
639* Calling the program through the shell is usually not required.
640
641* It's easier to look at the :attr:`returncode` attribute than the exit status.
642
643A more realistic example would look like this::
644
645 try:
646 retcode = call("mycmd" + " myarg", shell=True)
647 if retcode < 0:
648 print >>sys.stderr, "Child was terminated by signal", -retcode
649 else:
650 print >>sys.stderr, "Child returned", retcode
651 except OSError, e:
652 print >>sys.stderr, "Execution failed:", e
653
654
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000655Replacing the :func:`os.spawn <os.spawnl>` family
656^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000657
658P_NOWAIT example::
659
660 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
661 ==>
662 pid = Popen(["/bin/mycmd", "myarg"]).pid
663
664P_WAIT example::
665
666 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
667 ==>
668 retcode = call(["/bin/mycmd", "myarg"])
669
670Vector example::
671
672 os.spawnvp(os.P_NOWAIT, path, args)
673 ==>
674 Popen([path] + args[1:])
675
676Environment example::
677
678 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
679 ==>
680 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
681
682
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000683Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3`
684^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000685
686::
687
Philip Jenvey8b902042009-09-29 19:10:15 +0000688 pipe = os.popen("cmd", 'r', bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000689 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000690 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout
Georg Brandl8ec7f652007-08-15 14:28:01 +0000691
692::
693
Philip Jenvey8b902042009-09-29 19:10:15 +0000694 pipe = os.popen("cmd", 'w', bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000695 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000696 pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin
Georg Brandl8ec7f652007-08-15 14:28:01 +0000697
698::
699
Philip Jenvey8b902042009-09-29 19:10:15 +0000700 (child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000701 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000702 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000703 stdin=PIPE, stdout=PIPE, close_fds=True)
704 (child_stdin, child_stdout) = (p.stdin, p.stdout)
705
706::
707
708 (child_stdin,
709 child_stdout,
Philip Jenvey8b902042009-09-29 19:10:15 +0000710 child_stderr) = os.popen3("cmd", mode, bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000711 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000712 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000713 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
714 (child_stdin,
715 child_stdout,
716 child_stderr) = (p.stdin, p.stdout, p.stderr)
717
718::
719
Philip Jenvey8b902042009-09-29 19:10:15 +0000720 (child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode,
721 bufsize)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000722 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000723 p = Popen("cmd", shell=True, bufsize=bufsize,
Georg Brandl8ec7f652007-08-15 14:28:01 +0000724 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
725 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
726
Philip Jenvey8b902042009-09-29 19:10:15 +0000727On Unix, os.popen2, os.popen3 and os.popen4 also accept a sequence as
728the command to execute, in which case arguments will be passed
729directly to the program without shell intervention. This usage can be
730replaced as follows::
731
732 (child_stdin, child_stdout) = os.popen2(["/bin/ls", "-l"], mode,
733 bufsize)
734 ==>
735 p = Popen(["/bin/ls", "-l"], bufsize=bufsize, stdin=PIPE, stdout=PIPE)
736 (child_stdin, child_stdout) = (p.stdin, p.stdout)
737
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000738Return code handling translates as follows::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000739
Philip Jenvey8b902042009-09-29 19:10:15 +0000740 pipe = os.popen("cmd", 'w')
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000741 ...
742 rc = pipe.close()
Stefan Kraha253dc12010-07-14 10:06:07 +0000743 if rc is not None and rc >> 8:
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000744 print "There were some errors"
745 ==>
Philip Jenvey8b902042009-09-29 19:10:15 +0000746 process = Popen("cmd", 'w', shell=True, stdin=PIPE)
R. David Murrayccb9d4b2009-06-09 00:44:22 +0000747 ...
748 process.stdin.close()
749 if process.wait() != 0:
750 print "There were some errors"
751
752
753Replacing functions from the :mod:`popen2` module
754^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl8ec7f652007-08-15 14:28:01 +0000755
Georg Brandl8ec7f652007-08-15 14:28:01 +0000756::
757
758 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
759 ==>
760 p = Popen(["somestring"], shell=True, bufsize=bufsize,
761 stdin=PIPE, stdout=PIPE, close_fds=True)
762 (child_stdout, child_stdin) = (p.stdout, p.stdin)
763
Philip Jenvey8b902042009-09-29 19:10:15 +0000764On Unix, popen2 also accepts a sequence as the command to execute, in
765which case arguments will be passed directly to the program without
766shell intervention. This usage can be replaced as follows::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000767
Philip Jenvey8b902042009-09-29 19:10:15 +0000768 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize,
769 mode)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000770 ==>
771 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
772 stdin=PIPE, stdout=PIPE, close_fds=True)
773 (child_stdout, child_stdin) = (p.stdout, p.stdin)
774
Georg Brandlf5d5a662008-12-06 11:57:12 +0000775:class:`popen2.Popen3` and :class:`popen2.Popen4` basically work as
776:class:`subprocess.Popen`, except that:
Georg Brandl8ec7f652007-08-15 14:28:01 +0000777
Georg Brandlf5d5a662008-12-06 11:57:12 +0000778* :class:`Popen` raises an exception if the execution fails.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000779
780* the *capturestderr* argument is replaced with the *stderr* argument.
781
Georg Brandlf5d5a662008-12-06 11:57:12 +0000782* ``stdin=PIPE`` and ``stdout=PIPE`` must be specified.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000783
784* popen2 closes all file descriptors by default, but you have to specify
Georg Brandlf5d5a662008-12-06 11:57:12 +0000785 ``close_fds=True`` with :class:`Popen`.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000786
Eli Bendersky929e2762011-04-15 07:35:06 +0300787Notes
788-----
789
790.. _converting-argument-sequence:
791
792Converting an argument sequence to a string on Windows
793^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
794
795On Windows, an *args* sequence is converted to a string that can be parsed
796using the following rules (which correspond to the rules used by the MS C
797runtime):
798
7991. Arguments are delimited by white space, which is either a
800 space or a tab.
801
8022. A string surrounded by double quotation marks is
803 interpreted as a single argument, regardless of white space
804 contained within. A quoted string can be embedded in an
805 argument.
806
8073. A double quotation mark preceded by a backslash is
808 interpreted as a literal double quotation mark.
809
8104. Backslashes are interpreted literally, unless they
811 immediately precede a double quotation mark.
812
8135. If backslashes immediately precede a double quotation mark,
814 every pair of backslashes is interpreted as a literal
815 backslash. If the number of backslashes is odd, the last
816 backslash escapes the next double quotation mark as
817 described in rule 3.
818