Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 1 | .. currentmodule:: asyncio |
| 2 | |
Victor Stinner | 778015b | 2014-07-11 12:13:39 +0200 | [diff] [blame] | 3 | .. _asyncio-subprocess: |
| 4 | |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 5 | Subprocess |
| 6 | ========== |
| 7 | |
lf | 627d2c8 | 2017-07-25 17:03:51 -0600 | [diff] [blame] | 8 | **Source code:** :source:`Lib/asyncio/subprocess.py` |
| 9 | |
Victor Stinner | aea8229 | 2014-07-08 23:42:38 +0200 | [diff] [blame] | 10 | Windows event loop |
| 11 | ------------------ |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 12 | |
Victor Stinner | aea8229 | 2014-07-08 23:42:38 +0200 | [diff] [blame] | 13 | On Windows, the default event loop is :class:`SelectorEventLoop` which does not |
| 14 | support subprocesses. :class:`ProactorEventLoop` should be used instead. |
| 15 | Example to use it on Windows:: |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 16 | |
Guido van Rossum | 8778c6b | 2015-11-02 09:15:47 -0800 | [diff] [blame] | 17 | import asyncio, sys |
Victor Stinner | aea8229 | 2014-07-08 23:42:38 +0200 | [diff] [blame] | 18 | |
Guido van Rossum | 8778c6b | 2015-11-02 09:15:47 -0800 | [diff] [blame] | 19 | if sys.platform == 'win32': |
Victor Stinner | aea8229 | 2014-07-08 23:42:38 +0200 | [diff] [blame] | 20 | loop = asyncio.ProactorEventLoop() |
| 21 | asyncio.set_event_loop(loop) |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 22 | |
Victor Stinner | 778015b | 2014-07-11 12:13:39 +0200 | [diff] [blame] | 23 | .. seealso:: |
| 24 | |
| 25 | :ref:`Available event loops <asyncio-event-loops>` and :ref:`Platform |
| 26 | support <asyncio-platform-support>`. |
| 27 | |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 28 | |
| 29 | Create a subprocess: high-level API using Process |
| 30 | ------------------------------------------------- |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 31 | |
Victor Stinner | bdd574d | 2015-02-12 22:49:18 +0100 | [diff] [blame] | 32 | .. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 33 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 34 | Create a subprocess. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 35 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 36 | The *limit* parameter sets the buffer limit passed to the |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 37 | :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_exec` for other |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 38 | parameters. |
| 39 | |
| 40 | Return a :class:`~asyncio.subprocess.Process` instance. |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 41 | |
Victor Stinner | 2fb3b82 | 2014-03-13 10:58:03 +0100 | [diff] [blame] | 42 | This function is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 43 | |
Victor Stinner | bdd574d | 2015-02-12 22:49:18 +0100 | [diff] [blame] | 44 | .. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 45 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 46 | Run the shell command *cmd*. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 47 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 48 | The *limit* parameter sets the buffer limit passed to the |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 49 | :class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_shell` for other |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 50 | parameters. |
| 51 | |
| 52 | Return a :class:`~asyncio.subprocess.Process` instance. |
| 53 | |
| 54 | It is the application's responsibility to ensure that all whitespace and |
| 55 | metacharacters are quoted appropriately to avoid `shell injection |
Georg Brandl | 5d94134 | 2016-02-26 19:37:12 +0100 | [diff] [blame] | 56 | <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 57 | vulnerabilities. The :func:`shlex.quote` function can be used to properly |
| 58 | escape whitespace and shell metacharacters in strings that are going to be |
| 59 | used to construct shell commands. |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 60 | |
Victor Stinner | 2fb3b82 | 2014-03-13 10:58:03 +0100 | [diff] [blame] | 61 | This function is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 62 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 63 | Use the :meth:`AbstractEventLoop.connect_read_pipe` and |
| 64 | :meth:`AbstractEventLoop.connect_write_pipe` methods to connect pipes. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 65 | |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 66 | |
| 67 | Create a subprocess: low-level API using subprocess.Popen |
| 68 | --------------------------------------------------------- |
| 69 | |
| 70 | Run subprocesses asynchronously using the :mod:`subprocess` module. |
| 71 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 72 | .. coroutinemethod:: AbstractEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 73 | |
Victor Stinner | 6bfd854 | 2014-06-19 12:50:27 +0200 | [diff] [blame] | 74 | Create a subprocess from one or more string arguments (character strings or |
| 75 | bytes strings encoded to the :ref:`filesystem encoding |
| 76 | <filesystem-encoding>`), where the first string |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 77 | specifies the program to execute, and the remaining strings specify the |
| 78 | program's arguments. (Thus, together the string arguments form the |
| 79 | ``sys.argv`` value of the program, assuming it is a Python script.) This is |
| 80 | similar to the standard library :class:`subprocess.Popen` class called with |
| 81 | shell=False and the list of strings passed as the first argument; |
| 82 | however, where :class:`~subprocess.Popen` takes a single argument which is |
| 83 | list of strings, :func:`subprocess_exec` takes multiple string arguments. |
| 84 | |
J. W | 6abaed0 | 2017-03-04 22:51:08 +0000 | [diff] [blame] | 85 | The *protocol_factory* must instantiate a subclass of the |
Victor Stinner | 3c95062 | 2014-10-14 00:02:10 +0200 | [diff] [blame] | 86 | :class:`asyncio.SubprocessProtocol` class. |
| 87 | |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 88 | Other parameters: |
| 89 | |
| 90 | * *stdin*: Either a file-like object representing the pipe to be connected |
| 91 | to the subprocess's standard input stream using |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 92 | :meth:`~AbstractEventLoop.connect_write_pipe`, or the constant |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 93 | :const:`subprocess.PIPE` (the default). By default a new pipe will be |
| 94 | created and connected. |
| 95 | |
| 96 | * *stdout*: Either a file-like object representing the pipe to be connected |
| 97 | to the subprocess's standard output stream using |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 98 | :meth:`~AbstractEventLoop.connect_read_pipe`, or the constant |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 99 | :const:`subprocess.PIPE` (the default). By default a new pipe will be |
| 100 | created and connected. |
| 101 | |
| 102 | * *stderr*: Either a file-like object representing the pipe to be connected |
| 103 | to the subprocess's standard error stream using |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 104 | :meth:`~AbstractEventLoop.connect_read_pipe`, or one of the constants |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 105 | :const:`subprocess.PIPE` (the default) or :const:`subprocess.STDOUT`. |
| 106 | By default a new pipe will be created and connected. When |
| 107 | :const:`subprocess.STDOUT` is specified, the subprocess's standard error |
| 108 | stream will be connected to the same pipe as the standard output stream. |
| 109 | |
| 110 | * All other keyword arguments are passed to :class:`subprocess.Popen` |
| 111 | without interpretation, except for *bufsize*, *universal_newlines* and |
| 112 | *shell*, which should not be specified at all. |
| 113 | |
| 114 | Returns a pair of ``(transport, protocol)``, where *transport* is an |
| 115 | instance of :class:`BaseSubprocessTransport`. |
| 116 | |
| 117 | This method is a :ref:`coroutine <coroutine>`. |
| 118 | |
| 119 | See the constructor of the :class:`subprocess.Popen` class for parameters. |
| 120 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 121 | .. coroutinemethod:: AbstractEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 122 | |
Victor Stinner | 6bfd854 | 2014-06-19 12:50:27 +0200 | [diff] [blame] | 123 | Create a subprocess from *cmd*, which is a character string or a bytes |
| 124 | string encoded to the :ref:`filesystem encoding <filesystem-encoding>`, |
| 125 | using the platform's "shell" syntax. This is similar to the standard library |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 126 | :class:`subprocess.Popen` class called with ``shell=True``. |
| 127 | |
J. W | 6abaed0 | 2017-03-04 22:51:08 +0000 | [diff] [blame] | 128 | The *protocol_factory* must instantiate a subclass of the |
Victor Stinner | 3c95062 | 2014-10-14 00:02:10 +0200 | [diff] [blame] | 129 | :class:`asyncio.SubprocessProtocol` class. |
| 130 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 131 | See :meth:`~AbstractEventLoop.subprocess_exec` for more details about |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 132 | the remaining arguments. |
| 133 | |
| 134 | Returns a pair of ``(transport, protocol)``, where *transport* is an |
| 135 | instance of :class:`BaseSubprocessTransport`. |
| 136 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 137 | It is the application's responsibility to ensure that all whitespace and |
| 138 | metacharacters are quoted appropriately to avoid `shell injection |
Georg Brandl | 5d94134 | 2016-02-26 19:37:12 +0100 | [diff] [blame] | 139 | <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 140 | vulnerabilities. The :func:`shlex.quote` function can be used to properly |
| 141 | escape whitespace and shell metacharacters in strings that are going to be |
| 142 | used to construct shell commands. |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 143 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 144 | This method is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 984600f | 2014-03-25 09:40:26 +0100 | [diff] [blame] | 145 | |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 146 | .. seealso:: |
| 147 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 148 | The :meth:`AbstractEventLoop.connect_read_pipe` and |
| 149 | :meth:`AbstractEventLoop.connect_write_pipe` methods. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 150 | |
| 151 | |
| 152 | Constants |
| 153 | --------- |
| 154 | |
| 155 | .. data:: asyncio.subprocess.PIPE |
| 156 | |
| 157 | Special value that can be used as the *stdin*, *stdout* or *stderr* argument |
| 158 | to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and |
| 159 | indicates that a pipe to the standard stream should be opened. |
| 160 | |
| 161 | .. data:: asyncio.subprocess.STDOUT |
| 162 | |
| 163 | Special value that can be used as the *stderr* argument to |
| 164 | :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and |
| 165 | indicates that standard error should go into the same handle as standard |
| 166 | output. |
| 167 | |
| 168 | .. data:: asyncio.subprocess.DEVNULL |
| 169 | |
Victor Stinner | 7280486 | 2014-03-21 11:44:49 +0100 | [diff] [blame] | 170 | Special value that can be used as the *stdin*, *stdout* or *stderr* argument |
| 171 | to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and |
| 172 | indicates that the special file :data:`os.devnull` will be used. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 173 | |
| 174 | |
| 175 | Process |
| 176 | ------- |
| 177 | |
| 178 | .. class:: asyncio.subprocess.Process |
| 179 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 180 | A subprocess created by the :func:`create_subprocess_exec` or the |
| 181 | :func:`create_subprocess_shell` function. |
Victor Stinner | b79eb05 | 2014-02-03 23:08:14 +0100 | [diff] [blame] | 182 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 183 | The API of the :class:`~asyncio.subprocess.Process` class was designed to be |
R David Murray | 2249d9f | 2015-05-14 08:50:38 -0400 | [diff] [blame] | 184 | close to the API of the :class:`subprocess.Popen` class, but there are some |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 185 | differences: |
Victor Stinner | b79eb05 | 2014-02-03 23:08:14 +0100 | [diff] [blame] | 186 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 187 | * There is no explicit :meth:`~subprocess.Popen.poll` method |
| 188 | * The :meth:`~subprocess.Popen.communicate` and |
| 189 | :meth:`~subprocess.Popen.wait` methods don't take a *timeout* parameter: |
| 190 | use the :func:`wait_for` function |
| 191 | * The *universal_newlines* parameter is not supported (only bytes strings |
| 192 | are supported) |
| 193 | * The :meth:`~asyncio.subprocess.Process.wait` method of |
| 194 | the :class:`~asyncio.subprocess.Process` class is asynchronous whereas the |
| 195 | :meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen` |
| 196 | class is implemented as a busy loop. |
Victor Stinner | b79eb05 | 2014-02-03 23:08:14 +0100 | [diff] [blame] | 197 | |
Victor Stinner | 8370496 | 2015-02-25 14:24:15 +0100 | [diff] [blame] | 198 | This class is :ref:`not thread safe <asyncio-multithreading>`. See also the |
| 199 | :ref:`Subprocess and threads <asyncio-subprocess-threads>` section. |
| 200 | |
Victor Stinner | bdd574d | 2015-02-12 22:49:18 +0100 | [diff] [blame] | 201 | .. coroutinemethod:: wait() |
Victor Stinner | b79eb05 | 2014-02-03 23:08:14 +0100 | [diff] [blame] | 202 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 203 | Wait for child process to terminate. Set and return :attr:`returncode` |
| 204 | attribute. |
Victor Stinner | b79eb05 | 2014-02-03 23:08:14 +0100 | [diff] [blame] | 205 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 206 | This method is a :ref:`coroutine <coroutine>`. |
Victor Stinner | b79eb05 | 2014-02-03 23:08:14 +0100 | [diff] [blame] | 207 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 208 | .. note:: |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 209 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 210 | This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` and |
| 211 | the child process generates enough output to a pipe such that it |
| 212 | blocks waiting for the OS pipe buffer to accept more data. Use the |
| 213 | :meth:`communicate` method when using pipes to avoid that. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 214 | |
Victor Stinner | bdd574d | 2015-02-12 22:49:18 +0100 | [diff] [blame] | 215 | .. coroutinemethod:: communicate(input=None) |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 216 | |
| 217 | Interact with process: Send data to stdin. Read data from stdout and |
| 218 | stderr, until end-of-file is reached. Wait for process to terminate. |
| 219 | The optional *input* argument should be data to be sent to the child |
| 220 | process, or ``None``, if no data should be sent to the child. The type |
| 221 | of *input* must be bytes. |
| 222 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 223 | :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. |
| 224 | |
Victor Stinner | d55b54d | 2014-07-17 13:12:03 +0200 | [diff] [blame] | 225 | If a :exc:`BrokenPipeError` or :exc:`ConnectionResetError` exception is |
| 226 | raised when writing *input* into stdin, the exception is ignored. It |
| 227 | occurs when the process exits before all data are written into stdin. |
Victor Stinner | cc996b5 | 2014-07-17 12:25:27 +0200 | [diff] [blame] | 228 | |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 229 | Note that if you want to send data to the process's stdin, you need to |
Victor Stinner | 0c3949c | 2014-02-09 02:51:40 +0100 | [diff] [blame] | 230 | create the Process object with ``stdin=PIPE``. Similarly, to get anything |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 231 | other than ``None`` in the result tuple, you need to give ``stdout=PIPE`` |
| 232 | and/or ``stderr=PIPE`` too. |
| 233 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 234 | This method is a :ref:`coroutine <coroutine>`. |
| 235 | |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 236 | .. note:: |
| 237 | |
| 238 | The data read is buffered in memory, so do not use this method if the |
| 239 | data size is large or unlimited. |
| 240 | |
Victor Stinner | cc996b5 | 2014-07-17 12:25:27 +0200 | [diff] [blame] | 241 | .. versionchanged:: 3.4.2 |
Victor Stinner | d55b54d | 2014-07-17 13:12:03 +0200 | [diff] [blame] | 242 | The method now ignores :exc:`BrokenPipeError` and |
| 243 | :exc:`ConnectionResetError`. |
Victor Stinner | cc996b5 | 2014-07-17 12:25:27 +0200 | [diff] [blame] | 244 | |
Brian Curtin | a1afeec | 2014-02-08 18:36:14 -0600 | [diff] [blame] | 245 | .. method:: send_signal(signal) |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 246 | |
| 247 | Sends the signal *signal* to the child process. |
| 248 | |
| 249 | .. note:: |
| 250 | |
| 251 | On Windows, :py:data:`SIGTERM` is an alias for :meth:`terminate`. |
| 252 | ``CTRL_C_EVENT`` and ``CTRL_BREAK_EVENT`` can be sent to processes |
| 253 | started with a *creationflags* parameter which includes |
| 254 | ``CREATE_NEW_PROCESS_GROUP``. |
| 255 | |
| 256 | .. method:: terminate() |
| 257 | |
| 258 | Stop the child. On Posix OSs the method sends :py:data:`signal.SIGTERM` |
| 259 | to the child. On Windows the Win32 API function |
| 260 | :c:func:`TerminateProcess` is called to stop the child. |
| 261 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 262 | .. method:: kill() |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 263 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 264 | Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to |
| 265 | the child. On Windows :meth:`kill` is an alias for :meth:`terminate`. |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 266 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 267 | .. attribute:: stdin |
| 268 | |
| 269 | Standard input stream (:class:`StreamWriter`), ``None`` if the process |
| 270 | was created with ``stdin=None``. |
| 271 | |
| 272 | .. attribute:: stdout |
| 273 | |
| 274 | Standard output stream (:class:`StreamReader`), ``None`` if the process |
| 275 | was created with ``stdout=None``. |
| 276 | |
| 277 | .. attribute:: stderr |
| 278 | |
| 279 | Standard error stream (:class:`StreamReader`), ``None`` if the process |
| 280 | was created with ``stderr=None``. |
| 281 | |
| 282 | .. warning:: |
| 283 | |
| 284 | Use the :meth:`communicate` method rather than :attr:`.stdin.write |
| 285 | <stdin>`, :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` |
| 286 | to avoid deadlocks due to streams pausing reading or writing and blocking |
| 287 | the child process. |
| 288 | |
| 289 | .. attribute:: pid |
| 290 | |
| 291 | The identifier of the process. |
| 292 | |
| 293 | Note that for processes created by the :func:`create_subprocess_shell` |
| 294 | function, this attribute is the process identifier of the spawned shell. |
| 295 | |
| 296 | .. attribute:: returncode |
| 297 | |
| 298 | Return code of the process when it exited. A ``None`` value indicates |
| 299 | that the process has not terminated yet. |
| 300 | |
| 301 | A negative value ``-N`` indicates that the child was terminated by signal |
| 302 | ``N`` (Unix only). |
Victor Stinner | 7bdf786 | 2014-03-16 21:29:31 +0100 | [diff] [blame] | 303 | |
Victor Stinner | e48d4db | 2014-02-03 23:26:28 +0100 | [diff] [blame] | 304 | |
Victor Stinner | 399c59d | 2015-01-09 01:32:02 +0100 | [diff] [blame] | 305 | .. _asyncio-subprocess-threads: |
| 306 | |
| 307 | Subprocess and threads |
Victor Stinner | 5492d35 | 2015-09-02 15:39:01 +0200 | [diff] [blame] | 308 | ---------------------- |
Victor Stinner | 399c59d | 2015-01-09 01:32:02 +0100 | [diff] [blame] | 309 | |
| 310 | asyncio supports running subprocesses from different threads, but there |
| 311 | are limits: |
| 312 | |
| 313 | * An event loop must run in the main thread |
| 314 | * The child watcher must be instantiated in the main thread, before executing |
| 315 | subprocesses from other threads. Call the :func:`get_child_watcher` |
| 316 | function in the main thread to instantiate the child watcher. |
| 317 | |
Victor Stinner | 8370496 | 2015-02-25 14:24:15 +0100 | [diff] [blame] | 318 | The :class:`asyncio.subprocess.Process` class is not thread safe. |
| 319 | |
Victor Stinner | 399c59d | 2015-01-09 01:32:02 +0100 | [diff] [blame] | 320 | .. seealso:: |
| 321 | |
| 322 | The :ref:`Concurrency and multithreading in asyncio |
| 323 | <asyncio-multithreading>` section. |
| 324 | |
| 325 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 326 | Subprocess examples |
Victor Stinner | 5492d35 | 2015-09-02 15:39:01 +0200 | [diff] [blame] | 327 | ------------------- |
Victor Stinner | e48d4db | 2014-02-03 23:26:28 +0100 | [diff] [blame] | 328 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 329 | Subprocess using transport and protocol |
Victor Stinner | 5492d35 | 2015-09-02 15:39:01 +0200 | [diff] [blame] | 330 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 331 | |
| 332 | Example of a subprocess protocol using to get the output of a subprocess and to |
| 333 | wait for the subprocess exit. The subprocess is created by the |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 334 | :meth:`AbstractEventLoop.subprocess_exec` method:: |
Victor Stinner | e48d4db | 2014-02-03 23:26:28 +0100 | [diff] [blame] | 335 | |
| 336 | import asyncio |
| 337 | import sys |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 338 | |
| 339 | class DateProtocol(asyncio.SubprocessProtocol): |
| 340 | def __init__(self, exit_future): |
| 341 | self.exit_future = exit_future |
| 342 | self.output = bytearray() |
| 343 | |
| 344 | def pipe_data_received(self, fd, data): |
| 345 | self.output.extend(data) |
| 346 | |
| 347 | def process_exited(self): |
| 348 | self.exit_future.set_result(True) |
Victor Stinner | e48d4db | 2014-02-03 23:26:28 +0100 | [diff] [blame] | 349 | |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 350 | async def get_date(loop): |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 351 | code = 'import datetime; print(datetime.datetime.now())' |
| 352 | exit_future = asyncio.Future(loop=loop) |
Victor Stinner | e48d4db | 2014-02-03 23:26:28 +0100 | [diff] [blame] | 353 | |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 354 | # Create the subprocess controlled by the protocol DateProtocol, |
| 355 | # redirect the standard output into a pipe |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 356 | transport, protocol = await loop.subprocess_exec( |
| 357 | lambda: DateProtocol(exit_future), |
| 358 | sys.executable, '-c', code, |
| 359 | stdin=None, stderr=None) |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 360 | |
| 361 | # Wait for the subprocess exit using the process_exited() method |
| 362 | # of the protocol |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 363 | await exit_future |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 364 | |
| 365 | # Close the stdout pipe |
| 366 | transport.close() |
| 367 | |
| 368 | # Read the output which was collected by the pipe_data_received() |
| 369 | # method of the protocol |
| 370 | data = bytes(protocol.output) |
| 371 | return data.decode('ascii').rstrip() |
| 372 | |
| 373 | if sys.platform == "win32": |
Victor Stinner | 6bc2396 | 2014-03-21 11:56:40 +0100 | [diff] [blame] | 374 | loop = asyncio.ProactorEventLoop() |
| 375 | asyncio.set_event_loop(loop) |
| 376 | else: |
| 377 | loop = asyncio.get_event_loop() |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 378 | |
| 379 | date = loop.run_until_complete(get_date(loop)) |
| 380 | print("Current date: %s" % date) |
| 381 | loop.close() |
| 382 | |
| 383 | |
| 384 | Subprocess using streams |
Victor Stinner | 5492d35 | 2015-09-02 15:39:01 +0200 | [diff] [blame] | 385 | ^^^^^^^^^^^^^^^^^^^^^^^^ |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 386 | |
| 387 | Example using the :class:`~asyncio.subprocess.Process` class to control the |
| 388 | subprocess and the :class:`StreamReader` class to read from the standard |
| 389 | output. The subprocess is created by the :func:`create_subprocess_exec` |
| 390 | function:: |
| 391 | |
| 392 | import asyncio.subprocess |
| 393 | import sys |
| 394 | |
Mikhail Terekhov | d2ac400 | 2018-08-07 16:29:06 -0400 | [diff] [blame] | 395 | async def get_date(): |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 396 | code = 'import datetime; print(datetime.datetime.now())' |
| 397 | |
| 398 | # Create the subprocess, redirect the standard output into a pipe |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 399 | proc = await asyncio.create_subprocess_exec( |
| 400 | sys.executable, '-c', code, |
| 401 | stdout=asyncio.subprocess.PIPE) |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 402 | |
| 403 | # Read one line of output |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 404 | data = await proc.stdout.readline() |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 405 | line = data.decode('ascii').rstrip() |
| 406 | |
| 407 | # Wait for the subprocess exit |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 408 | await proc.wait() |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 409 | return line |
| 410 | |
| 411 | if sys.platform == "win32": |
| 412 | loop = asyncio.ProactorEventLoop() |
| 413 | asyncio.set_event_loop(loop) |
Victor Stinner | e48d4db | 2014-02-03 23:26:28 +0100 | [diff] [blame] | 414 | else: |
Victor Stinner | 3989205 | 2014-10-14 00:52:07 +0200 | [diff] [blame] | 415 | loop = asyncio.get_event_loop() |
| 416 | |
| 417 | date = loop.run_until_complete(get_date()) |
| 418 | print("Current date: %s" % date) |
Victor Stinner | e48d4db | 2014-02-03 23:26:28 +0100 | [diff] [blame] | 419 | loop.close() |