Guido van Rossum | 7a46564 | 2013-11-22 11:47:22 -0800 | [diff] [blame] | 1 | :mod:`asyncio` -- Asynchronous I/O, event loop, coroutines and tasks |
| 2 | ==================================================================== |
| 3 | |
| 4 | .. module:: asyncio |
| 5 | :synopsis: Asynchronous I/O, event loop, coroutines and tasks. |
| 6 | |
| 7 | .. versionadded:: 3.4 |
| 8 | |
Guido van Rossum | f8d0ff9 | 2013-11-22 16:53:25 -0800 | [diff] [blame] | 9 | **Source code:** :source:`Lib/asyncio/` |
| 10 | |
| 11 | -------------- |
Guido van Rossum | 7a46564 | 2013-11-22 11:47:22 -0800 | [diff] [blame] | 12 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 13 | This module provides infrastructure for writing single-threaded concurrent |
| 14 | code using coroutines, multiplexing I/O access over sockets and other |
| 15 | resources, running network clients and servers, and other related primitives. |
Guido van Rossum | 7a46564 | 2013-11-22 11:47:22 -0800 | [diff] [blame] | 16 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 17 | Here is a more detailed list of the package contents: |
| 18 | |
| 19 | * a pluggable :ref:`event loop <event-loop>` with various system-specific |
| 20 | implementations; |
| 21 | |
| 22 | * :ref:`transport <transport>` and :ref:`protocol <protocol>` abstractions |
| 23 | (similar to those in `Twisted <http://twistedmatrix.com/>`_); |
| 24 | |
| 25 | * concrete support for TCP, UDP, SSL, subprocess pipes, delayed calls, and |
| 26 | others (some may be system-dependent); |
| 27 | |
| 28 | * a Future class that mimicks the one in the :mod:`concurrent.futures` module, |
| 29 | but adapted for use with the event loop; |
| 30 | |
| 31 | * coroutines and tasks based on ``yield from`` (:PEP:`380`), to help write |
| 32 | concurrent code in a sequential fashion; |
| 33 | |
| 34 | * cancellation support for Futures and coroutines; |
| 35 | |
| 36 | * :ref:`synchronization primitives <sync>` for use between coroutines in |
| 37 | a single thread, mimicking those in the :mod:`threading` module; |
| 38 | |
Guido van Rossum | f0f5d38 | 2013-11-22 15:45:02 -0800 | [diff] [blame] | 39 | * an interface for passing work off to a threadpool, for times when |
| 40 | you absolutely, positively have to use a library that makes blocking |
| 41 | I/O calls. |
| 42 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 43 | |
| 44 | Disclaimer |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 45 | ========== |
Guido van Rossum | 7a46564 | 2013-11-22 11:47:22 -0800 | [diff] [blame] | 46 | |
| 47 | Full documentation is not yet ready; we hope to have it written |
| 48 | before Python 3.4 leaves beta. Until then, the best reference is |
| 49 | :PEP:`3156`. For a motivational primer on transports and protocols, |
| 50 | see :PEP:`3153`. |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 51 | |
| 52 | |
| 53 | .. XXX should the asyncio documentation come in several pages, as for logging? |
| 54 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 55 | .. _event-loop: |
| 56 | |
| 57 | Event loops |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 58 | =========== |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 59 | |
Antoine Pitrou | 9a62a19 | 2013-11-23 13:10:08 +0100 | [diff] [blame] | 60 | The event loop is the central execution device provided by :mod:`asyncio`. |
| 61 | It provides multiple facilities, amongst which: |
| 62 | |
| 63 | * Registering, executing and cancelling delayed calls (timeouts) |
| 64 | |
| 65 | * Creating client and server :ref:`transports <transport>` for various |
| 66 | kinds of communication |
| 67 | |
| 68 | * Launching subprocesses and the associated :ref:`transports <transport>` |
| 69 | for communication with an external program |
| 70 | |
| 71 | * Delegating costly function calls to a pool of threads |
| 72 | |
Victor Stinner | e91f180 | 2013-12-02 17:40:48 +0100 | [diff] [blame] | 73 | Event loop functions |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 74 | -------------------- |
Antoine Pitrou | 9a62a19 | 2013-11-23 13:10:08 +0100 | [diff] [blame] | 75 | |
| 76 | The easiest way to get an event loop is to call the :func:`get_event_loop` |
| 77 | function. |
| 78 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 79 | .. function:: get_event_loop() |
| 80 | |
| 81 | Get the event loop for current context. Returns an event loop object |
| 82 | implementing :class:`BaseEventLoop` interface, or raises an exception in case no |
| 83 | event loop has been set for the current context and the current policy does |
| 84 | not specify to create one. It should never return ``None``. |
| 85 | |
Victor Stinner | e91f180 | 2013-12-02 17:40:48 +0100 | [diff] [blame] | 86 | .. function:: set_event_loop(loop) |
| 87 | |
| 88 | XXX |
| 89 | |
| 90 | .. function:: new_event_loop() |
| 91 | |
| 92 | XXX |
| 93 | |
| 94 | |
| 95 | Event loop policy |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 96 | ----------------- |
Victor Stinner | e91f180 | 2013-12-02 17:40:48 +0100 | [diff] [blame] | 97 | |
| 98 | .. function:: get_event_loop_policy() |
| 99 | |
| 100 | XXX |
| 101 | |
| 102 | .. function:: set_event_loop_policy(policy) |
| 103 | |
| 104 | XXX |
| 105 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 106 | |
| 107 | Run an event loop |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 108 | ----------------- |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 109 | |
| 110 | .. method:: BaseEventLoop.run_forever() |
| 111 | |
| 112 | Run until :meth:`stop` is called. |
| 113 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 114 | .. method:: BaseEventLoop.run_until_complete(future) |
| 115 | |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 116 | Run until the :class:`~concurrent.futures.Future` is done. |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 117 | |
| 118 | If the argument is a coroutine, it is wrapped in a :class:`Task`. |
| 119 | |
| 120 | Return the Future's result, or raise its exception. |
| 121 | |
Victor Stinner | 3c3c4f5 | 2013-12-02 13:04:25 +0100 | [diff] [blame] | 122 | .. method:: BaseEventLoop.is_running() |
| 123 | |
| 124 | Returns running status of event loop. |
| 125 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 126 | .. method:: stop() |
| 127 | |
| 128 | Stop running the event loop. |
| 129 | |
| 130 | Every callback scheduled before :meth:`stop` is called will run. |
| 131 | Callback scheduled after :meth:`stop` is called won't. However, those |
| 132 | callbacks will run if :meth:`run_forever` is called again later. |
| 133 | |
| 134 | .. method:: BaseEventLoop.close() |
| 135 | |
Victor Stinner | e91f180 | 2013-12-02 17:40:48 +0100 | [diff] [blame] | 136 | Close the event loop. The loop should not be running. |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 137 | |
| 138 | This clears the queues and shuts down the executor, but does not wait for |
| 139 | the executor to finish. |
| 140 | |
Victor Stinner | e91f180 | 2013-12-02 17:40:48 +0100 | [diff] [blame] | 141 | This is idempotent and irreversible. No other methods should be called after |
| 142 | this one. |
| 143 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 144 | |
| 145 | Calls |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 146 | ----- |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 147 | |
| 148 | .. method:: BaseEventLoop.call_soon(callback, \*args) |
| 149 | |
| 150 | Arrange for a callback to be called as soon as possible. |
| 151 | |
| 152 | This operates as a FIFO queue, callbacks are called in the order in |
| 153 | which they are registered. Each callback will be called exactly once. |
| 154 | |
| 155 | Any positional arguments after the callback will be passed to the |
| 156 | callback when it is called. |
| 157 | |
Victor Stinner | 3c3c4f5 | 2013-12-02 13:04:25 +0100 | [diff] [blame] | 158 | .. method:: BaseEventLoop.call_soon_threadsafe(callback, \*args) |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 159 | |
| 160 | Like :meth:`call_soon`, but thread safe. |
| 161 | |
Antoine Pitrou | 9a62a19 | 2013-11-23 13:10:08 +0100 | [diff] [blame] | 162 | |
| 163 | Delayed calls |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 164 | ------------- |
Antoine Pitrou | 9a62a19 | 2013-11-23 13:10:08 +0100 | [diff] [blame] | 165 | |
| 166 | The event loop has its own internal clock for computing timeouts. |
| 167 | Which clock is used depends on the (platform-specific) event loop |
| 168 | implementation; ideally it is a monotonic clock. This will generally be |
| 169 | a different clock than :func:`time.time`. |
| 170 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 171 | .. method:: BaseEventLoop.call_later(delay, callback, *args) |
Antoine Pitrou | 9a62a19 | 2013-11-23 13:10:08 +0100 | [diff] [blame] | 172 | |
| 173 | Arrange for the *callback* to be called after the given *delay* |
| 174 | seconds (either an int or float). |
| 175 | |
| 176 | A "handle" is returned: an opaque object with a :meth:`cancel` method |
| 177 | that can be used to cancel the call. |
| 178 | |
| 179 | *callback* will be called exactly once per call to :meth:`call_later`. |
| 180 | If two callbacks are scheduled for exactly the same time, it is |
| 181 | undefined which will be called first. |
| 182 | |
| 183 | The optional positional *args* will be passed to the callback when it |
| 184 | is called. If you want the callback to be called with some named |
| 185 | arguments, use a closure or :func:`functools.partial`. |
| 186 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 187 | .. method:: BaseEventLoop.call_at(when, callback, *args) |
Antoine Pitrou | 9a62a19 | 2013-11-23 13:10:08 +0100 | [diff] [blame] | 188 | |
| 189 | Arrange for the *callback* to be called at the given absolute timestamp |
| 190 | *when* (an int or float), using the same time reference as :meth:`time`. |
| 191 | |
| 192 | This method's behavior is the same as :meth:`call_later`. |
| 193 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 194 | .. method:: BaseEventLoop.time() |
| 195 | |
Victor Stinner | 3c3c4f5 | 2013-12-02 13:04:25 +0100 | [diff] [blame] | 196 | Return the current time, as a :class:`float` value, according to the |
| 197 | event loop's internal clock. |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 198 | |
Victor Stinner | 3c3c4f5 | 2013-12-02 13:04:25 +0100 | [diff] [blame] | 199 | |
Antoine Pitrou | a30d82f | 2013-11-23 13:55:35 +0100 | [diff] [blame] | 200 | Creating connections |
| 201 | ^^^^^^^^^^^^^^^^^^^^ |
| 202 | |
Victor Stinner | dd339a2 | 2013-12-03 00:32:48 +0100 | [diff] [blame] | 203 | .. method:: BaseEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None) |
Antoine Pitrou | a30d82f | 2013-11-23 13:55:35 +0100 | [diff] [blame] | 204 | |
| 205 | Create a streaming transport connection to a given Internet *host* and |
| 206 | *port*. *protocol_factory* must be a callable returning a |
| 207 | :ref:`protocol <protocol>` instance. |
| 208 | |
| 209 | This method returns a :ref:`coroutine <coroutine>` which will try to |
| 210 | establish the connection in the background. When successful, the |
| 211 | coroutine returns a ``(transport, protocol)`` pair. |
| 212 | |
| 213 | The chronological synopsis of the underlying operation is as follows: |
| 214 | |
| 215 | #. The connection is established, and a :ref:`transport <transport>` |
| 216 | is created to represent it. |
| 217 | |
| 218 | #. *protocol_factory* is called without arguments and must return a |
| 219 | :ref:`protocol <protocol>` instance. |
| 220 | |
| 221 | #. The protocol instance is tied to the transport, and its |
| 222 | :meth:`connection_made` method is called. |
| 223 | |
| 224 | #. The coroutine returns successfully with the ``(transport, protocol)`` |
| 225 | pair. |
| 226 | |
| 227 | The created transport is an implementation-dependent bidirectional stream. |
| 228 | |
| 229 | .. note:: |
| 230 | *protocol_factory* can be any kind of callable, not necessarily |
| 231 | a class. For example, if you want to use a pre-created |
| 232 | protocol instance, you can pass ``lambda: my_protocol``. |
| 233 | |
Victor Stinner | dd339a2 | 2013-12-03 00:32:48 +0100 | [diff] [blame] | 234 | Options allowing to change how the connection is created: |
Antoine Pitrou | a30d82f | 2013-11-23 13:55:35 +0100 | [diff] [blame] | 235 | |
| 236 | * *ssl*: if given and not false, a SSL/TLS transport is created |
| 237 | (by default a plain TCP transport is created). If *ssl* is |
| 238 | a :class:`ssl.SSLContext` object, this context is used to create |
| 239 | the transport; if *ssl* is :const:`True`, a context with some |
| 240 | unspecified default settings is used. |
| 241 | |
| 242 | * *server_hostname*, is only for use together with *ssl*, |
| 243 | and sets or overrides the hostname that the target server's certificate |
| 244 | will be matched against. By default the value of the *host* argument |
| 245 | is used. If *host* is empty, there is no default and you must pass a |
| 246 | value for *server_hostname*. If *server_hostname* is an empty |
| 247 | string, hostname matching is disabled (which is a serious security |
| 248 | risk, allowing for man-in-the-middle-attacks). |
| 249 | |
| 250 | * *family*, *proto*, *flags* are the optional address family, protocol |
| 251 | and flags to be passed through to getaddrinfo() for *host* resolution. |
| 252 | If given, these should all be integers from the corresponding |
| 253 | :mod:`socket` module constants. |
| 254 | |
| 255 | * *sock*, if given, should be an existing, already connected |
| 256 | :class:`socket.socket` object to be used by the transport. |
| 257 | If *sock* is given, none of *host*, *port*, *family*, *proto*, *flags* |
| 258 | and *local_addr* should be specified. |
| 259 | |
| 260 | * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used |
| 261 | to bind the socket to locally. The *local_host* and *local_port* |
| 262 | are looked up using getaddrinfo(), similarly to *host* and *port*. |
| 263 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 264 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 265 | Creating listening connections |
| 266 | ------------------------------ |
| 267 | |
| 268 | .. method:: BaseEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None) |
| 269 | |
| 270 | A :ref:`coroutine <coroutine>` which creates a TCP server bound to host and |
| 271 | port. |
| 272 | |
| 273 | The return value is a :class:`AbstractServer` object which can be used to stop |
| 274 | the service. |
| 275 | |
| 276 | If *host* is an empty string or None all interfaces are assumed |
| 277 | and a list of multiple sockets will be returned (most likely |
| 278 | one for IPv4 and another one for IPv6). |
| 279 | |
| 280 | *family* can be set to either :data:`~socket.AF_INET` or |
| 281 | :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set |
| 282 | it will be determined from host (defaults to :data:`~socket.AF_UNSPEC`). |
| 283 | |
| 284 | *flags* is a bitmask for :meth:`getaddrinfo`. |
| 285 | |
| 286 | *sock* can optionally be specified in order to use a preexisting |
| 287 | socket object. |
| 288 | |
| 289 | *backlog* is the maximum number of queued connections passed to |
| 290 | :meth:`~socket.socket.listen` (defaults to 100). |
| 291 | |
| 292 | ssl can be set to an :class:`~ssl.SSLContext` to enable SSL over the |
| 293 | accepted connections. |
| 294 | |
| 295 | *reuse_address* tells the kernel to reuse a local socket in |
| 296 | TIME_WAIT state, without waiting for its natural timeout to |
| 297 | expire. If not specified will automatically be set to True on |
| 298 | UNIX. |
| 299 | |
| 300 | This method returns a :ref:`coroutine <coroutine>`. |
| 301 | |
| 302 | .. method:: BaseEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0) |
| 303 | |
| 304 | Create datagram connection. |
| 305 | |
| 306 | This method returns a :ref:`coroutine <coroutine>`. |
| 307 | |
| 308 | |
| 309 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 310 | Resolve name |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 311 | ------------ |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 312 | |
| 313 | .. method:: BaseEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) |
| 314 | |
| 315 | XXX |
| 316 | |
| 317 | .. method:: BaseEventLoop.getnameinfo(sockaddr, flags=0) |
| 318 | |
| 319 | XXX |
| 320 | |
| 321 | |
| 322 | Running subprocesses |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 323 | -------------------- |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 324 | |
Victor Stinner | 3c3c4f5 | 2013-12-02 13:04:25 +0100 | [diff] [blame] | 325 | Run subprocesses asynchronously using the :mod:`subprocess` module. |
| 326 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 327 | .. method:: BaseEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=False, shell=False, bufsize=0, \*\*kwargs) |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 328 | |
| 329 | XXX |
| 330 | |
Victor Stinner | 3c3c4f5 | 2013-12-02 13:04:25 +0100 | [diff] [blame] | 331 | This method returns a :ref:`coroutine <coroutine>`. |
| 332 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 333 | See the constructor of the :class:`subprocess.Popen` class for parameters. |
| 334 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 335 | .. method:: BaseEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=False, shell=True, bufsize=0, \*\*kwargs) |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 336 | |
| 337 | XXX |
| 338 | |
Victor Stinner | 3c3c4f5 | 2013-12-02 13:04:25 +0100 | [diff] [blame] | 339 | This method returns a :ref:`coroutine <coroutine>`. |
| 340 | |
Victor Stinner | 550a09e | 2013-12-02 12:41:54 +0100 | [diff] [blame] | 341 | See the constructor of the :class:`subprocess.Popen` class for parameters. |
| 342 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 343 | .. method:: BaseEventLoop.connect_read_pipe(protocol_factory, pipe) |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 344 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 345 | Register read pipe in eventloop. |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 346 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 347 | *protocol_factory* should instantiate object with :class:`Protocol` |
| 348 | interface. pipe is file-like object already switched to nonblocking. |
| 349 | Return pair (transport, protocol), where transport support |
| 350 | :class:`ReadTransport` interface. |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 351 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 352 | This method returns a :ref:`coroutine <coroutine>`. |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 353 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 354 | .. method:: BaseEventLoop.connect_write_pipe(protocol_factory, pipe) |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 355 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 356 | Register write pipe in eventloop. |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 357 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 358 | *protocol_factory* should instantiate object with :class:`BaseProtocol` |
| 359 | interface. Pipe is file-like object already switched to nonblocking. |
| 360 | Return pair (transport, protocol), where transport support |
| 361 | :class:`WriteTransport` interface. |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 362 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 363 | This method returns a :ref:`coroutine <coroutine>`. |
| 364 | |
| 365 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 366 | Executor |
| 367 | -------- |
| 368 | |
| 369 | Call a function in an :class:`~concurrent.futures.Executor` (pool of threads or |
| 370 | pool of processes). By default, an event loop uses a thread pool executor |
| 371 | (:class:`~concurrent.futures.ThreadPoolExecutor`). |
| 372 | |
| 373 | .. method:: BaseEventLoop.run_in_executor(executor, callback, \*args) |
| 374 | |
| 375 | Arrange for a callback to be called in the specified executor. |
| 376 | |
| 377 | *executor* is a :class:`~concurrent.futures.Executor` instance, |
| 378 | the default executor is used if *executor* is ``None``. |
| 379 | |
| 380 | .. method:: BaseEventLoop.set_default_executor(executor) |
| 381 | |
| 382 | Set the default executor used by :meth:`run_in_executor`. |
| 383 | |
| 384 | |
| 385 | Tasks and coroutines |
| 386 | ==================== |
| 387 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 388 | .. _coroutine: |
| 389 | |
| 390 | Coroutines |
| 391 | ---------- |
| 392 | |
| 393 | A coroutine is a generator that follows certain conventions. For |
| 394 | documentation purposes, all coroutines should be decorated with |
| 395 | ``@asyncio.coroutine``, but this cannot be strictly enforced. |
| 396 | |
| 397 | Coroutines use the ``yield from`` syntax introduced in :pep:`380`, |
| 398 | instead of the original ``yield`` syntax. |
| 399 | |
| 400 | The word "coroutine", like the word "generator", is used for two |
| 401 | different (though related) concepts: |
| 402 | |
| 403 | - The function that defines a coroutine (a function definition |
| 404 | decorated with ``asyncio.coroutine``). If disambiguation is needed |
| 405 | we will call this a *coroutine function*. |
| 406 | |
| 407 | - The object obtained by calling a coroutine function. This object |
| 408 | represents a computation or an I/O operation (usually a combination) |
| 409 | that will complete eventually. If disambiguation is needed we will |
| 410 | call it a *coroutine object*. |
| 411 | |
| 412 | Things a coroutine can do: |
| 413 | |
| 414 | - ``result = yield from future`` -- suspends the coroutine until the |
| 415 | future is done, then returns the future's result, or raises an |
| 416 | exception, which will be propagated. (If the future is cancelled, |
| 417 | it will raise a ``CancelledError`` exception.) Note that tasks are |
| 418 | futures, and everything said about futures also applies to tasks. |
| 419 | |
| 420 | - ``result = yield from coroutine`` -- wait for another coroutine to |
| 421 | produce a result (or raise an exception, which will be propagated). |
| 422 | The ``coroutine`` expression must be a *call* to another coroutine. |
| 423 | |
| 424 | - ``return expression`` -- produce a result to the coroutine that is |
| 425 | waiting for this one using ``yield from``. |
| 426 | |
| 427 | - ``raise exception`` -- raise an exception in the coroutine that is |
| 428 | waiting for this one using ``yield from``. |
| 429 | |
| 430 | Calling a coroutine does not start its code running -- it is just a |
| 431 | generator, and the coroutine object returned by the call is really a |
| 432 | generator object, which doesn't do anything until you iterate over it. |
| 433 | In the case of a coroutine object, there are two basic ways to start |
| 434 | it running: call ``yield from coroutine`` from another coroutine |
| 435 | (assuming the other coroutine is already running!), or convert it to a |
| 436 | :class:`Task`. |
| 437 | |
| 438 | Coroutines (and tasks) can only run when the event loop is running. |
| 439 | |
| 440 | |
| 441 | Task |
| 442 | ---- |
| 443 | |
| 444 | .. class:: Task(coro, \*, loop=None) |
| 445 | |
| 446 | A coroutine wrapped in a :class:`~concurrent.futures.Future`. |
| 447 | |
| 448 | .. classmethod:: all_tasks(loop=None) |
| 449 | |
| 450 | Return a set of all tasks for an event loop. |
| 451 | |
| 452 | By default all tasks for the current event loop are returned. |
| 453 | |
| 454 | .. method:: cancel() |
| 455 | |
| 456 | Cancel the task. |
| 457 | |
| 458 | .. method:: get_stack(self, \*, limit=None) |
| 459 | |
| 460 | Return the list of stack frames for this task's coroutine. |
| 461 | |
| 462 | If the coroutine is active, this returns the stack where it is suspended. |
| 463 | If the coroutine has completed successfully or was cancelled, this |
| 464 | returns an empty list. If the coroutine was terminated by an exception, |
| 465 | this returns the list of traceback frames. |
| 466 | |
| 467 | The frames are always ordered from oldest to newest. |
| 468 | |
| 469 | The optional limit gives the maximum nummber of frames to return; by |
| 470 | default all available frames are returned. Its meaning differs depending |
| 471 | on whether a stack or a traceback is returned: the newest frames of a |
| 472 | stack are returned, but the oldest frames of a traceback are returned. |
| 473 | (This matches the behavior of the traceback module.) |
| 474 | |
| 475 | For reasons beyond our control, only one stack frame is returned for a |
| 476 | suspended coroutine. |
| 477 | |
| 478 | .. method:: print_stack(\*, limit=None, file=None) |
| 479 | |
| 480 | Print the stack or traceback for this task's coroutine. |
| 481 | |
| 482 | This produces output similar to that of the traceback module, for the |
| 483 | frames retrieved by get_stack(). The limit argument is passed to |
| 484 | get_stack(). The file argument is an I/O stream to which the output |
| 485 | goes; by default it goes to sys.stderr. |
| 486 | |
| 487 | |
| 488 | Task functions |
| 489 | -------------- |
| 490 | |
| 491 | .. function:: as_completed(fs, *, loop=None, timeout=None) |
| 492 | |
| 493 | Return an iterator whose values, when waited for, are |
| 494 | :class:`~concurrent.futures.Future` instances. |
| 495 | |
| 496 | Raises :exc:`TimeoutError` if the timeout occurs before all Futures are done. |
| 497 | |
| 498 | Example:: |
| 499 | |
| 500 | for f in as_completed(fs): |
| 501 | result = yield from f # The 'yield from' may raise |
| 502 | # Use result |
| 503 | |
| 504 | .. note:: |
| 505 | |
| 506 | The futures ``f`` are not necessarily members of fs. |
| 507 | |
| 508 | .. function:: async(coro_or_future, *, loop=None) |
| 509 | |
| 510 | Wrap a :ref:`coroutine <coroutine>` in a future. |
| 511 | |
| 512 | If the argument is a :class:`~concurrent.futures.Future`, it is returned |
| 513 | directly. |
| 514 | |
| 515 | .. function:: gather(*coros_or_futures, loop=None, return_exceptions=False) |
| 516 | |
| 517 | Return a future aggregating results from the given coroutines or futures. |
| 518 | |
| 519 | All futures must share the same event loop. If all the tasks are done |
| 520 | successfully, the returned future's result is the list of results (in the |
| 521 | order of the original sequence, not necessarily the order of results |
| 522 | arrival). If *result_exception* is True, exceptions in the tasks are |
| 523 | treated the same as successful results, and gathered in the result list; |
| 524 | otherwise, the first raised exception will be immediately propagated to the |
| 525 | returned future. |
| 526 | |
| 527 | Cancellation: if the outer Future is cancelled, all children (that have not |
| 528 | completed yet) are also cancelled. If any child is cancelled, this is |
| 529 | treated as if it raised :exc:`~concurrent.futures.CancelledError` -- the |
| 530 | outer Future is *not* cancelled in this case. (This is to prevent the |
| 531 | cancellation of one child to cause other children to be cancelled.) |
| 532 | |
| 533 | .. function:: tasks.iscoroutinefunction(func) |
| 534 | |
| 535 | Return ``True`` if *func* is a decorated coroutine function. |
| 536 | |
| 537 | .. function:: tasks.iscoroutine(obj) |
| 538 | |
| 539 | Return ``True`` if *obj* is a coroutine object. |
| 540 | |
| 541 | .. function:: sleep(delay, result=None, \*, loop=None) |
| 542 | |
| 543 | Create a :ref:`coroutine <coroutine>` that completes after a given time |
| 544 | (in seconds). |
| 545 | |
| 546 | .. function:: shield(arg, \*, loop=None) |
| 547 | |
| 548 | Wait for a future, shielding it from cancellation. |
| 549 | |
| 550 | The statement:: |
| 551 | |
| 552 | res = yield from shield(something()) |
| 553 | |
| 554 | is exactly equivalent to the statement:: |
| 555 | |
| 556 | res = yield from something() |
| 557 | |
| 558 | *except* that if the coroutine containing it is cancelled, the task running |
| 559 | in ``something()`` is not cancelled. From the point of view of |
| 560 | ``something()``, the cancellation did not happen. But its caller is still |
| 561 | cancelled, so the yield-from expression still raises |
| 562 | :exc:`~concurrent.futures.CancelledError`. Note: If ``something()`` is |
| 563 | cancelled by other means this will still cancel ``shield()``. |
| 564 | |
| 565 | If you want to completely ignore cancellation (not recommended) you can |
| 566 | combine ``shield()`` with a try/except clause, as follows:: |
| 567 | |
| 568 | try: |
| 569 | res = yield from shield(something()) |
| 570 | except CancelledError: |
| 571 | res = None |
| 572 | |
| 573 | .. function:: wait(fs, \*, loop=None, timeout=None, return_when=ALL_COMPLETED) |
| 574 | |
| 575 | Wait for the Futures and coroutines given by fs to complete. Coroutines will |
| 576 | be wrapped in Tasks. Returns two sets of |
| 577 | :class:`~concurrent.futures.Future`: (done, pending). |
| 578 | |
| 579 | *timeout* can be used to control the maximum number of seconds to wait before |
| 580 | returning. *timeout* can be an int or float. If *timeout* is not specified |
| 581 | or ``None``, there is no limit to the wait time. |
| 582 | |
| 583 | *return_when* indicates when this function should return. It must be one of |
| 584 | the following constants of the :mod`concurrent.futures` module: |
| 585 | |
| 586 | .. tabularcolumns:: |l|L| |
| 587 | |
| 588 | +-----------------------------+----------------------------------------+ |
| 589 | | Constant | Description | |
| 590 | +=============================+========================================+ |
| 591 | | :const:`FIRST_COMPLETED` | The function will return when any | |
| 592 | | | future finishes or is cancelled. | |
| 593 | +-----------------------------+----------------------------------------+ |
| 594 | | :const:`FIRST_EXCEPTION` | The function will return when any | |
| 595 | | | future finishes by raising an | |
| 596 | | | exception. If no future raises an | |
| 597 | | | exception then it is equivalent to | |
| 598 | | | :const:`ALL_COMPLETED`. | |
| 599 | +-----------------------------+----------------------------------------+ |
| 600 | | :const:`ALL_COMPLETED` | The function will return when all | |
| 601 | | | futures finish or are cancelled. | |
| 602 | +-----------------------------+----------------------------------------+ |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 603 | |
| 604 | This function returns a :ref:`coroutine <coroutine>`. |
| 605 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 606 | Usage:: |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 607 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 608 | done, pending = yield from asyncio.wait(fs) |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 609 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 610 | .. note:: |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 611 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 612 | This does not raise :exc:`TimeoutError`! Futures that aren't done when |
| 613 | the timeout occurs are returned in the second set. |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 614 | |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 615 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 616 | .. _transport: |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 617 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 618 | Transports |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 619 | ========== |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 620 | |
| 621 | Transports are classed provided by :mod:`asyncio` in order to abstract |
| 622 | various kinds of communication channels. You generally won't instantiate |
| 623 | a transport yourself; instead, you will call a :class:`BaseEventLoop` method |
| 624 | which will create the transport and try to initiate the underlying |
| 625 | communication channel, calling you back when it succeeds. |
| 626 | |
| 627 | Once the communication channel is established, a transport is always |
| 628 | paired with a :ref:`protocol <protocol>` instance. The protocol can |
| 629 | then call the transport's methods for various purposes. |
| 630 | |
| 631 | :mod:`asyncio` currently implements transports for TCP, UDP, SSL, and |
| 632 | subprocess pipes. The methods available on a transport depend on |
| 633 | the transport's kind. |
| 634 | |
| 635 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 636 | BaseTransport: Methods common to all transports |
| 637 | ----------------------------------------------- |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 638 | |
| 639 | .. class:: BaseTransport |
| 640 | |
| 641 | Base class for transports. |
| 642 | |
| 643 | .. method:: close(self) |
| 644 | |
| 645 | Close the transport. If the transport has a buffer for outgoing |
| 646 | data, buffered data will be flushed asynchronously. No more data |
| 647 | will be received. After all buffered data is flushed, the |
| 648 | protocol's :meth:`connection_lost` method will be called with |
| 649 | :const:`None` as its argument. |
| 650 | |
| 651 | |
| 652 | .. method:: get_extra_info(name, default=None) |
| 653 | |
| 654 | Return optional transport information. *name* is a string representing |
| 655 | the piece of transport-specific information to get, *default* is the |
| 656 | value to return if the information doesn't exist. |
| 657 | |
| 658 | This method allows transport implementations to easily expose |
| 659 | channel-specific information. |
| 660 | |
| 661 | * socket: |
| 662 | |
| 663 | - ``'peername'``: the remote address to which the socket is connected, |
| 664 | result of :meth:`socket.socket.getpeername` (``None`` on error) |
| 665 | - ``'socket'``: :class:`socket.socket` instance |
| 666 | - ``'sockname'``: the socket's own address, |
| 667 | result of :meth:`socket.socket.getsockname` |
| 668 | |
| 669 | * SSL socket: |
| 670 | |
| 671 | - ``'compression'``: the compression algorithm being used as a string, |
| 672 | or ``None`` if the connection isn't compressed; result of |
| 673 | :meth:`ssl.SSLSocket.compression` |
| 674 | - ``'cipher'``: a three-value tuple containing the name of the cipher |
| 675 | being used, the version of the SSL protocol that defines its use, and |
| 676 | the number of secret bits being used; result of |
| 677 | :meth:`ssl.SSLSocket.cipher` |
| 678 | - ``'peercert'``: peer certificate; result of |
| 679 | :meth:`ssl.SSLSocket.getpeercert` |
| 680 | - ``'sslcontext'``: :class:`ssl.SSLContext` instance |
| 681 | |
| 682 | * pipe: |
| 683 | |
| 684 | - ``'pipe'``: pipe object |
| 685 | |
| 686 | * subprocess: |
| 687 | |
| 688 | - ``'subprocess'``: :class:`subprocess.Popen` instance |
| 689 | |
| 690 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 691 | ReadTransport: Methods of readable streaming transports |
| 692 | ------------------------------------------------------- |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 693 | |
| 694 | .. class:: ReadTransport |
| 695 | |
| 696 | Interface for read-only transports. |
| 697 | |
| 698 | .. method:: pause_reading() |
| 699 | |
| 700 | Pause the receiving end of the transport. No data will be passed to |
| 701 | the protocol's :meth:`data_received` method until meth:`resume_reading` |
| 702 | is called. |
| 703 | |
| 704 | .. method:: resume_reading() |
| 705 | |
| 706 | Resume the receiving end. The protocol's :meth:`data_received` method |
| 707 | will be called once again if some data is available for reading. |
| 708 | |
| 709 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 710 | WriteTransport: Methods of writable streaming transports |
| 711 | -------------------------------------------------------- |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 712 | |
| 713 | .. class:: WriteTransport |
| 714 | |
| 715 | Interface for write-only transports. |
| 716 | |
| 717 | .. method:: abort() |
| 718 | |
| 719 | Close the transport immediately, without waiting for pending operations |
| 720 | to complete. Buffered data will be lost. No more data will be received. |
| 721 | The protocol's :meth:`connection_lost` method will eventually be |
| 722 | called with :const:`None` as its argument. |
| 723 | |
| 724 | .. method:: can_write_eof() |
| 725 | |
| 726 | Return :const:`True` if the transport supports :meth:`write_eof`, |
| 727 | :const:`False` if not. |
| 728 | |
| 729 | .. method:: get_write_buffer_size() |
| 730 | |
| 731 | Return the current size of the output buffer used by the transport. |
| 732 | |
| 733 | .. method:: set_write_buffer_limits(high=None, low=None) |
| 734 | |
| 735 | Set the *high*- and *low*-water limits for write flow control. |
| 736 | |
| 737 | These two values control when call the protocol's |
| 738 | :meth:`pause_writing` and :meth:`resume_writing` methods are called. |
| 739 | If specified, the low-water limit must be less than or equal to the |
| 740 | high-water limit. Neither *high* nor *low* can be negative. |
| 741 | |
| 742 | The defaults are implementation-specific. If only the |
| 743 | high-water limit is given, the low-water limit defaults to a |
| 744 | implementation-specific value less than or equal to the |
| 745 | high-water limit. Setting *high* to zero forces *low* to zero as |
| 746 | well, and causes :meth:`pause_writing` to be called whenever the |
| 747 | buffer becomes non-empty. Setting *low* to zero causes |
| 748 | :meth:`resume_writing` to be called only once the buffer is empty. |
| 749 | Use of zero for either limit is generally sub-optimal as it |
| 750 | reduces opportunities for doing I/O and computation |
| 751 | concurrently. |
| 752 | |
| 753 | .. method:: write(data) |
| 754 | |
| 755 | Write some *data* bytes to the transport. |
| 756 | |
| 757 | This method does not block; it buffers the data and arranges for it |
| 758 | to be sent out asynchronously. |
| 759 | |
| 760 | .. method:: writelines(list_of_data) |
| 761 | |
| 762 | Write a list (or any iterable) of data bytes to the transport. |
| 763 | This is functionally equivalent to calling :meth:`write` on each |
| 764 | element yielded by the iterable, but may be implemented more efficiently. |
| 765 | |
| 766 | .. method:: write_eof() |
| 767 | |
| 768 | Close the write end of the transport after flushing buffered data. |
| 769 | Data may still be received. |
| 770 | |
| 771 | This method can raise :exc:`NotImplementedError` if the transport |
| 772 | (e.g. SSL) doesn't support half-closes. |
| 773 | |
| 774 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 775 | DatagramTransport: Methods of datagram transports |
| 776 | ------------------------------------------------- |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 777 | |
| 778 | .. method:: DatagramTransport.sendto(data, addr=None) |
| 779 | |
| 780 | Send the *data* bytes to the remote peer given by *addr* (a |
| 781 | transport-dependent target address). If *addr* is :const:`None`, the |
| 782 | data is sent to the target address given on transport creation. |
| 783 | |
| 784 | This method does not block; it buffers the data and arranges for it |
| 785 | to be sent out asynchronously. |
| 786 | |
| 787 | .. method:: DatagramTransport.abort() |
| 788 | |
| 789 | Close the transport immediately, without waiting for pending operations |
| 790 | to complete. Buffered data will be lost. No more data will be received. |
| 791 | The protocol's :meth:`connection_lost` method will eventually be |
| 792 | called with :const:`None` as its argument. |
| 793 | |
| 794 | |
| 795 | Methods of subprocess transports |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 796 | -------------------------------- |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 797 | |
| 798 | .. class:: BaseSubprocessTransport |
| 799 | |
| 800 | .. method:: get_pid() |
| 801 | |
| 802 | Return the subprocess process id as an integer. |
| 803 | |
| 804 | .. method:: get_returncode() |
| 805 | |
| 806 | Return the subprocess returncode as an integer or :const:`None` |
| 807 | if it hasn't returned, similarly to the |
| 808 | :attr:`subprocess.Popen.returncode` attribute. |
| 809 | |
| 810 | .. method:: get_pipe_transport(fd) |
| 811 | |
| 812 | Return the transport for the communication pipe correspondong to the |
| 813 | integer file descriptor *fd*. The return value can be a readable or |
| 814 | writable streaming transport, depending on the *fd*. If *fd* doesn't |
| 815 | correspond to a pipe belonging to this transport, :const:`None` is |
| 816 | returned. |
| 817 | |
| 818 | .. method:: send_signal(signal) |
| 819 | |
| 820 | Send the *signal* number to the subprocess, as in |
| 821 | :meth:`subprocess.Popen.send_signal`. |
| 822 | |
| 823 | .. method:: terminate() |
| 824 | |
| 825 | Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`. |
| 826 | This method is an alias for the :meth:`close` method. |
| 827 | |
| 828 | On POSIX systems, this method sends SIGTERM to the subprocess. |
| 829 | On Windows, the Windows API function TerminateProcess() is called to |
| 830 | stop the subprocess. |
| 831 | |
| 832 | .. method:: kill(self) |
| 833 | |
| 834 | Kill the subprocess, as in :meth:`subprocess.Popen.kill` |
| 835 | |
| 836 | On POSIX systems, the function sends SIGKILL to the subprocess. |
| 837 | On Windows, this method is an alias for :meth:`terminate`. |
| 838 | |
| 839 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 840 | Stream reader |
| 841 | ------------- |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 842 | |
| 843 | .. class:: StreamWriter(transport, protocol, reader, loop) |
| 844 | |
| 845 | Wraps a Transport. |
| 846 | |
| 847 | This exposes :meth:`write`, :meth:`writelines`, :meth:`can_write_eof()`, :meth:`write_eof`, :meth:`get_extra_info` and |
| 848 | :meth:`close`. It adds :meth:`drain` which returns an optional :class:`~concurrent.futures.Future` on which you can |
| 849 | wait for flow control. It also adds a transport attribute which references |
| 850 | the :class:`Transport` directly. |
| 851 | |
| 852 | .. attribute:: transport |
| 853 | |
| 854 | Transport. |
| 855 | |
| 856 | .. method:: close() |
| 857 | |
| 858 | Close the transport: see :meth:`BaseTransport.close`. |
| 859 | |
| 860 | .. method:: drain() |
| 861 | |
| 862 | This method has an unusual return value. |
| 863 | |
| 864 | The intended use is to write:: |
| 865 | |
| 866 | w.write(data) |
| 867 | yield from w.drain() |
| 868 | |
| 869 | When there's nothing to wait for, :meth:`drain()` returns ``()``, and the |
| 870 | yield-from continues immediately. When the transport buffer is full (the |
| 871 | protocol is paused), :meth:`drain` creates and returns a |
| 872 | :class:`~concurrent.futures.Future` and the yield-from will block until |
| 873 | that Future is completed, which will happen when the buffer is |
| 874 | (partially) drained and the protocol is resumed. |
| 875 | |
| 876 | .. method:: get_extra_info(name, default=None) |
| 877 | |
| 878 | Return optional transport information: see |
| 879 | :meth:`BaseTransport.get_extra_info`. |
| 880 | |
| 881 | .. method:: write(data) |
| 882 | |
| 883 | Write some *data* bytes to the transport: see |
| 884 | :meth:`WriteTransport.write`. |
| 885 | |
| 886 | .. method:: writelines(data) |
| 887 | |
| 888 | Write a list (or any iterable) of data bytes to the transport: |
| 889 | see :meth:`WriteTransport.writelines`. |
| 890 | |
| 891 | .. method:: can_write_eof() |
| 892 | |
| 893 | Return :const:`True` if the transport supports :meth:`write_eof`, |
| 894 | :const:`False` if not. See :meth:`WriteTransport.can_write_eof`. |
| 895 | |
| 896 | .. method:: write_eof() |
| 897 | |
| 898 | Close the write end of the transport after flushing buffered data: |
| 899 | see :meth:`WriteTransport.write_eof`. |
| 900 | |
| 901 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 902 | Stream writer |
| 903 | ------------- |
| 904 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 905 | .. class:: StreamReader(limit=_DEFAULT_LIMIT, loop=None) |
| 906 | |
| 907 | .. method:: exception() |
| 908 | |
| 909 | Get the exception. |
| 910 | |
| 911 | .. method:: feed_eof() |
| 912 | |
| 913 | XXX |
| 914 | |
| 915 | .. method:: feed_data(data) |
| 916 | |
| 917 | XXX |
| 918 | |
| 919 | .. method:: set_exception(exc) |
| 920 | |
| 921 | Set the exception. |
| 922 | |
| 923 | .. method:: set_transport(transport) |
| 924 | |
| 925 | Set the transport. |
| 926 | |
| 927 | .. method:: read(n=-1) |
| 928 | |
| 929 | XXX |
| 930 | |
| 931 | This method returns a :ref:`coroutine <coroutine>`. |
| 932 | |
| 933 | .. method:: readline() |
| 934 | |
| 935 | XXX |
| 936 | |
| 937 | This method returns a :ref:`coroutine <coroutine>`. |
| 938 | |
| 939 | .. method:: readexactly(n) |
| 940 | |
| 941 | XXX |
| 942 | |
| 943 | This method returns a :ref:`coroutine <coroutine>`. |
| 944 | |
Victor Stinner | be49063 | 2013-12-02 17:28:32 +0100 | [diff] [blame] | 945 | |
| 946 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 947 | .. _protocol: |
| 948 | |
| 949 | Protocols |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 950 | ========= |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 951 | |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 952 | :mod:`asyncio` provides base classes that you can subclass to implement |
| 953 | your network protocols. Those classes are used in conjunction with |
| 954 | :ref:`transports <transport>` (see below): the protocol parses incoming |
| 955 | data and asks for the writing of outgoing data, while the transport is |
| 956 | responsible for the actual I/O and buffering. |
| 957 | |
| 958 | When subclassing a protocol class, it is recommended you override certain |
| 959 | methods. Those methods are callbacks: they will be called by the transport |
| 960 | on certain events (for example when some data is received); you shouldn't |
| 961 | call them yourself, unless you are implementing a transport. |
| 962 | |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 963 | .. note:: |
| 964 | All callbacks have default implementations, which are empty. Therefore, |
| 965 | you only need to implement the callbacks for the events in which you |
| 966 | are interested. |
| 967 | |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 968 | |
| 969 | Protocol classes |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 970 | ---------------- |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 971 | |
| 972 | .. class:: Protocol |
| 973 | |
| 974 | The base class for implementing streaming protocols (for use with |
| 975 | e.g. TCP and SSL transports). |
| 976 | |
| 977 | .. class:: DatagramProtocol |
| 978 | |
| 979 | The base class for implementing datagram protocols (for use with |
| 980 | e.g. UDP transports). |
| 981 | |
| 982 | .. class:: SubprocessProtocol |
| 983 | |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 984 | The base class for implementing protocols communicating with child |
| 985 | processes (through a set of unidirectional pipes). |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 986 | |
| 987 | |
| 988 | Connection callbacks |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 989 | -------------------- |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 990 | |
| 991 | These callbacks may be called on :class:`Protocol` and |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 992 | :class:`SubprocessProtocol` instances: |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 993 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 994 | .. method:: BaseProtocol.connection_made(transport) |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 995 | |
| 996 | Called when a connection is made. |
| 997 | |
| 998 | The *transport* argument is the transport representing the |
| 999 | connection. You are responsible for storing it somewhere |
| 1000 | (e.g. as an attribute) if you need to. |
| 1001 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1002 | .. method:: BaseProtocol.connection_lost(exc) |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1003 | |
| 1004 | Called when the connection is lost or closed. |
| 1005 | |
| 1006 | The argument is either an exception object or :const:`None`. |
| 1007 | The latter means a regular EOF is received, or the connection was |
| 1008 | aborted or closed by this side of the connection. |
| 1009 | |
| 1010 | :meth:`connection_made` and :meth:`connection_lost` are called exactly once |
| 1011 | per successful connection. All other callbacks will be called between those |
| 1012 | two methods, which allows for easier resource management in your protocol |
| 1013 | implementation. |
| 1014 | |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1015 | The following callbacks may be called only on :class:`SubprocessProtocol` |
| 1016 | instances: |
| 1017 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1018 | .. method:: SubprocessProtocol.pipe_data_received(fd, data) |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1019 | |
| 1020 | Called when the child process writes data into its stdout or stderr pipe. |
| 1021 | *fd* is the integer file descriptor of the pipe. *data* is a non-empty |
| 1022 | bytes object containing the data. |
| 1023 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1024 | .. method:: SubprocessProtocol.pipe_connection_lost(fd, exc) |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1025 | |
| 1026 | Called when one of the pipes communicating with the child process |
| 1027 | is closed. *fd* is the integer file descriptor that was closed. |
| 1028 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1029 | .. method:: SubprocessProtocol.process_exited() |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1030 | |
| 1031 | Called when the child process has exited. |
| 1032 | |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1033 | |
| 1034 | Data reception callbacks |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1035 | ------------------------ |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1036 | |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1037 | Streaming protocols |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1038 | ^^^^^^^^^^^^^^^^^^^ |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1039 | |
| 1040 | The following callbacks are called on :class:`Protocol` instances: |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1041 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1042 | .. method:: Protocol.data_received(data) |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1043 | |
| 1044 | Called when some data is received. *data* is a non-empty bytes object |
| 1045 | containing the incoming data. |
| 1046 | |
| 1047 | .. note:: |
| 1048 | Whether the data is buffered, chunked or reassembled depends on |
| 1049 | the transport. In general, you shouldn't rely on specific semantics |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1050 | and instead make your parsing generic and flexible enough. However, |
| 1051 | data is always received in the correct order. |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1052 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1053 | .. method:: Protocol.eof_received() |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1054 | |
| 1055 | Calls when the other end signals it won't send any more data |
| 1056 | (for example by calling :meth:`write_eof`, if the other end also uses |
| 1057 | asyncio). |
| 1058 | |
| 1059 | This method may return a false value (including None), in which case |
| 1060 | the transport will close itself. Conversely, if this method returns a |
| 1061 | true value, closing the transport is up to the protocol. Since the |
| 1062 | default implementation returns None, it implicitly closes the connection. |
| 1063 | |
| 1064 | .. note:: |
| 1065 | Some transports such as SSL don't support half-closed connections, |
| 1066 | in which case returning true from this method will not prevent closing |
| 1067 | the connection. |
| 1068 | |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1069 | :meth:`data_received` can be called an arbitrary number of times during |
| 1070 | a connection. However, :meth:`eof_received` is called at most once |
| 1071 | and, if called, :meth:`data_received` won't be called after it. |
| 1072 | |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1073 | Datagram protocols |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1074 | ^^^^^^^^^^^^^^^^^^ |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1075 | |
| 1076 | The following callbacks are called on :class:`DatagramProtocol` instances. |
| 1077 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1078 | .. method:: DatagramProtocol.datagram_received(data, addr) |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1079 | |
| 1080 | Called when a datagram is received. *data* is a bytes object containing |
| 1081 | the incoming data. *addr* is the address of the peer sending the data; |
| 1082 | the exact format depends on the transport. |
| 1083 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1084 | .. method:: DatagramProtocol.error_received(exc) |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1085 | |
| 1086 | Called when a previous send or receive operation raises an |
| 1087 | :class:`OSError`. *exc* is the :class:`OSError` instance. |
| 1088 | |
| 1089 | This method is called in rare conditions, when the transport (e.g. UDP) |
| 1090 | detects that a datagram couldn't be delivered to its recipient. |
| 1091 | In many conditions though, undeliverable datagrams will be silently |
| 1092 | dropped. |
| 1093 | |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1094 | |
| 1095 | Flow control callbacks |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1096 | ---------------------- |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1097 | |
| 1098 | These callbacks may be called on :class:`Protocol` and |
Antoine Pitrou | 74193af | 2013-11-23 01:21:11 +0100 | [diff] [blame] | 1099 | :class:`SubprocessProtocol` instances: |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1100 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1101 | .. method:: BaseProtocol.pause_writing() |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1102 | |
| 1103 | Called when the transport's buffer goes over the high-water mark. |
| 1104 | |
Victor Stinner | 8dc434e | 2013-12-02 12:20:57 +0100 | [diff] [blame] | 1105 | .. method:: BaseProtocol.resume_writing() |
Antoine Pitrou | a035e1b | 2013-11-23 01:08:43 +0100 | [diff] [blame] | 1106 | |
| 1107 | Called when the transport's buffer drains below the low-water mark. |
| 1108 | |
| 1109 | |
| 1110 | :meth:`pause_writing` and :meth:`resume_writing` calls are paired -- |
| 1111 | :meth:`pause_writing` is called once when the buffer goes strictly over |
| 1112 | the high-water mark (even if subsequent writes increases the buffer size |
| 1113 | even more), and eventually :meth:`resume_writing` is called once when the |
| 1114 | buffer size reaches the low-water mark. |
| 1115 | |
| 1116 | .. note:: |
| 1117 | If the buffer size equals the high-water mark, |
| 1118 | :meth:`pause_writing` is not called -- it must go strictly over. |
| 1119 | Conversely, :meth:`resume_writing` is called when the buffer size is |
| 1120 | equal or lower than the low-water mark. These end conditions |
| 1121 | are important to ensure that things go as expected when either |
| 1122 | mark is zero. |
| 1123 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 1124 | |
Victor Stinner | e91f180 | 2013-12-02 17:40:48 +0100 | [diff] [blame] | 1125 | Server |
| 1126 | ------ |
| 1127 | |
| 1128 | .. class:: AbstractServer |
| 1129 | |
| 1130 | Abstract server returned by create_service(). |
| 1131 | |
| 1132 | .. method:: close() |
| 1133 | |
| 1134 | Stop serving. This leaves existing connections open. |
| 1135 | |
| 1136 | .. method:: wait_closed() |
| 1137 | |
| 1138 | Coroutine to wait until service is closed. |
| 1139 | |
| 1140 | |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 1141 | Network functions |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1142 | ================= |
Victor Stinner | b3be72c | 2013-12-03 00:49:26 +0100 | [diff] [blame] | 1143 | |
| 1144 | .. function:: open_connection(host=None, port=None, *, loop=None, limit=_DEFAULT_LIMIT, **kwds) |
| 1145 | |
| 1146 | A wrapper for create_connection() returning a (reader, writer) pair. |
| 1147 | |
| 1148 | The reader returned is a StreamReader instance; the writer is a |
| 1149 | :class:`Transport`. |
| 1150 | |
| 1151 | The arguments are all the usual arguments to |
| 1152 | :meth:`BaseEventLoop.create_connection` except *protocol_factory*; most |
| 1153 | common are positional host and port, with various optional keyword arguments |
| 1154 | following. |
| 1155 | |
| 1156 | Additional optional keyword arguments are *loop* (to set the event loop |
| 1157 | instance to use) and *limit* (to set the buffer limit passed to the |
| 1158 | StreamReader). |
| 1159 | |
| 1160 | (If you want to customize the :class:`StreamReader` and/or |
| 1161 | :class:`StreamReaderProtocol` classes, just copy the code -- there's really |
| 1162 | nothing special here except some convenience.) |
| 1163 | |
| 1164 | This function returns a :ref:`coroutine <coroutine>`. |
| 1165 | |
| 1166 | .. function:: start_server(client_connected_cb, host=None, port=None, *, loop=None, limit=_DEFAULT_LIMIT, **kwds) |
| 1167 | |
| 1168 | Start a socket server, call back for each client connected. |
| 1169 | |
| 1170 | The first parameter, *client_connected_cb*, takes two parameters: |
| 1171 | *client_reader*, *client_writer*. *client_reader* is a |
| 1172 | :class:`StreamReader` object, while *client_writer* is a |
| 1173 | :class:`StreamWriter` object. This parameter can either be a plain callback |
| 1174 | function or a :ref:`coroutine <coroutine>`; if it is a coroutine, it will be |
| 1175 | automatically converted into a :class:`Task`. |
| 1176 | |
| 1177 | The rest of the arguments are all the usual arguments to |
| 1178 | :meth:`~BaseEventLoop.create_server()` except *protocol_factory*; most |
| 1179 | common are positional host and port, with various optional keyword arguments |
| 1180 | following. The return value is the same as |
| 1181 | :meth:`~BaseEventLoop.create_server()`. |
| 1182 | |
| 1183 | Additional optional keyword arguments are *loop* (to set the event loop |
| 1184 | instance to use) and *limit* (to set the buffer limit passed to the |
| 1185 | :class:`StreamReader`). |
| 1186 | |
| 1187 | The return value is the same as :meth:`~BaseEventLoop.create_server()`, i.e. |
| 1188 | a :class:`AbstractServer` object which can be used to stop the service. |
| 1189 | |
| 1190 | This function returns a :ref:`coroutine <coroutine>`. |
| 1191 | |
| 1192 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 1193 | .. _sync: |
| 1194 | |
| 1195 | Synchronization primitives |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1196 | ========================== |
| 1197 | |
| 1198 | Locks |
| 1199 | ----- |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 1200 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1201 | .. class:: Lock(\*, loop=None) |
| 1202 | |
| 1203 | Primitive lock objects. |
| 1204 | |
| 1205 | A primitive lock is a synchronization primitive that is not owned by a |
| 1206 | particular coroutine when locked. A primitive lock is in one of two states, |
| 1207 | 'locked' or 'unlocked'. |
| 1208 | |
| 1209 | It is created in the unlocked state. It has two basic methods, :meth:`acquire` |
| 1210 | and :meth:`release`. When the state is unlocked, acquire() changes the state to |
| 1211 | locked and returns immediately. When the state is locked, acquire() blocks |
| 1212 | until a call to release() in another coroutine changes it to unlocked, then |
| 1213 | the acquire() call resets it to locked and returns. The release() method |
| 1214 | should only be called in the locked state; it changes the state to unlocked |
| 1215 | and returns immediately. If an attempt is made to release an unlocked lock, |
| 1216 | a :exc:`RuntimeError` will be raised. |
| 1217 | |
| 1218 | When more than one coroutine is blocked in acquire() waiting for the state |
| 1219 | to turn to unlocked, only one coroutine proceeds when a release() call |
| 1220 | resets the state to unlocked; first coroutine which is blocked in acquire() |
| 1221 | is being processed. |
| 1222 | |
| 1223 | :meth:`acquire` is a coroutine and should be called with ``yield from``. |
| 1224 | |
| 1225 | Locks also support the context manager protocol. ``(yield from lock)`` |
| 1226 | should be used as context manager expression. |
| 1227 | |
| 1228 | Usage:: |
| 1229 | |
| 1230 | lock = Lock() |
| 1231 | ... |
| 1232 | yield from lock |
| 1233 | try: |
| 1234 | ... |
| 1235 | finally: |
| 1236 | lock.release() |
| 1237 | |
| 1238 | Context manager usage:: |
| 1239 | |
| 1240 | lock = Lock() |
| 1241 | ... |
| 1242 | with (yield from lock): |
| 1243 | ... |
| 1244 | |
| 1245 | Lock objects can be tested for locking state:: |
| 1246 | |
| 1247 | if not lock.locked(): |
| 1248 | yield from lock |
| 1249 | else: |
| 1250 | # lock is acquired |
| 1251 | ... |
| 1252 | |
| 1253 | .. method:: locked() |
| 1254 | |
| 1255 | Return ``True`` if lock is acquired. |
| 1256 | |
| 1257 | .. method:: acquire() |
| 1258 | |
| 1259 | Acquire a lock. |
| 1260 | |
| 1261 | This method blocks until the lock is unlocked, then sets it to locked and |
| 1262 | returns ``True``. |
| 1263 | |
| 1264 | This method returns a :ref:`coroutine <coroutine>`. |
| 1265 | |
| 1266 | .. method:: release() |
| 1267 | |
| 1268 | Release a lock. |
| 1269 | |
| 1270 | When the lock is locked, reset it to unlocked, and return. If any other |
| 1271 | coroutines are blocked waiting for the lock to become unlocked, allow |
| 1272 | exactly one of them to proceed. |
| 1273 | |
| 1274 | When invoked on an unlocked lock, a :exc:`RuntimeError` is raised. |
| 1275 | |
| 1276 | There is no return value. |
| 1277 | |
| 1278 | |
| 1279 | .. class:: Event(\*, loop=None) |
| 1280 | |
| 1281 | An Event implementation, asynchronous equivalent to :class:`threading.Event`. |
| 1282 | |
| 1283 | Class implementing event objects. An event manages a flag that can be set to |
| 1284 | true with the :meth:`set` method and reset to false with the :meth:`clear` |
| 1285 | method. The :meth:`wait` method blocks until the flag is true. The flag is |
| 1286 | initially false. |
| 1287 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 1288 | .. method:: clear() |
| 1289 | |
| 1290 | Reset the internal flag to false. Subsequently, coroutines calling |
| 1291 | :meth:`wait` will block until :meth:`set` is called to set the internal |
| 1292 | flag to true again. |
| 1293 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1294 | .. method:: is_set() |
| 1295 | |
| 1296 | Return ``True`` if and only if the internal flag is true. |
| 1297 | |
| 1298 | .. method:: set() |
| 1299 | |
| 1300 | Set the internal flag to true. All coroutines waiting for it to become |
| 1301 | true are awakened. Coroutine that call :meth:`wait` once the flag is true |
| 1302 | will not block at all. |
| 1303 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1304 | .. method:: wait() |
| 1305 | |
| 1306 | Block until the internal flag is true. |
| 1307 | |
| 1308 | If the internal flag is true on entry, return ``True`` immediately. |
| 1309 | Otherwise, block until another coroutine calls :meth:`set` to set the |
| 1310 | flag to true, then return ``True``. |
| 1311 | |
| 1312 | This method returns a :ref:`coroutine <coroutine>`. |
| 1313 | |
| 1314 | |
| 1315 | .. class:: Condition(\*, loop=None) |
| 1316 | |
| 1317 | A Condition implementation, asynchronous equivalent to |
| 1318 | :class:`threading.Condition`. |
| 1319 | |
| 1320 | This class implements condition variable objects. A condition variable |
| 1321 | allows one or more coroutines to wait until they are notified by another |
| 1322 | coroutine. |
| 1323 | |
| 1324 | A new :class:`Lock` object is created and used as the underlying lock. |
| 1325 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 1326 | .. method:: notify(n=1) |
| 1327 | |
| 1328 | By default, wake up one coroutine waiting on this condition, if any. |
| 1329 | If the calling coroutine has not acquired the lock when this method is |
| 1330 | called, a :exc:`RuntimeError` is raised. |
| 1331 | |
| 1332 | This method wakes up at most *n* of the coroutines waiting for the |
| 1333 | condition variable; it is a no-op if no coroutines are waiting. |
| 1334 | |
| 1335 | .. note:: |
| 1336 | |
| 1337 | An awakened coroutine does not actually return from its :meth:`wait` |
| 1338 | call until it can reacquire the lock. Since :meth:`notify` does not |
| 1339 | release the lock, its caller should. |
| 1340 | |
| 1341 | .. method:: notify_all() |
| 1342 | |
| 1343 | Wake up all threads waiting on this condition. This method acts like |
| 1344 | :meth:`notify`, but wakes up all waiting threads instead of one. If the |
| 1345 | calling thread has not acquired the lock when this method is called, a |
| 1346 | :exc:`RuntimeError` is raised. |
| 1347 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1348 | .. method:: wait() |
| 1349 | |
| 1350 | Wait until notified. |
| 1351 | |
| 1352 | If the calling coroutine has not acquired the lock when this method is |
| 1353 | called, a :exc:`RuntimeError` is raised. |
| 1354 | |
| 1355 | This method releases the underlying lock, and then blocks until it is |
| 1356 | awakened by a :meth:`notify` or :meth:`notify_all` call for the same |
| 1357 | condition variable in another coroutine. Once awakened, it re-acquires |
| 1358 | the lock and returns ``True``. |
| 1359 | |
| 1360 | This method returns a :ref:`coroutine <coroutine>`. |
| 1361 | |
| 1362 | .. method:: wait_for(predicate) |
| 1363 | |
| 1364 | Wait until a predicate becomes true. |
| 1365 | |
| 1366 | The predicate should be a callable which result will be interpreted as a |
| 1367 | boolean value. The final predicate value is the return value. |
| 1368 | |
| 1369 | This method returns a :ref:`coroutine <coroutine>`. |
| 1370 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1371 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1372 | Semaphores |
| 1373 | ---------- |
| 1374 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1375 | .. class:: Semaphore(value=1, \*, loop=None) |
| 1376 | |
| 1377 | A Semaphore implementation. |
| 1378 | |
| 1379 | A semaphore manages an internal counter which is decremented by each |
| 1380 | :meth:`acquire` call and incremented by each :meth:`release` call. The |
| 1381 | counter can never go below zero; when :meth:`acquire` finds that it is zero, |
| 1382 | it blocks, waiting until some other thread calls :meth:`release`. |
| 1383 | |
| 1384 | Semaphores also support the context manager protocol. |
| 1385 | |
| 1386 | The optional argument gives the initial value for the internal counter; it |
| 1387 | defaults to ``1``. If the value given is less than ``0``, :exc:`ValueError` |
| 1388 | is raised. |
| 1389 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1390 | .. method:: acquire() |
| 1391 | |
| 1392 | Acquire a semaphore. |
| 1393 | |
| 1394 | If the internal counter is larger than zero on entry, decrement it by one |
| 1395 | and return ``True`` immediately. If it is zero on entry, block, waiting |
| 1396 | until some other coroutine has called :meth:`release` to make it larger |
| 1397 | than ``0``, and then return ``True``. |
| 1398 | |
| 1399 | This method returns a :ref:`coroutine <coroutine>`. |
| 1400 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 1401 | .. method:: locked() |
| 1402 | |
| 1403 | Returns ``True`` if semaphore can not be acquired immediately. |
| 1404 | |
Victor Stinner | c37dd61 | 2013-12-02 14:31:16 +0100 | [diff] [blame] | 1405 | .. method:: release() |
| 1406 | |
| 1407 | Release a semaphore, incrementing the internal counter by one. When it |
| 1408 | was zero on entry and another coroutine is waiting for it to become |
| 1409 | larger than zero again, wake up that coroutine. |
| 1410 | |
| 1411 | |
| 1412 | .. class:: BoundedSemaphore(value=1, \*, loop=None) |
| 1413 | |
| 1414 | A bounded semaphore implementation. Inherit from :class:`Semaphore`. |
| 1415 | |
| 1416 | This raises :exc:`ValueError` in :meth:`~Semaphore.release` if it would |
| 1417 | increase the value above the initial value. |
| 1418 | |
| 1419 | |
Victor Stinner | fa2ce78 | 2013-12-03 00:56:27 +0100 | [diff] [blame^] | 1420 | Queues |
| 1421 | ------ |
| 1422 | |
Victor Stinner | 0da1d29 | 2013-12-02 14:06:03 +0100 | [diff] [blame] | 1423 | .. class:: Queue(maxsize=0, \*, loop=None) |
| 1424 | |
| 1425 | A queue, useful for coordinating producer and consumer coroutines. |
| 1426 | |
| 1427 | If *maxsize* is less than or equal to zero, the queue size is infinite. If |
| 1428 | it is an integer greater than ``0``, then ``yield from put()`` will block |
| 1429 | when the queue reaches *maxsize*, until an item is removed by :meth:`get`. |
| 1430 | |
| 1431 | Unlike the standard library :mod:`queue`, you can reliably know this Queue's |
| 1432 | size with :meth:`qsize`, since your single-threaded Tulip application won't |
| 1433 | be interrupted between calling :meth:`qsize` and doing an operation on the |
| 1434 | Queue. |
| 1435 | |
| 1436 | .. method:: empty() |
| 1437 | |
| 1438 | Return ``True`` if the queue is empty, ``False`` otherwise. |
| 1439 | |
| 1440 | .. method:: full() |
| 1441 | |
| 1442 | Return ``True`` if there are maxsize items in the queue. |
| 1443 | |
| 1444 | .. note:: |
| 1445 | |
| 1446 | If the Queue was initialized with ``maxsize=0`` (the default), then |
| 1447 | :meth:`full()` is never ``True``. |
| 1448 | |
| 1449 | .. method:: get() |
| 1450 | |
| 1451 | Remove and return an item from the queue. |
| 1452 | |
| 1453 | If you yield from :meth:`get()`, wait until a item is available. |
| 1454 | |
| 1455 | This method returns a :ref:`coroutine <coroutine>`. |
| 1456 | |
| 1457 | .. method:: get_nowait() |
| 1458 | |
| 1459 | Remove and return an item from the queue. |
| 1460 | |
| 1461 | Return an item if one is immediately available, else raise |
| 1462 | :exc:`~queue.Empty`. |
| 1463 | |
| 1464 | .. method:: put(item) |
| 1465 | |
| 1466 | Put an item into the queue. |
| 1467 | |
| 1468 | If you yield from ``put()``, wait until a free slot is available before |
| 1469 | adding item. |
| 1470 | |
| 1471 | This method returns a :ref:`coroutine <coroutine>`. |
| 1472 | |
| 1473 | .. method:: put_nowait(item) |
| 1474 | |
| 1475 | Put an item into the queue without blocking. |
| 1476 | |
| 1477 | If no free slot is immediately available, raise :exc:`~queue.Full`. |
| 1478 | |
| 1479 | .. method:: qsize() |
| 1480 | |
| 1481 | Number of items in the queue. |
| 1482 | |
| 1483 | .. attribute:: maxsize |
| 1484 | |
| 1485 | Number of items allowed in the queue. |
| 1486 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 1487 | |
Victor Stinner | 0da1d29 | 2013-12-02 14:06:03 +0100 | [diff] [blame] | 1488 | .. class:: PriorityQueue |
| 1489 | |
| 1490 | A subclass of :class:`Queue`; retrieves entries in priority order (lowest |
| 1491 | first). |
| 1492 | |
| 1493 | Entries are typically tuples of the form: (priority number, data). |
| 1494 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 1495 | |
Victor Stinner | 0da1d29 | 2013-12-02 14:06:03 +0100 | [diff] [blame] | 1496 | .. class:: LifoQueue |
| 1497 | |
| 1498 | A subclass of :class:`Queue` that retrieves most recently added entries |
| 1499 | first. |
| 1500 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 1501 | |
Victor Stinner | 0da1d29 | 2013-12-02 14:06:03 +0100 | [diff] [blame] | 1502 | .. class:: JoinableQueue |
| 1503 | |
| 1504 | A subclass of :class:`Queue` with :meth:`task_done` and :meth:`join` |
| 1505 | methods. |
| 1506 | |
Victor Stinner | 0c924b8 | 2013-12-02 17:52:31 +0100 | [diff] [blame] | 1507 | .. method:: join() |
| 1508 | |
| 1509 | Block until all items in the queue have been gotten and processed. |
| 1510 | |
| 1511 | The count of unfinished tasks goes up whenever an item is added to the |
| 1512 | queue. The count goes down whenever a consumer thread calls |
| 1513 | :meth:`task_done` to indicate that the item was retrieved and all work on |
| 1514 | it is complete. When the count of unfinished tasks drops to zero, |
| 1515 | :meth:`join` unblocks. |
| 1516 | |
| 1517 | This method returns a :ref:`coroutine <coroutine>`. |
| 1518 | |
Victor Stinner | 0da1d29 | 2013-12-02 14:06:03 +0100 | [diff] [blame] | 1519 | .. method:: task_done() |
| 1520 | |
| 1521 | Indicate that a formerly enqueued task is complete. |
| 1522 | |
| 1523 | Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a |
| 1524 | subsequent call to :meth:`task_done` tells the queue that the processing |
| 1525 | on the task is complete. |
| 1526 | |
| 1527 | If a :meth:`join` is currently blocking, it will resume when all items |
| 1528 | have been processed (meaning that a :meth:`task_done` call was received |
| 1529 | for every item that had been :meth:`~Queue.put` into the queue). |
| 1530 | |
| 1531 | Raises :exc:`ValueError` if called more times than there were items |
| 1532 | placed in the queue. |
| 1533 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 1534 | |
| 1535 | Examples |
| 1536 | -------- |
| 1537 | |
Victor Stinner | 4e70bb8 | 2013-12-02 12:21:30 +0100 | [diff] [blame] | 1538 | Hello World (callback) |
| 1539 | ^^^^^^^^^^^^^^^^^^^^^^ |
| 1540 | |
| 1541 | Print ``Hello World`` every two seconds, using a callback:: |
| 1542 | |
| 1543 | import asyncio |
| 1544 | |
| 1545 | def print_and_repeat(loop): |
| 1546 | print('Hello World') |
| 1547 | loop.call_later(2, print_and_repeat, loop) |
| 1548 | |
| 1549 | loop = asyncio.get_event_loop() |
| 1550 | print_and_repeat(loop) |
| 1551 | loop.run_forever() |
| 1552 | |
| 1553 | |
| 1554 | Hello World (callback) |
| 1555 | ^^^^^^^^^^^^^^^^^^^^^^ |
| 1556 | |
| 1557 | Print ``Hello World`` every two seconds, using a coroutine:: |
| 1558 | |
| 1559 | import asyncio |
| 1560 | |
| 1561 | @asyncio.coroutine |
| 1562 | def greet_every_two_seconds(): |
| 1563 | while True: |
| 1564 | print('Hello World') |
| 1565 | yield from asyncio.sleep(2) |
| 1566 | |
| 1567 | loop = asyncio.get_event_loop() |
| 1568 | loop.run_until_complete(greet_every_two_seconds()) |
| 1569 | |
| 1570 | |
| 1571 | Echo server |
| 1572 | ^^^^^^^^^^^ |
| 1573 | |
Antoine Pitrou | bba8682 | 2013-11-23 00:34:26 +0100 | [diff] [blame] | 1574 | A :class:`Protocol` implementing an echo server:: |
| 1575 | |
| 1576 | class EchoServer(asyncio.Protocol): |
| 1577 | |
| 1578 | TIMEOUT = 5.0 |
| 1579 | |
| 1580 | def timeout(self): |
| 1581 | print('connection timeout, closing.') |
| 1582 | self.transport.close() |
| 1583 | |
| 1584 | def connection_made(self, transport): |
| 1585 | print('connection made') |
| 1586 | self.transport = transport |
| 1587 | |
| 1588 | # start 5 seconds timeout timer |
| 1589 | self.h_timeout = asyncio.get_event_loop().call_later( |
| 1590 | self.TIMEOUT, self.timeout) |
| 1591 | |
| 1592 | def data_received(self, data): |
| 1593 | print('data received: ', data.decode()) |
| 1594 | self.transport.write(b'Re: ' + data) |
| 1595 | |
| 1596 | # restart timeout timer |
| 1597 | self.h_timeout.cancel() |
| 1598 | self.h_timeout = asyncio.get_event_loop().call_later( |
| 1599 | self.TIMEOUT, self.timeout) |
| 1600 | |
| 1601 | def eof_received(self): |
| 1602 | pass |
| 1603 | |
| 1604 | def connection_lost(self, exc): |
| 1605 | print('connection lost:', exc) |
| 1606 | self.h_timeout.cancel() |
| 1607 | |