Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 1 | .. currentmodule:: asyncio |
| 2 | |
Victor Stinner | 9592edb | 2014-02-02 15:03:02 +0100 | [diff] [blame] | 3 | .. _asyncio-streams: |
Victor Stinner | 4b4f9eb | 2014-01-24 17:33:20 +0100 | [diff] [blame] | 4 | |
Victor Stinner | 1374bd4 | 2014-01-24 15:34:19 +0100 | [diff] [blame] | 5 | ++++++++++++++++++++++++ |
| 6 | Streams (high-level API) |
| 7 | ++++++++++++++++++++++++ |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 8 | |
| 9 | Stream functions |
| 10 | ================ |
| 11 | |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 12 | .. function:: open_connection(host=None, port=None, \*, loop=None, limit=None, **kwds) |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 13 | |
| 14 | A wrapper for :meth:`~BaseEventLoop.create_connection()` returning a (reader, |
| 15 | writer) pair. |
| 16 | |
| 17 | The reader returned is a :class:`StreamReader` instance; the writer is |
| 18 | a :class:`StreamWriter` instance. |
| 19 | |
| 20 | The arguments are all the usual arguments to |
| 21 | :meth:`BaseEventLoop.create_connection` except *protocol_factory*; most |
| 22 | common are positional host and port, with various optional keyword arguments |
| 23 | following. |
| 24 | |
| 25 | Additional optional keyword arguments are *loop* (to set the event loop |
| 26 | instance to use) and *limit* (to set the buffer limit passed to the |
| 27 | :class:`StreamReader`). |
| 28 | |
| 29 | (If you want to customize the :class:`StreamReader` and/or |
| 30 | :class:`StreamReaderProtocol` classes, just copy the code -- there's really |
| 31 | nothing special here except some convenience.) |
| 32 | |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 33 | This function is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 34 | |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 35 | .. function:: start_server(client_connected_cb, host=None, port=None, \*, loop=None, limit=None, **kwds) |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 36 | |
Victor Stinner | 8ebeb03 | 2014-07-11 23:47:40 +0200 | [diff] [blame] | 37 | Start a socket server, with a callback for each client connected. The return |
| 38 | value is the same as :meth:`~BaseEventLoop.create_server()`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 39 | |
Victor Stinner | 8ebeb03 | 2014-07-11 23:47:40 +0200 | [diff] [blame] | 40 | The *client_connected_cb* parameter is called with two parameters: |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 41 | *client_reader*, *client_writer*. *client_reader* is a |
| 42 | :class:`StreamReader` object, while *client_writer* is a |
Victor Stinner | 8ebeb03 | 2014-07-11 23:47:40 +0200 | [diff] [blame] | 43 | :class:`StreamWriter` object. The *client_connected_cb* parameter can |
| 44 | either be a plain callback function or a :ref:`coroutine function |
| 45 | <coroutine>`; if it is a coroutine function, it will be automatically |
Victor Stinner | 337e03f | 2014-08-11 01:11:13 +0200 | [diff] [blame] | 46 | converted into a :class:`Task`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 47 | |
| 48 | The rest of the arguments are all the usual arguments to |
| 49 | :meth:`~BaseEventLoop.create_server()` except *protocol_factory*; most |
Victor Stinner | 8ebeb03 | 2014-07-11 23:47:40 +0200 | [diff] [blame] | 50 | common are positional *host* and *port*, with various optional keyword |
| 51 | arguments following. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 52 | |
| 53 | Additional optional keyword arguments are *loop* (to set the event loop |
| 54 | instance to use) and *limit* (to set the buffer limit passed to the |
| 55 | :class:`StreamReader`). |
| 56 | |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 57 | This function is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 58 | |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 59 | .. function:: open_unix_connection(path=None, \*, loop=None, limit=None, **kwds) |
| 60 | |
| 61 | A wrapper for :meth:`~BaseEventLoop.create_unix_connection()` returning |
| 62 | a (reader, writer) pair. |
| 63 | |
| 64 | See :func:`open_connection` for information about return value and other |
| 65 | details. |
| 66 | |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 67 | This function is a :ref:`coroutine <coroutine>`. |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 68 | |
| 69 | Availability: UNIX. |
| 70 | |
| 71 | .. function:: start_unix_server(client_connected_cb, path=None, \*, loop=None, limit=None, **kwds) |
| 72 | |
| 73 | Start a UNIX Domain Socket server, with a callback for each client connected. |
| 74 | |
| 75 | See :func:`start_server` for information about return value and other |
| 76 | details. |
| 77 | |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 78 | This function is a :ref:`coroutine <coroutine>`. |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 79 | |
| 80 | Availability: UNIX. |
| 81 | |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 82 | |
| 83 | StreamReader |
| 84 | ============ |
| 85 | |
Victor Stinner | 0844438 | 2014-02-02 22:43:39 +0100 | [diff] [blame] | 86 | .. class:: StreamReader(limit=None, loop=None) |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 87 | |
| 88 | .. method:: exception() |
| 89 | |
| 90 | Get the exception. |
| 91 | |
| 92 | .. method:: feed_eof() |
| 93 | |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 94 | Acknowledge the EOF. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 95 | |
| 96 | .. method:: feed_data(data) |
| 97 | |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 98 | Feed *data* bytes in the internal buffer. Any operations waiting |
| 99 | for the data will be resumed. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 100 | |
| 101 | .. method:: set_exception(exc) |
| 102 | |
| 103 | Set the exception. |
| 104 | |
| 105 | .. method:: set_transport(transport) |
| 106 | |
| 107 | Set the transport. |
| 108 | |
| 109 | .. method:: read(n=-1) |
| 110 | |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 111 | Read up to *n* bytes. If *n* is not provided, or set to ``-1``, |
| 112 | read until EOF and return all read bytes. |
| 113 | |
| 114 | If the EOF was received and the internal buffer is empty, |
| 115 | return an empty ``bytes`` object. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 116 | |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 117 | This method is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 118 | |
| 119 | .. method:: readline() |
| 120 | |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 121 | Read one line, where "line" is a sequence of bytes ending with ``\n``. |
| 122 | |
| 123 | If EOF is received, and ``\n`` was not found, the method will |
| 124 | return the partial read bytes. |
| 125 | |
| 126 | If the EOF was received and the internal buffer is empty, |
| 127 | return an empty ``bytes`` object. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 128 | |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 129 | This method is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 130 | |
| 131 | .. method:: readexactly(n) |
| 132 | |
Victor Stinner | b7f19ff | 2014-01-27 11:58:49 +0100 | [diff] [blame] | 133 | Read exactly *n* bytes. Raise an :exc:`IncompleteReadError` if the end of |
| 134 | the stream is reached before *n* can be read, the |
| 135 | :attr:`IncompleteReadError.partial` attribute of the exception contains |
| 136 | the partial read bytes. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 137 | |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 138 | This method is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 139 | |
Yury Selivanov | d3f8e30 | 2014-02-20 14:10:02 -0500 | [diff] [blame] | 140 | .. method:: at_eof() |
| 141 | |
| 142 | Return ``True`` if the buffer is empty and :meth:`feed_eof` was called. |
| 143 | |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 144 | |
| 145 | StreamWriter |
| 146 | ============ |
| 147 | |
| 148 | .. class:: StreamWriter(transport, protocol, reader, loop) |
| 149 | |
| 150 | Wraps a Transport. |
| 151 | |
| 152 | This exposes :meth:`write`, :meth:`writelines`, :meth:`can_write_eof()`, |
| 153 | :meth:`write_eof`, :meth:`get_extra_info` and :meth:`close`. It adds |
| 154 | :meth:`drain` which returns an optional :class:`Future` on which you can |
| 155 | wait for flow control. It also adds a transport attribute which references |
| 156 | the :class:`Transport` directly. |
| 157 | |
| 158 | .. attribute:: transport |
| 159 | |
| 160 | Transport. |
| 161 | |
Victor Stinner | ffbe3c6 | 2014-02-08 22:50:07 +0100 | [diff] [blame] | 162 | .. method:: can_write_eof() |
| 163 | |
| 164 | Return :const:`True` if the transport supports :meth:`write_eof`, |
| 165 | :const:`False` if not. See :meth:`WriteTransport.can_write_eof`. |
| 166 | |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 167 | .. method:: close() |
| 168 | |
| 169 | Close the transport: see :meth:`BaseTransport.close`. |
| 170 | |
| 171 | .. method:: drain() |
| 172 | |
Victor Stinner | 62f8ecc | 2014-01-24 18:47:26 +0100 | [diff] [blame] | 173 | Wait until the write buffer of the underlying transport is flushed. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 174 | |
Victor Stinner | d71dcbb | 2014-08-25 17:04:12 +0200 | [diff] [blame] | 175 | The intended use is to write:: |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 176 | |
| 177 | w.write(data) |
| 178 | yield from w.drain() |
| 179 | |
Victor Stinner | d71dcbb | 2014-08-25 17:04:12 +0200 | [diff] [blame] | 180 | When the transport buffer is full (the protocol is paused), block until |
| 181 | the buffer is (partially) drained and the protocol is resumed. When there |
| 182 | is nothing to wait for, the yield-from continues immediately. |
| 183 | |
| 184 | This method is a :ref:`coroutine <coroutine>`. |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 185 | |
| 186 | .. method:: get_extra_info(name, default=None) |
| 187 | |
| 188 | Return optional transport information: see |
| 189 | :meth:`BaseTransport.get_extra_info`. |
| 190 | |
| 191 | .. method:: write(data) |
| 192 | |
| 193 | Write some *data* bytes to the transport: see |
| 194 | :meth:`WriteTransport.write`. |
| 195 | |
| 196 | .. method:: writelines(data) |
| 197 | |
| 198 | Write a list (or any iterable) of data bytes to the transport: |
| 199 | see :meth:`WriteTransport.writelines`. |
| 200 | |
Victor Stinner | 24f8ebf | 2014-01-23 11:05:01 +0100 | [diff] [blame] | 201 | .. method:: write_eof() |
| 202 | |
| 203 | Close the write end of the transport after flushing buffered data: |
| 204 | see :meth:`WriteTransport.write_eof`. |
| 205 | |
| 206 | |
| 207 | StreamReaderProtocol |
| 208 | ==================== |
| 209 | |
| 210 | .. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, loop=None) |
| 211 | |
| 212 | Trivial helper class to adapt between :class:`Protocol` and |
| 213 | :class:`StreamReader`. Sublclass of :class:`Protocol`. |
| 214 | |
| 215 | *stream_reader* is a :class:`StreamReader` instance, *client_connected_cb* |
| 216 | is an optional function called with (stream_reader, stream_writer) when a |
| 217 | connection is made, *loop* is the event loop instance to use. |
| 218 | |
| 219 | (This is a helper class instead of making :class:`StreamReader` itself a |
| 220 | :class:`Protocol` subclass, because the :class:`StreamReader` has other |
| 221 | potential uses, and to prevent the user of the :class:`StreamReader` to |
| 222 | accidentally call inappropriate methods of the protocol.) |
| 223 | |
Victor Stinner | c520edc | 2014-01-23 11:25:48 +0100 | [diff] [blame] | 224 | |
Victor Stinner | b7f19ff | 2014-01-27 11:58:49 +0100 | [diff] [blame] | 225 | IncompleteReadError |
| 226 | =================== |
| 227 | |
| 228 | .. exception:: IncompleteReadError |
| 229 | |
Victor Stinner | 32970b8 | 2014-01-27 12:18:49 +0100 | [diff] [blame] | 230 | Incomplete read error, subclass of :exc:`EOFError`. |
Victor Stinner | b7f19ff | 2014-01-27 11:58:49 +0100 | [diff] [blame] | 231 | |
| 232 | .. attribute:: expected |
| 233 | |
| 234 | Total number of expected bytes (:class:`int`). |
| 235 | |
| 236 | .. attribute:: partial |
| 237 | |
| 238 | Read bytes string before the end of stream was reached (:class:`bytes`). |
| 239 | |
| 240 | |
Victor Stinner | 5121a9b | 2014-10-11 15:52:14 +0200 | [diff] [blame] | 241 | Stream examples |
| 242 | =============== |
| 243 | |
| 244 | Get HTTP headers |
| 245 | ---------------- |
Victor Stinner | c520edc | 2014-01-23 11:25:48 +0100 | [diff] [blame] | 246 | |
| 247 | Simple example querying HTTP headers of the URL passed on the command line:: |
| 248 | |
| 249 | import asyncio |
| 250 | import urllib.parse |
| 251 | import sys |
| 252 | |
| 253 | @asyncio.coroutine |
| 254 | def print_http_headers(url): |
| 255 | url = urllib.parse.urlsplit(url) |
Victor Stinner | 5121a9b | 2014-10-11 15:52:14 +0200 | [diff] [blame] | 256 | if url.scheme == 'https': |
| 257 | connect = asyncio.open_connection(url.hostname, 443, ssl=True) |
| 258 | else: |
| 259 | connect = asyncio.open_connection(url.hostname, 80) |
| 260 | reader, writer = yield from connect |
| 261 | query = ('HEAD {path} HTTP/1.0\r\n' |
| 262 | 'Host: {hostname}\r\n' |
| 263 | '\r\n').format(path=url.path or '/', hostname=url.hostname) |
Victor Stinner | c520edc | 2014-01-23 11:25:48 +0100 | [diff] [blame] | 264 | writer.write(query.encode('latin-1')) |
| 265 | while True: |
| 266 | line = yield from reader.readline() |
| 267 | if not line: |
| 268 | break |
| 269 | line = line.decode('latin1').rstrip() |
| 270 | if line: |
| 271 | print('HTTP header> %s' % line) |
| 272 | |
Victor Stinner | 5121a9b | 2014-10-11 15:52:14 +0200 | [diff] [blame] | 273 | # Ignore the body, close the socket |
| 274 | writer.close() |
| 275 | |
Victor Stinner | c520edc | 2014-01-23 11:25:48 +0100 | [diff] [blame] | 276 | url = sys.argv[1] |
| 277 | loop = asyncio.get_event_loop() |
| 278 | task = asyncio.async(print_http_headers(url)) |
| 279 | loop.run_until_complete(task) |
Victor Stinner | f40c663 | 2014-01-28 23:32:40 +0100 | [diff] [blame] | 280 | loop.close() |
Victor Stinner | c520edc | 2014-01-23 11:25:48 +0100 | [diff] [blame] | 281 | |
| 282 | Usage:: |
| 283 | |
| 284 | python example.py http://example.com/path/page.html |
| 285 | |
Victor Stinner | 04e6df3 | 2014-10-11 16:16:27 +0200 | [diff] [blame^] | 286 | or with HTTPS:: |
| 287 | |
| 288 | python example.py https://example.com/path/page.html |
| 289 | |
| 290 | .. _asyncio-register-socket-streams: |
| 291 | |
| 292 | Register an open socket to wait for data using streams |
| 293 | ------------------------------------------------------ |
| 294 | |
| 295 | Coroutine waiting until a socket receives data using the |
| 296 | :func:`open_connection` function:: |
| 297 | |
| 298 | import asyncio |
| 299 | import socket |
| 300 | |
| 301 | def wait_for_data(loop): |
| 302 | # Create a pair of connected sockets |
| 303 | rsock, wsock = socket.socketpair() |
| 304 | |
| 305 | # Register the open socket to wait for data |
| 306 | reader, writer = yield from asyncio.open_connection(sock=rsock, loop=loop) |
| 307 | |
| 308 | # Simulate the reception of data from the network |
| 309 | loop.call_soon(wsock.send, 'abc'.encode()) |
| 310 | |
| 311 | # Wait for data |
| 312 | data = yield from reader.read(100) |
| 313 | |
| 314 | # Got data, we are done: close the socket |
| 315 | print("Received:", data.decode()) |
| 316 | writer.close() |
| 317 | |
| 318 | # Close the second socket |
| 319 | wsock.close() |
| 320 | |
| 321 | loop = asyncio.get_event_loop() |
| 322 | loop.run_until_complete(wait_for_data(loop)) |
| 323 | loop.close() |
| 324 | |
| 325 | .. seealso:: |
| 326 | |
| 327 | The :ref:`register an open socket to wait for data using a protocol |
| 328 | <asyncio-register-socket>` example uses a low-level protocol created by the |
| 329 | :meth:`BaseEventLoop.create_connection` method. |
| 330 | |
| 331 | The :ref:`watch a file descriptor for read events |
| 332 | <asyncio-watch-read-event>` example uses the low-level |
| 333 | :meth:`BaseEventLoop.add_reader` method to register the file descriptor of a |
| 334 | socket. |
| 335 | |