Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`asyncore` --- Asynchronous socket handler |
| 2 | =============================================== |
| 3 | |
| 4 | .. module:: asyncore |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 5 | :synopsis: A base class for developing asynchronous socket handling |
| 6 | services. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 7 | .. moduleauthor:: Sam Rushing <rushing@nightmare.com> |
| 8 | .. sectionauthor:: Christopher Petrilli <petrilli@amber.org> |
| 9 | .. sectionauthor:: Steve Holden <sholden@holdenweb.com> |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 10 | .. heavily adapted from original documentation by Sam Rushing |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 11 | |
| 12 | |
| 13 | This module provides the basic infrastructure for writing asynchronous socket |
| 14 | service clients and servers. |
| 15 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 16 | There are only two ways to have a program on a single processor do "more than |
| 17 | one thing at a time." Multi-threaded programming is the simplest and most |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 18 | popular way to do it, but there is another very different technique, that lets |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 19 | you have nearly all the advantages of multi-threading, without actually using |
| 20 | multiple threads. It's really only practical if your program is largely I/O |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 21 | bound. If your program is processor bound, then pre-emptive scheduled threads |
| 22 | are probably what you really need. Network servers are rarely processor |
| 23 | bound, however. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 24 | |
| 25 | If your operating system supports the :cfunc:`select` system call in its I/O |
| 26 | library (and nearly all do), then you can use it to juggle multiple |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 27 | communication channels at once; doing other work while your I/O is taking |
| 28 | place in the "background." Although this strategy can seem strange and |
| 29 | complex, especially at first, it is in many ways easier to understand and |
| 30 | control than multi-threaded programming. The :mod:`asyncore` module solves |
| 31 | many of the difficult problems for you, making the task of building |
| 32 | sophisticated high-performance network servers and clients a snap. For |
| 33 | "conversational" applications and protocols the companion :mod:`asynchat` |
| 34 | module is invaluable. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 35 | |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 36 | The basic idea behind both modules is to create one or more network |
| 37 | *channels*, instances of class :class:`asyncore.dispatcher` and |
| 38 | :class:`asynchat.async_chat`. Creating the channels adds them to a global |
| 39 | map, used by the :func:`loop` function if you do not provide it with your own |
| 40 | *map*. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 41 | |
| 42 | Once the initial channel(s) is(are) created, calling the :func:`loop` function |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 43 | activates channel service, which continues until the last channel (including |
| 44 | any that have been added to the map during asynchronous service) is closed. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 45 | |
| 46 | |
| 47 | .. function:: loop([timeout[, use_poll[, map[,count]]]]) |
| 48 | |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 49 | Enter a polling loop that terminates after count passes or all open |
| 50 | channels have been closed. All arguments are optional. The *count* |
| 51 | parameter defaults to None, resulting in the loop terminating only when all |
| 52 | channels have been closed. The *timeout* argument sets the timeout |
| 53 | parameter for the appropriate :func:`select` or :func:`poll` call, measured |
| 54 | in seconds; the default is 30 seconds. The *use_poll* parameter, if true, |
| 55 | indicates that :func:`poll` should be used in preference to :func:`select` |
| 56 | (the default is ``False``). |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 57 | |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 58 | The *map* parameter is a dictionary whose items are the channels to watch. |
| 59 | As channels are closed they are deleted from their map. If *map* is |
| 60 | omitted, a global map is used. Channels (instances of |
| 61 | :class:`asyncore.dispatcher`, :class:`asynchat.async_chat` and subclasses |
| 62 | thereof) can freely be mixed in the map. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 63 | |
| 64 | |
| 65 | .. class:: dispatcher() |
| 66 | |
| 67 | The :class:`dispatcher` class is a thin wrapper around a low-level socket |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 68 | object. To make it more useful, it has a few methods for event-handling |
| 69 | which are called from the asynchronous loop. Otherwise, it can be treated |
| 70 | as a normal non-blocking socket object. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 71 | |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 72 | The firing of low-level events at certain times or in certain connection |
| 73 | states tells the asynchronous loop that certain higher-level events have |
| 74 | taken place. For example, if we have asked for a socket to connect to |
| 75 | another host, we know that the connection has been made when the socket |
| 76 | becomes writable for the first time (at this point you know that you may |
| 77 | write to it with the expectation of success). The implied higher-level |
| 78 | events are: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 79 | |
| 80 | +----------------------+----------------------------------------+ |
| 81 | | Event | Description | |
| 82 | +======================+========================================+ |
R. David Murray | 78532ba | 2009-04-12 15:35:44 +0000 | [diff] [blame] | 83 | | ``handle_connect()`` | Implied by the first read or write | |
| 84 | | | event | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 85 | +----------------------+----------------------------------------+ |
| 86 | | ``handle_close()`` | Implied by a read event with no data | |
| 87 | | | available | |
| 88 | +----------------------+----------------------------------------+ |
| 89 | | ``handle_accept()`` | Implied by a read event on a listening | |
| 90 | | | socket | |
| 91 | +----------------------+----------------------------------------+ |
| 92 | |
| 93 | During asynchronous processing, each mapped channel's :meth:`readable` and |
| 94 | :meth:`writable` methods are used to determine whether the channel's socket |
Georg Brandl | 9afde1c | 2007-11-01 20:32:30 +0000 | [diff] [blame] | 95 | should be added to the list of channels :cfunc:`select`\ ed or |
| 96 | :cfunc:`poll`\ ed for read and write events. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 97 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 98 | Thus, the set of channel events is larger than the basic socket events. The |
| 99 | full set of methods that can be overridden in your subclass follows: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 100 | |
| 101 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 102 | .. method:: handle_read() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 103 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 104 | Called when the asynchronous loop detects that a :meth:`read` call on the |
| 105 | channel's socket will succeed. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 106 | |
| 107 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 108 | .. method:: handle_write() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 109 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 110 | Called when the asynchronous loop detects that a writable socket can be |
| 111 | written. Often this method will implement the necessary buffering for |
| 112 | performance. For example:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 113 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 114 | def handle_write(self): |
| 115 | sent = self.send(self.buffer) |
| 116 | self.buffer = self.buffer[sent:] |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 117 | |
| 118 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 119 | .. method:: handle_expt() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 120 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 121 | Called when there is out of band (OOB) data for a socket connection. This |
| 122 | will almost never happen, as OOB is tenuously supported and rarely used. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 123 | |
| 124 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 125 | .. method:: handle_connect() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 126 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 127 | Called when the active opener's socket actually makes a connection. Might |
| 128 | send a "welcome" banner, or initiate a protocol negotiation with the |
| 129 | remote endpoint, for example. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 130 | |
| 131 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 132 | .. method:: handle_close() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 133 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 134 | Called when the socket is closed. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 135 | |
| 136 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 137 | .. method:: handle_error() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 138 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 139 | Called when an exception is raised and not otherwise handled. The default |
| 140 | version prints a condensed traceback. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 141 | |
| 142 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 143 | .. method:: handle_accept() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 144 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 145 | Called on listening channels (passive openers) when a connection can be |
| 146 | established with a new remote endpoint that has issued a :meth:`connect` |
| 147 | call for the local endpoint. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 148 | |
| 149 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 150 | .. method:: readable() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 151 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 152 | Called each time around the asynchronous loop to determine whether a |
| 153 | channel's socket should be added to the list on which read events can |
| 154 | occur. The default method simply returns ``True``, indicating that by |
| 155 | default, all channels will be interested in read events. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 156 | |
| 157 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 158 | .. method:: writable() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 159 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 160 | Called each time around the asynchronous loop to determine whether a |
| 161 | channel's socket should be added to the list on which write events can |
| 162 | occur. The default method simply returns ``True``, indicating that by |
| 163 | default, all channels will be interested in write events. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 164 | |
| 165 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 166 | In addition, each channel delegates or extends many of the socket methods. |
| 167 | Most of these are nearly identical to their socket partners. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 168 | |
| 169 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 170 | .. method:: create_socket(family, type) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 171 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 172 | This is identical to the creation of a normal socket, and will use the |
| 173 | same options for creation. Refer to the :mod:`socket` documentation for |
| 174 | information on creating sockets. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 175 | |
| 176 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 177 | .. method:: connect(address) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 178 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 179 | As with the normal socket object, *address* is a tuple with the first |
| 180 | element the host to connect to, and the second the port number. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 181 | |
| 182 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 183 | .. method:: send(data) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 184 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 185 | Send *data* to the remote end-point of the socket. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 186 | |
| 187 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 188 | .. method:: recv(buffer_size) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 189 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 190 | Read at most *buffer_size* bytes from the socket's remote end-point. An |
| 191 | empty string implies that the channel has been closed from the other end. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 192 | |
| 193 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 194 | .. method:: listen(backlog) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 195 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 196 | Listen for connections made to the socket. The *backlog* argument |
| 197 | specifies the maximum number of queued connections and should be at least |
| 198 | 1; the maximum value is system-dependent (usually 5). |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 199 | |
| 200 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 201 | .. method:: bind(address) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 202 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 203 | Bind the socket to *address*. The socket must not already be bound. (The |
| 204 | format of *address* depends on the address family --- see above.) To mark |
| 205 | the socket as re-usable (setting the :const:`SO_REUSEADDR` option), call |
| 206 | the :class:`dispatcher` object's :meth:`set_reuse_addr` method. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 207 | |
| 208 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 209 | .. method:: accept() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 210 | |
Benjamin Peterson | e41251e | 2008-04-25 01:59:09 +0000 | [diff] [blame] | 211 | Accept a connection. The socket must be bound to an address and listening |
| 212 | for connections. The return value is a pair ``(conn, address)`` where |
| 213 | *conn* is a *new* socket object usable to send and receive data on the |
| 214 | connection, and *address* is the address bound to the socket on the other |
| 215 | end of the connection. |
| 216 | |
| 217 | |
| 218 | .. method:: close() |
| 219 | |
| 220 | Close the socket. All future operations on the socket object will fail. |
| 221 | The remote end-point will receive no more data (after queued data is |
| 222 | flushed). Sockets are automatically closed when they are |
| 223 | garbage-collected. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 224 | |
Josiah Carlson | d74900e | 2008-07-07 04:15:08 +0000 | [diff] [blame] | 225 | .. class:: file_dispatcher() |
| 226 | |
| 227 | A file_dispatcher takes a file descriptor or file object along with an |
| 228 | optional map argument and wraps it for use with the :cfunc:`poll` or |
| 229 | :cfunc:`loop` functions. If provided a file object or anything with a |
| 230 | :cfunc:`fileno` method, that method will be called and passed to the |
| 231 | :class:`file_wrapper` constructor. Availability: UNIX. |
| 232 | |
| 233 | .. class:: file_wrapper() |
| 234 | |
| 235 | A file_wrapper takes an integer file descriptor and calls :func:`os.dup` to |
| 236 | duplicate the handle so that the original handle may be closed independently |
| 237 | of the file_wrapper. This class implements sufficient methods to emulate a |
| 238 | socket for use by the :class:`file_dispatcher` class. Availability: UNIX. |
| 239 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 240 | |
| 241 | .. _asyncore-example: |
| 242 | |
| 243 | asyncore Example basic HTTP client |
| 244 | ---------------------------------- |
| 245 | |
| 246 | Here is a very basic HTTP client that uses the :class:`dispatcher` class to |
| 247 | implement its socket handling:: |
| 248 | |
| 249 | import asyncore, socket |
| 250 | |
| 251 | class http_client(asyncore.dispatcher): |
| 252 | |
| 253 | def __init__(self, host, path): |
| 254 | asyncore.dispatcher.__init__(self) |
| 255 | self.create_socket(socket.AF_INET, socket.SOCK_STREAM) |
| 256 | self.connect( (host, 80) ) |
Josiah Carlson | 1893ce7 | 2008-07-07 04:23:14 +0000 | [diff] [blame] | 257 | self.buffer = bytes('GET %s HTTP/1.0\r\n\r\n' % path, 'ascii') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 258 | |
| 259 | def handle_connect(self): |
| 260 | pass |
| 261 | |
| 262 | def handle_close(self): |
| 263 | self.close() |
| 264 | |
| 265 | def handle_read(self): |
Georg Brandl | 6911e3c | 2007-09-04 07:15:32 +0000 | [diff] [blame] | 266 | print(self.recv(8192)) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 267 | |
| 268 | def writable(self): |
| 269 | return (len(self.buffer) > 0) |
| 270 | |
| 271 | def handle_write(self): |
| 272 | sent = self.send(self.buffer) |
| 273 | self.buffer = self.buffer[sent:] |
| 274 | |
| 275 | c = http_client('www.python.org', '/') |
| 276 | |
| 277 | asyncore.loop() |