Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 1 | \section{\module{asynchat} --- |
| 2 | Asynchronous socket command/response handler} |
| 3 | |
| 4 | \declaremodule{standard}{asynchat} |
| 5 | \modulesynopsis{Support for asynchronous command/response protocols.} |
| 6 | \moduleauthor{Sam Rushing}{rushing@nightmare.com} |
| 7 | \sectionauthor{Steve Holden}{sholden@holdenweb.com} |
| 8 | |
| 9 | This module builds on the \refmodule{asyncore} infrastructure, |
| 10 | simplifying asynchronous clients and servers and making it easier to |
| 11 | handle protocols whose elements are terminated by arbitrary strings, or |
| 12 | are of variable length. \refmodule{asynchat} defines the abstract class |
| 13 | \class{async_chat} that you subclass, providing implementations of the |
| 14 | \method{collect_incoming_data()} and \method{found_terminator()} |
| 15 | methods. It uses the same asynchronous loop as \refmodule{asyncore}, and |
Thomas Heller | c687401 | 2002-09-17 15:19:12 +0000 | [diff] [blame] | 16 | the two types of channel, \class{asyncore.dispatcher} and |
Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 17 | \class{asynchat.async_chat}, can freely be mixed in the channel map. |
Thomas Heller | c687401 | 2002-09-17 15:19:12 +0000 | [diff] [blame] | 18 | Typically an \class{asyncore.dispatcher} server channel generates new |
Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 19 | \class{asynchat.async_chat} channel objects as it receives incoming |
| 20 | connection requests. |
| 21 | |
| 22 | \begin{classdesc}{async_chat}{} |
Thomas Heller | c687401 | 2002-09-17 15:19:12 +0000 | [diff] [blame] | 23 | This class is an abstract subclass of \class{asyncore.dispatcher}. To make |
Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 24 | practical use of the code you must subclass \class{async_chat}, providing |
| 25 | meaningful \method{collect_incoming_data()} and \method{found_terminator()} |
Thomas Heller | c687401 | 2002-09-17 15:19:12 +0000 | [diff] [blame] | 26 | methods. The \class{asyncore.dispatcher} methods can be |
Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 27 | used, although not all make sense in a message/response context. |
| 28 | |
Thomas Heller | c687401 | 2002-09-17 15:19:12 +0000 | [diff] [blame] | 29 | Like \class{asyncore.dispatcher}, \class{async_chat} defines a set of events |
Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 30 | that are generated by an analysis of socket conditions after a |
| 31 | \cfunction{select()} call. Once the polling loop has been started the |
| 32 | \class{async_chat} object's methods are called by the event-processing |
| 33 | framework with no action on the part of the programmer. |
| 34 | |
Thomas Heller | c687401 | 2002-09-17 15:19:12 +0000 | [diff] [blame] | 35 | Unlike \class{asyncore.dispatcher}, \class{async_chat} allows you to define |
Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 36 | a first-in-first-out queue (fifo) of \emph{producers}. A producer need have |
| 37 | only one method, \method{more()}, which should return data to be transmitted |
| 38 | on the channel. The producer indicates exhaustion (\emph{i.e.} that it contains |
| 39 | no more data) by having its \method{more()} method return the empty string. At |
| 40 | this point the \class{async_chat} object removes the producer from the fifo |
| 41 | and starts using the next producer, if any. When the producer fifo is empty |
| 42 | the \method{handle_write()} method does nothing. You use the channel object's |
| 43 | \method{set_terminator()} method to describe how to recognize the end |
| 44 | of, or an important breakpoint in, an incoming transmission from the |
| 45 | remote endpoint. |
| 46 | |
| 47 | To build a functioning \class{async_chat} subclass your |
| 48 | input methods \method{collect_incoming_data()} and |
| 49 | \method{found_terminator()} must handle the data that the channel receives |
| 50 | asynchronously. The methods are described below. |
| 51 | \end{classdesc} |
| 52 | |
| 53 | \begin{methoddesc}{close_when_done}{} |
| 54 | Pushes a \code{None} on to the producer fifo. When this producer is |
| 55 | popped off the fifo it causes the channel to be closed. |
| 56 | \end{methoddesc} |
| 57 | |
| 58 | \begin{methoddesc}{collect_incoming_data}{data} |
| 59 | Called with \var{data} holding an arbitrary amount of received data. |
| 60 | The default method, which must be overridden, raises a \exception{NotImplementedError} exception. |
| 61 | \end{methoddesc} |
| 62 | |
| 63 | \begin{methoddesc}{discard_buffers}{} |
| 64 | In emergencies this method will discard any data held in the input and/or |
| 65 | output buffers and the producer fifo. |
| 66 | \end{methoddesc} |
| 67 | |
| 68 | \begin{methoddesc}{found_terminator}{} |
| 69 | Called when the incoming data stream matches the termination condition |
| 70 | set by \method{set_terminator}. The default method, which must be overridden, |
| 71 | raises a \exception{NotImplementedError} exception. The buffered input data should |
| 72 | be available via an instance attribute. |
| 73 | \end{methoddesc} |
| 74 | |
| 75 | \begin{methoddesc}{get_terminator}{} |
| 76 | Returns the current terminator for the channel. |
| 77 | \end{methoddesc} |
| 78 | |
| 79 | \begin{methoddesc}{handle_close}{} |
| 80 | Called when the channel is closed. The default method silently closes |
| 81 | the channel's socket. |
| 82 | \end{methoddesc} |
| 83 | |
| 84 | \begin{methoddesc}{handle_read}{} |
| 85 | Called when a read event fires on the channel's socket in the |
| 86 | asynchronous loop. The default method checks for the termination |
| 87 | condition established by \method{set_terminator()}, which can be either |
| 88 | the appearance of a particular string in the input stream or the receipt |
| 89 | of a particular number of characters. When the terminator is found, |
| 90 | \method{handle_read} calls the \method{found_terminator()} method after |
| 91 | calling \method{collect_incoming_data()} with any data preceding the |
| 92 | terminating condition. |
| 93 | \end{methoddesc} |
| 94 | |
| 95 | \begin{methoddesc}{handle_write}{} |
| 96 | Called when the application may write data to the channel. |
| 97 | The default method calls the \method{initiate_send()} method, which in turn |
| 98 | will call \method{refill_buffer()} to collect data from the producer |
| 99 | fifo associated with the channel. |
| 100 | \end{methoddesc} |
| 101 | |
| 102 | \begin{methoddesc}{push}{data} |
| 103 | Creates a \class{simple_producer} object (\emph{see below}) containing the data and |
| 104 | pushes it on to the channel's \code{producer_fifo} to ensure its |
| 105 | transmission. This is all you need to do to have the channel write |
| 106 | the data out to the network, although it is possible to use your |
| 107 | own producers in more complex schemes to implement encryption and |
| 108 | chunking, for example. |
| 109 | \end{methoddesc} |
| 110 | |
| 111 | \begin{methoddesc}{push_with_producer}{producer} |
| 112 | Takes a producer object and adds it to the producer fifo associated with |
| 113 | the channel. When all currently-pushed producers have been exhausted |
| 114 | the channel will consume this producer's data by calling its |
| 115 | \method{more()} method and send the data to the remote endpoint. |
| 116 | \end{methoddesc} |
| 117 | |
| 118 | \begin{methoddesc}{readable}{} |
| 119 | Should return \code{True} for the channel to be included in the set of |
| 120 | channels tested by the \cfunction{select()} loop for readability. |
| 121 | \end{methoddesc} |
| 122 | |
| 123 | \begin{methoddesc}{refill_buffer}{} |
| 124 | Refills the output buffer by calling the \method{more()} method of the |
| 125 | producer at the head of the fifo. If it is exhausted then the |
| 126 | producer is popped off the fifo and the next producer is activated. |
| 127 | If the current producer is, or becomes, \code{None} then the channel |
| 128 | is closed. |
| 129 | \end{methoddesc} |
| 130 | |
| 131 | \begin{methoddesc}{set_terminator}{term} |
| 132 | Sets the terminating condition to be recognised on the channel. \code{term} |
| 133 | may be any of three types of value, corresponding to three different ways |
| 134 | to handle incoming protocol data. |
| 135 | |
| 136 | \begin{tableii}{l|l}{}{term}{Description} |
| 137 | \lineii{\emph{string}}{Will call \method{found_terminator()} when the |
| 138 | string is found in the input stream} |
| 139 | \lineii{\emph{integer}}{Will call \method{found_terminator()} when the |
| 140 | indicated number of characters have been received} |
| 141 | \lineii{\code{None}}{The channel continues to collect data forever} |
| 142 | \end{tableii} |
| 143 | |
| 144 | Note that any data following the terminator will be available for reading by |
| 145 | the channel after \method{found_terminator()} is called. |
| 146 | \end{methoddesc} |
| 147 | |
| 148 | \begin{methoddesc}{writable}{} |
| 149 | Should return \code{True} as long as items remain on the producer fifo, |
| 150 | or the channel is connected and the channel's output buffer is non-empty. |
| 151 | \end{methoddesc} |
| 152 | |
| 153 | \subsection{asynchat - Auxiliary Classes and Functions} |
| 154 | |
| 155 | \begin{classdesc}{simple_producer}{data\optional{, buffer_size=512}} |
| 156 | A \class{simple_producer} takes a chunk of data and an optional buffer size. |
| 157 | Repeated calls to its \method{more()} method yield successive chunks of the |
| 158 | data no larger than \var{buffer_size}. |
| 159 | \end{classdesc} |
| 160 | |
| 161 | \begin{methoddesc}{more}{} |
| 162 | Produces the next chunk of information from the producer, or returns the empty string. |
| 163 | \end{methoddesc} |
| 164 | |
| 165 | \begin{classdesc}{fifo}{\optional{list=None}} |
| 166 | Each channel maintains a \class{fifo} holding data which has been pushed by the |
| 167 | application but not yet popped for writing to the channel. |
| 168 | A \class{fifo} is a list used to hold data and/or producers until they are required. |
| 169 | If the \var{list} argument is provided then it should contain producers or |
| 170 | data items to be written to the channel. |
| 171 | \end{classdesc} |
| 172 | |
| 173 | \begin{methoddesc}{is_empty}{} |
| 174 | Returns \code{True} iff the fifo is empty. |
| 175 | \end{methoddesc} |
| 176 | |
| 177 | \begin{methoddesc}{first}{} |
| 178 | Returns the least-recently \method{push()}ed item from the fifo. |
| 179 | \end{methoddesc} |
| 180 | |
| 181 | \begin{methoddesc}{push}{data} |
| 182 | Adds the given data (which may be a string or a producer object) to the |
| 183 | producer fifo. |
| 184 | \end{methoddesc} |
| 185 | |
| 186 | \begin{methoddesc}{pop}{} |
| 187 | If the fifo is not empty, returns \code{True, first()}, deleting the popped |
| 188 | item. Returns \code{False, None} for an empty fifo. |
| 189 | \end{methoddesc} |
| 190 | |
| 191 | The \module{asynchat} module also defines one utility function, which may be |
| 192 | of use in network and textual analysis operations. |
| 193 | |
| 194 | \begin{funcdesc}{find_prefix_at_end}{haystack, needle} |
| 195 | Returns \code{True} if string \var{haystack} ends with any non-empty |
| 196 | prefix of string \var{needle}. |
| 197 | \end{funcdesc} |
| 198 | |
| 199 | \subsection{asynchat Example \label{asynchat-example}} |
| 200 | |
| 201 | The following partial example shows how HTTP requests can be read with |
| 202 | \class{async_chat}. A web server might create an \class{http_request_handler} object for |
| 203 | each incoming client connection. Notice that initially the |
| 204 | channel terminator is set to match the blank line at the end of the HTTP |
| 205 | headers, and a flag indicates that the headers are being read. |
| 206 | |
| 207 | Once the headers have been read, if the request is of type POST |
| 208 | (indicating that further data are present in the input stream) then the |
| 209 | \code{Content-Length:} header is used to set a numeric terminator to |
| 210 | read the right amount of data from the channel. |
| 211 | |
| 212 | The \method{handle_request()} method is called once all relevant input |
| 213 | has been marshalled, after setting the channel terminator to \code{None} |
| 214 | to ensure that any extraneous data sent by the web client are ignored. |
| 215 | |
| 216 | \begin{verbatim} |
| 217 | class http_request_handler(asynchat.async_chat): |
| 218 | |
| 219 | def __init__(self, conn, addr, sessions, log): |
| 220 | asynchat.async_chat.__init__(self, conn=conn) |
| 221 | self.addr = addr |
| 222 | self.sessions = sessions |
| 223 | self.ibuffer = [] |
| 224 | self.obuffer = "" |
| 225 | self.set_terminator("\r\n\r\n") |
| 226 | self.reading_headers = True |
| 227 | self.handling = False |
| 228 | self.cgi_data = None |
| 229 | self.log = log |
| 230 | |
| 231 | def collect_incoming_data(self, data): |
| 232 | """Buffer the data""" |
| 233 | self.ibuffer.append(data) |
| 234 | |
| 235 | def found_terminator(self): |
| 236 | if self.reading_headers: |
| 237 | self.reading_headers = False |
Steve Holden | c8389c9 | 2002-08-04 15:27:25 +0000 | [diff] [blame] | 238 | self.parse_headers("".join(self.ibuffer)) |
Steve Holden | b1af86a | 2002-07-03 18:36:39 +0000 | [diff] [blame] | 239 | self.ibuffer = [] |
| 240 | if self.op.upper() == "POST": |
| 241 | clen = self.headers.getheader("content-length") |
| 242 | self.set_terminator(int(clen)) |
| 243 | else: |
| 244 | self.handling = True |
| 245 | self.set_terminator(None) |
| 246 | self.handle_request() |
| 247 | elif not self.handling: |
| 248 | self.set_terminator(None) # browsers sometimes over-send |
| 249 | self.cgi_data = parse(self.headers, "".join(self.ibuffer)) |
| 250 | self.handling = True |
| 251 | self.ibuffer = [] |
| 252 | self.handle_request() |
| 253 | \end{verbatim} |
| 254 | |