joe.gregorio | 258eefb | 2008-09-06 01:30:25 +0000 | [diff] [blame] | 1 | .. % Template for a library manual section. |
| 2 | .. % PLEASE REMOVE THE COMMENTS AFTER USING THE TEMPLATE |
| 3 | .. % |
| 4 | .. % Complete documentation on the extended LaTeX markup used for Python |
| 5 | .. % documentation is available in ``Documenting Python'', which is part |
| 6 | .. % of the standard documentation for Python. It may be found online |
| 7 | .. % at: |
| 8 | .. % |
| 9 | .. % http://www.python.org/doc/current/doc/doc.html |
| 10 | .. % ==== 0. ==== |
| 11 | .. % Copy this file to <mydir>/lib<mymodule>.tex, and edit that file |
| 12 | .. % according to the instructions below. |
| 13 | |
| 14 | .. % ==== 1. ==== |
| 15 | .. % The section prologue. Give the section a title and provide some |
| 16 | .. % meta-information. References to the module should use |
| 17 | .. % \refbimodindex, \refstmodindex, \refexmodindex or \refmodindex, as |
| 18 | .. % appropriate. |
| 19 | |
| 20 | |
| 21 | :mod:`httplib2` A comprehensive HTTP client library. |
| 22 | ===================================================== |
| 23 | |
| 24 | .. module:: httplib2 |
| 25 | .. moduleauthor:: Joe Gregorio <joe@bitworking.org> |
| 26 | .. sectionauthor:: Joe Gregorio <joe@bitworking.org> |
| 27 | |
| 28 | |
| 29 | .. % Choose one of these to specify the module module name. If there's |
| 30 | .. % an underscore in the name, use |
| 31 | .. % \declaremodule[modname]{...}{mod_name} instead. |
| 32 | .. % |
| 33 | .. % not standard, in Python |
| 34 | .. % Portability statement: Uncomment and fill in the parameter to specify the |
| 35 | .. % availability of the module. The parameter can be Unix, IRIX, SunOS, Mac, |
| 36 | .. % Windows, or lots of other stuff. When ``Mac'' is specified, the availability |
| 37 | .. % statement will say ``Macintosh'' and the Module Index may say ``Mac''. |
| 38 | .. % Please use a name that has already been used whenever applicable. If this |
| 39 | .. % is omitted, no availability statement is produced or implied. |
| 40 | .. % |
| 41 | .. % \platform{Unix} |
| 42 | .. % These apply to all modules, and may be given more than once: |
| 43 | .. % Author of the module code; |
| 44 | .. % omit if not known. |
| 45 | .. % Author of the documentation, |
| 46 | .. % even if not a module section. |
| 47 | |
| 48 | |
| 49 | |
| 50 | .. % Leave at least one blank line after this, to simplify ad-hoc tools |
| 51 | .. % that are sometimes used to massage these files. |
| 52 | |
| 53 | The :mod:`httplib2` module is a comprehensive HTTP client library with the |
| 54 | following features: |
| 55 | |
| 56 | .. % ==== 2. ==== |
| 57 | .. % Give a short overview of what the module does. |
| 58 | .. % If it is platform specific, mention this. |
| 59 | .. % Mention other important restrictions or general operating principles. |
| 60 | .. % For example: |
| 61 | |
| 62 | .. describe:: HTTP and HTTPS |
| 63 | |
| 64 | HTTPS support is only available if the socket module was compiled with SSL |
| 65 | support. |
| 66 | |
| 67 | .. describe:: Keep-Alive |
| 68 | |
| 69 | Supports HTTP 1.1 Keep-Alive, keeping the socket open and performing multiple |
| 70 | requests over the same connection if possible. |
| 71 | |
| 72 | .. describe:: Authentication |
| 73 | |
| 74 | The following three types of HTTP Authentication are supported. These can be |
| 75 | used over both HTTP and HTTPS. |
| 76 | |
| 77 | * Digest |
| 78 | * Basic |
| 79 | * WSSE |
| 80 | |
| 81 | .. describe:: Caching |
| 82 | |
| 83 | The module can optionally operate with a private cache that understands the |
| 84 | Cache-Control: header and uses both the ETag and Last-Modified cache validators. |
| 85 | |
| 86 | .. describe:: All Methods |
| 87 | |
| 88 | The module can handle any HTTP request method, not just GET and POST. |
| 89 | |
| 90 | .. describe:: Redirects |
| 91 | |
| 92 | Automatically follows 3XX redirects on GETs. |
| 93 | |
| 94 | .. describe:: Compression |
| 95 | |
| 96 | Handles both ``deflate`` and ``gzip`` types of compression. |
| 97 | |
| 98 | .. describe:: Lost update support |
| 99 | |
| 100 | Automatically adds back ETags into PUT requests to resources we have already |
| 101 | cached. This implements Section 3.2 of Detecting the Lost Update Problem Using |
| 102 | Unreserved Checkout |
| 103 | |
| 104 | The :mod:`httplib2` module defines the following variables: |
| 105 | |
| 106 | .. % ==== 3. ==== |
| 107 | .. % List the public functions defined by the module. Begin with a |
| 108 | .. % standard phrase. You may also list the exceptions and other data |
| 109 | .. % items defined in the module, insofar as they are important for the |
| 110 | .. % user. |
| 111 | .. % ---- 3.2. ---- |
| 112 | .. % Data items are described using a ``datadesc'' block. This has only |
| 113 | .. % one parameter: the item's name. |
| 114 | |
| 115 | |
| 116 | .. data:: debuglevel |
| 117 | |
| 118 | The amount of debugging information to print. The default is 0. |
| 119 | |
| 120 | The :mod:`httplib2` module may raise the following Exceptions. Note that there |
| 121 | is an option that turns exceptions into normal responses with an HTTP status |
| 122 | code indicating an error occured. See |
| 123 | :attr:`Http.force_exception_to_status_code` |
| 124 | |
| 125 | .. % --- 3.3. --- |
| 126 | .. % Exceptions are described using a ``excdesc'' block. This has only |
| 127 | .. % one parameter: the exception name. Exceptions defined as classes in |
| 128 | .. % the source code should be documented using this environment, but |
| 129 | .. % constructor parameters must be omitted. |
| 130 | |
| 131 | |
| 132 | .. exception:: HttpLib2Error |
| 133 | |
| 134 | The Base Exception for all exceptions raised by httplib2. |
| 135 | |
| 136 | |
| 137 | .. exception:: RedirectMissingLocation |
| 138 | |
| 139 | A 3xx redirect response code was provided but no Location: header was provided |
| 140 | to point to the new location. |
| 141 | |
| 142 | |
| 143 | .. exception:: RedirectLimit |
| 144 | |
| 145 | The maximum number of redirections was reached without coming to a final URI. |
| 146 | |
| 147 | |
| 148 | .. exception:: ServerNotFoundError |
| 149 | |
| 150 | Unable to resolve the host name given. |
| 151 | |
| 152 | |
| 153 | .. exception:: RelativeURIError |
| 154 | |
| 155 | A relative, as opposed to an absolute URI, was passed into request(). |
| 156 | |
| 157 | |
| 158 | .. exception:: FailedToDecompressContent |
| 159 | |
| 160 | The headers claimed that the content of the response was compressed but the |
| 161 | decompression algorithm applied to the content failed. |
| 162 | |
| 163 | |
| 164 | .. exception:: UnimplementedDigestAuthOptionError |
| 165 | |
| 166 | The server requested a type of Digest authentication that we are unfamiliar |
| 167 | with. |
| 168 | |
| 169 | |
| 170 | .. exception:: UnimplementedHmacDigestAuthOptionError |
| 171 | |
| 172 | The server requested a type of HMACDigest authentication that we are unfamiliar |
| 173 | with. |
| 174 | |
| 175 | .. % ---- 3.4. ---- |
| 176 | .. % Other standard environments: |
| 177 | .. % |
| 178 | .. % classdesc - Python classes; same arguments are funcdesc |
| 179 | .. % methoddesc - methods, like funcdesc but has an optional parameter |
| 180 | .. % to give the type name: \begin{methoddesc}[mytype]{name}{args} |
| 181 | .. % By default, the type name will be the name of the |
| 182 | .. % last class defined using classdesc. The type name |
| 183 | .. % is required if the type is implemented in C (because |
| 184 | .. % there's no classdesc) or if the class isn't directly |
| 185 | .. % documented (if it's private). |
| 186 | .. % memberdesc - data members, like datadesc, but with an optional |
| 187 | .. % type name like methoddesc. |
| 188 | |
| 189 | |
| 190 | .. class:: Http([cache=None], [timeout=None]) |
| 191 | |
| 192 | The class that represents a client HTTP interface. The *cache* parameter is |
| 193 | either the name of a directory to be used as a flat file cache, or it must an |
| 194 | object that implements the required caching interface. The *timeout* parameter |
| 195 | is the socket level timeout. |
| 196 | |
| 197 | |
| 198 | .. class:: Response(info) |
| 199 | |
| 200 | Response is a subclass of :class:`dict` and instances of this class are |
| 201 | returned from calls to Http.request. The *info* parameter is either an |
| 202 | :class:`rfc822.Message` or an :class:`httplib.HTTPResponse` object. |
| 203 | |
| 204 | |
| 205 | .. class:: FileCache(dir_name, [safe=safename]) |
| 206 | |
| 207 | FileCache implements a Cache as a directory of files. The *dir_name* parameter |
| 208 | is the name of the directory to use. If the directory does not exist then |
| 209 | FileCache attempts to create the directory. The optional *safe* parameter is a |
| 210 | funtion which generates the cache filename for each URI. A FileCache object is |
| 211 | constructed and used for caching when you pass a directory name into the |
| 212 | constructor of :class:`Http`. |
| 213 | |
| 214 | Http objects have the following methods: |
| 215 | |
| 216 | .. % If your module defines new object types (for a built-in module) or |
| 217 | .. % classes (for a module written in Python), you should list the |
| 218 | .. % methods and instance variables (if any) of each type or class in a |
| 219 | .. % separate subsection. |
| 220 | |
| 221 | .. _http-objects: |
| 222 | |
| 223 | Http Objects |
| 224 | --------------- |
| 225 | |
| 226 | .. method:: Http.request(uri, [method="GET", body=None, headers=None, redirections=DEFAULT_MAX_REDIRECTS, connection_type=None]) |
| 227 | |
| 228 | Performs a single HTTP request. The *uri* is the URI of the HTTP resource and |
| 229 | can begin with either ``http`` or ``https``. The value of *uri* must be an |
| 230 | absolute URI. |
| 231 | |
| 232 | The *method* is the HTTP method to perform, such as ``GET``, ``POST``, |
| 233 | ``DELETE``, etc. There is no restriction on the methods allowed. |
| 234 | |
| 235 | The *body* is the entity body to be sent with the request. It is a string |
| 236 | object. |
| 237 | |
| 238 | Any extra headers that are to be sent with the request should be provided in the |
| 239 | *headers* dictionary. |
| 240 | |
| 241 | The maximum number of redirect to follow before raising an exception is |
| 242 | *redirections*. The default is 5. |
| 243 | |
| 244 | The *connection_type* is the type of connection object to use. The supplied |
| 245 | class should implement the interface of httplib.HTTPConnection. |
| 246 | |
| 247 | The return value is a tuple of (response, content), the first being and instance |
| 248 | of the :class:`Response` class, the second being a string that contains the |
| 249 | response entity body. |
| 250 | |
| 251 | |
| 252 | .. method:: Http.add_credentials(name, password, [domain=None]) |
| 253 | |
| 254 | Adds a name and password that will be used when a request requires |
| 255 | authentication. Supplying the optional *domain* name will restrict these |
| 256 | credentials to only be sent to the specified domain. If *domain* is not |
| 257 | specified then the given credentials will be used to try to satisfy every HTTP |
| 258 | 401 challenge. |
| 259 | |
| 260 | |
| 261 | .. method:: Http.add_certificate(key, cert, domain) |
| 262 | |
| 263 | Add a *key* and *cert* that will be used for an SSL connection to the specified |
| 264 | domain. *keyfile* is the name of a PEM formatted file that contains your |
| 265 | private key. *certfile* is a PEM formatted certificate chain file. |
| 266 | |
| 267 | |
| 268 | .. method:: Http.clear_credentials() |
| 269 | |
| 270 | Remove all the names and passwords used for authentication. |
| 271 | |
| 272 | |
| 273 | .. attribute:: Http.follow_redirects |
| 274 | |
| 275 | If ``True``, which is the default, safe redirects are followed, where safe means |
| 276 | that the client is only doing a ``GET`` or ``HEAD`` on the URI to which it is |
| 277 | being redirected. If ``False`` then no redirects are followed. Note that a False |
| 278 | 'follow_redirects' takes precedence over a True 'follow_all_redirects'. Another |
| 279 | way of saying that is for 'follow_all_redirects' to have any affect, |
| 280 | 'follow_redirects' must be True. |
| 281 | |
| 282 | |
| 283 | .. attribute:: Http.follow_all_redirects |
| 284 | |
| 285 | If ``False``, which is the default, only safe redirects are followed, where safe |
| 286 | means that the client is only doing a ``GET`` or ``HEAD`` on the URI to which it |
| 287 | is being redirected. If ``True`` then all redirects are followed. Note that a |
| 288 | False 'follow_redirects' takes precedence over a True 'follow_all_redirects'. |
| 289 | Another way of saying that is for 'follow_all_redirects' to have any affect, |
| 290 | 'follow_redirects' must be True. |
| 291 | |
| 292 | |
| 293 | .. attribute:: Http.force_exception_to_status_code |
| 294 | |
joe.gregorio | 79c204c | 2008-09-06 15:02:44 +0000 | [diff] [blame] | 295 | If ``True`` then no :mod:`httplib2` exceptions will be |
joe.gregorio | 258eefb | 2008-09-06 01:30:25 +0000 | [diff] [blame] | 296 | thrown. Instead, those error conditions will be turned into :class:`Response` |
| 297 | objects that will be returned normally. |
| 298 | |
joe.gregorio | 79c204c | 2008-09-06 15:02:44 +0000 | [diff] [blame] | 299 | If ``False``, which is the default, then exceptions will be thrown. |
joe.gregorio | 258eefb | 2008-09-06 01:30:25 +0000 | [diff] [blame] | 300 | |
| 301 | |
joe.gregorio | 700f04d | 2008-09-06 04:46:32 +0000 | [diff] [blame] | 302 | .. attribute:: Http.optimistic_concurrency_methods |
| 303 | |
| 304 | By default a list that only contains "PUT", this attribute |
| 305 | controls which methods will get 'if-match' headers attached |
| 306 | to them from cached responses with etags. You can append |
| 307 | new items to this list to add new methods that should |
| 308 | get this support, such as "PATCH". |
| 309 | |
joe.gregorio | 258eefb | 2008-09-06 01:30:25 +0000 | [diff] [blame] | 310 | .. attribute:: Http.ignore_etag |
| 311 | |
| 312 | Defaults to ``False``. If ``True``, then any etags present in the cached |
| 313 | response are ignored when processing the current request, i.e. httplib2 does |
| 314 | **not** use 'if-match' for PUT or 'if-none-match' when GET or HEAD requests are |
| 315 | made. This is mainly to deal with broken servers which supply an etag, but |
| 316 | change it capriciously. |
| 317 | |
| 318 | If you wish to supply your own caching implementation then you will need to pass |
| 319 | in an object that supports the following methods. Note that the :mod:`memcache` |
| 320 | module supports this interface natively. |
| 321 | |
| 322 | |
| 323 | .. _cache-objects: |
| 324 | |
| 325 | Cache Objects |
| 326 | -------------- |
| 327 | |
| 328 | .. method:: Cache.get(key) |
| 329 | |
| 330 | Takes a string *key* and returns the value as a string. |
| 331 | |
| 332 | |
| 333 | .. method:: Cache.set(key, value) |
| 334 | |
| 335 | Takes a string *key* and *value* and stores it in the cache. |
| 336 | |
| 337 | |
| 338 | .. method:: Cache.delete(key) |
| 339 | |
| 340 | Deletes the cached value stored at *key*. The value of *key* is a string. |
| 341 | |
| 342 | Response objects are derived from :class:`dict` and map header names (lower case |
| 343 | with the trailing colon removed) to header values. In addition to the dict |
| 344 | methods a Response object also has: |
| 345 | |
| 346 | |
| 347 | .. _response-objects: |
| 348 | |
| 349 | Response Objects |
| 350 | ------------------ |
| 351 | |
| 352 | |
| 353 | .. attribute:: Response.fromcache |
| 354 | |
| 355 | If ``true`` the the response was returned from the cache. |
| 356 | |
| 357 | |
| 358 | .. attribute:: Response.version |
| 359 | |
| 360 | The version of HTTP that the server supports. A value of 11 means '1.1'. |
| 361 | |
| 362 | |
| 363 | .. attribute:: Response.status |
| 364 | |
| 365 | The numerical HTTP status code returned in the response. |
| 366 | |
| 367 | |
| 368 | .. attribute:: Response.reason |
| 369 | |
| 370 | The human readable component of the HTTP response status code. |
| 371 | |
| 372 | |
| 373 | .. attribute:: Response.previous |
| 374 | |
| 375 | If redirects are followed then the :class:`Response` object returned is just for |
| 376 | the very last HTTP request and *previous* points to the previous |
| 377 | :class:`Response` object. In this manner they form a chain going back through |
| 378 | the responses to the very first response. Will be ``None`` if there are no |
| 379 | previous respones. |
| 380 | |
| 381 | The Response object also populates the header ``content-location``, that |
| 382 | contains the URI that was ultimately requested. This is useful if redirects were |
| 383 | encountered, you can determine the ultimate URI that the request was sent to. |
| 384 | All Response objects contain this key value, including ``previous`` responses so |
| 385 | you can determine the entire chain of redirects. If |
| 386 | :attr:`Http.force_exception_to_status_code` is ``True`` and the number of |
| 387 | redirects has exceeded the number of allowed number of redirects then the |
| 388 | :class:`Response` object will report the error in the status code, but the |
| 389 | complete chain of previous responses will still be in tact. |
| 390 | |
| 391 | To do a simple ``GET`` request just supply the absolute URI of the resource: |
| 392 | |
| 393 | .. % ==== 4. ==== |
| 394 | .. % Now is probably a good time for a complete example. (Alternatively, |
| 395 | .. % an example giving the flavor of the module may be given before the |
| 396 | .. % detailed list of functions.) |
| 397 | |
| 398 | .. _httplib2-example: |
| 399 | |
| 400 | Examples |
| 401 | --------- |
| 402 | |
| 403 | :: |
| 404 | |
| 405 | import httplib2 |
| 406 | h = httplib2.Http() |
| 407 | resp, content = h.request("http://bitworking.org/") |
| 408 | assert resp.status == 200 |
| 409 | assert resp['content-type'] == 'text/html' |
| 410 | |
| 411 | Here is more complex example that does a PUT of some text to a resource that |
| 412 | requires authentication. The Http instance also uses a file cache in the |
| 413 | directory ``.cache``. :: |
| 414 | |
| 415 | import httplib2 |
| 416 | h = httplib2.Http(".cache") |
| 417 | h.add_credentials('name', 'password') |
| 418 | resp, content = h.request("https://example.org/chap/2", |
| 419 | "PUT", body="This is text", |
| 420 | headers={'content-type':'text/plain'} ) |
| 421 | |
| 422 | Here is an example that connects to a server that supports the Atom Publishing |
| 423 | Protocol. :: |
| 424 | |
| 425 | import httplib2 |
| 426 | h = httplib2.Http() |
| 427 | h.add_credentials(myname, mypasswd) |
| 428 | h.follow_all_redirects = True |
| 429 | headers = {'Content-Type': 'application/atom+xml'} |
| 430 | body = """<?xml version="1.0" ?> |
| 431 | <entry xmlns="http://www.w3.org/2005/Atom"> |
| 432 | <title>Atom-Powered Robots Run Amok</title> |
| 433 | <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id> |
| 434 | <updated>2003-12-13T18:30:02Z</updated> |
| 435 | <author><name>John Doe</name></author> |
| 436 | <content>Some text.</content> |
| 437 | </entry> |
| 438 | """ |
| 439 | uri = "http://www.example.com/collection/" |
| 440 | resp, content = h.request(uri, "POST", body=body, headers=headers) |
| 441 | |
| 442 | Here is an example of providing data to an HTML form processor. In this case we |
| 443 | presume this is a POST form. We need to take our data and format it as |
| 444 | "application/x-www-form-urlencoded" data and use that as a body for a POST |
| 445 | request. |
| 446 | |
| 447 | |
| 448 | :: |
| 449 | |
| 450 | >>> import httplib2 |
| 451 | >>> import urllib |
| 452 | >>> data = {'name': 'fred', 'address': '123 shady lane'} |
| 453 | >>> body = urllib.urlencode(data) |
| 454 | >>> body |
| 455 | 'name=fred&address=123+shady+lane' |
| 456 | >>> h = httplib2.Http() |
| 457 | >>> resp, content = h.request("http://example.com", method="POST", body=body) |