blob: 0a9ee7eb516f4173605d767cfa8e726a480a2b0e [file] [log] [blame]
Georg Brandl24420152008-05-26 16:32:26 +00001:mod:`http.cookiejar` --- Cookie handling for HTTP clients
2==========================================================
Georg Brandl116aa622007-08-15 14:28:22 +00003
Georg Brandl24420152008-05-26 16:32:26 +00004.. module:: http.cookiejar
Georg Brandl116aa622007-08-15 14:28:22 +00005 :synopsis: Classes for automatic handling of HTTP cookies.
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04006
Georg Brandl116aa622007-08-15 14:28:22 +00007.. moduleauthor:: John J. Lee <jjl@pobox.com>
8.. sectionauthor:: John J. Lee <jjl@pobox.com>
9
Raymond Hettinger469271d2011-01-27 20:38:46 +000010**Source code:** :source:`Lib/http/cookiejar.py`
11
12--------------
Georg Brandl116aa622007-08-15 14:28:22 +000013
Georg Brandl24420152008-05-26 16:32:26 +000014The :mod:`http.cookiejar` module defines classes for automatic handling of HTTP
Georg Brandl116aa622007-08-15 14:28:22 +000015cookies. It is useful for accessing web sites that require small pieces of data
16-- :dfn:`cookies` -- to be set on the client machine by an HTTP response from a
17web server, and then returned to the server in later HTTP requests.
18
19Both the regular Netscape cookie protocol and the protocol defined by
20:rfc:`2965` are handled. RFC 2965 handling is switched off by default.
21:rfc:`2109` cookies are parsed as Netscape cookies and subsequently treated
22either as Netscape or RFC 2965 cookies according to the 'policy' in effect.
23Note that the great majority of cookies on the Internet are Netscape cookies.
Georg Brandl24420152008-05-26 16:32:26 +000024:mod:`http.cookiejar` attempts to follow the de-facto Netscape cookie protocol (which
Georg Brandl116aa622007-08-15 14:28:22 +000025differs substantially from that set out in the original Netscape specification),
26including taking note of the ``max-age`` and ``port`` cookie-attributes
27introduced with RFC 2965.
28
29.. note::
30
31 The various named parameters found in :mailheader:`Set-Cookie` and
32 :mailheader:`Set-Cookie2` headers (eg. ``domain`` and ``expires``) are
33 conventionally referred to as :dfn:`attributes`. To distinguish them from
34 Python attributes, the documentation for this module uses the term
35 :dfn:`cookie-attribute` instead.
36
37
38The module defines the following exception:
39
40
41.. exception:: LoadError
42
43 Instances of :class:`FileCookieJar` raise this exception on failure to load
Antoine Pitrou62ab10a02011-10-12 20:10:51 +020044 cookies from a file. :exc:`LoadError` is a subclass of :exc:`OSError`.
45
46 .. versionchanged:: 3.3
47 LoadError was made a subclass of :exc:`OSError` instead of
48 :exc:`IOError`.
Georg Brandl116aa622007-08-15 14:28:22 +000049
50
51The following classes are provided:
52
53
54.. class:: CookieJar(policy=None)
55
56 *policy* is an object implementing the :class:`CookiePolicy` interface.
57
58 The :class:`CookieJar` class stores HTTP cookies. It extracts cookies from HTTP
59 requests, and returns them in HTTP responses. :class:`CookieJar` instances
60 automatically expire contained cookies when necessary. Subclasses are also
61 responsible for storing and retrieving cookies from a file or database.
62
63
64.. class:: FileCookieJar(filename, delayload=None, policy=None)
65
66 *policy* is an object implementing the :class:`CookiePolicy` interface. For the
67 other arguments, see the documentation for the corresponding attributes.
68
69 A :class:`CookieJar` which can load cookies from, and perhaps save cookies to, a
70 file on disk. Cookies are **NOT** loaded from the named file until either the
71 :meth:`load` or :meth:`revert` method is called. Subclasses of this class are
72 documented in section :ref:`file-cookie-jar-classes`.
73
Stéphane Wirtel4b219ce2019-03-01 21:40:54 +010074 .. versionchanged:: 3.8
75
76 The filename parameter supports a :term:`path-like object`.
77
Georg Brandl116aa622007-08-15 14:28:22 +000078
79.. class:: CookiePolicy()
80
81 This class is responsible for deciding whether each cookie should be accepted
82 from / returned to the server.
83
84
Paul Bailey4c339972018-10-08 13:49:29 -050085.. class:: DefaultCookiePolicy( blocked_domains=None, allowed_domains=None, netscape=True, rfc2965=False, rfc2109_as_netscape=None, hide_cookie2=False, strict_domain=False, strict_rfc2965_unverifiable=True, strict_ns_unverifiable=False, strict_ns_domain=DefaultCookiePolicy.DomainLiberal, strict_ns_set_initial_dollar=False, strict_ns_set_path=False, secure_protocols=("https", "wss") )
Georg Brandl116aa622007-08-15 14:28:22 +000086
87 Constructor arguments should be passed as keyword arguments only.
88 *blocked_domains* is a sequence of domain names that we never accept cookies
89 from, nor return cookies to. *allowed_domains* if not :const:`None`, this is a
Paul Bailey4c339972018-10-08 13:49:29 -050090 sequence of the only domains for which we accept and return cookies.
91 *secure_protocols* is a sequence of protocols for which secure cookies can be
92 added to. By default *https* and *wss* (secure websocket) are considered
93 secure protocols. For all other arguments, see the documentation for
94 :class:`CookiePolicy` and :class:`DefaultCookiePolicy` objects.
Georg Brandl116aa622007-08-15 14:28:22 +000095
96 :class:`DefaultCookiePolicy` implements the standard accept / reject rules for
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +030097 Netscape and :rfc:`2965` cookies. By default, :rfc:`2109` cookies (ie. cookies
Georg Brandl116aa622007-08-15 14:28:22 +000098 received in a :mailheader:`Set-Cookie` header with a version cookie-attribute of
99 1) are treated according to the RFC 2965 rules. However, if RFC 2965 handling
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200100 is turned off or :attr:`rfc2109_as_netscape` is ``True``, RFC 2109 cookies are
Georg Brandl116aa622007-08-15 14:28:22 +0000101 'downgraded' by the :class:`CookieJar` instance to Netscape cookies, by
102 setting the :attr:`version` attribute of the :class:`Cookie` instance to 0.
103 :class:`DefaultCookiePolicy` also provides some parameters to allow some
104 fine-tuning of policy.
105
106
107.. class:: Cookie()
108
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300109 This class represents Netscape, :rfc:`2109` and :rfc:`2965` cookies. It is not
Georg Brandl24420152008-05-26 16:32:26 +0000110 expected that users of :mod:`http.cookiejar` construct their own :class:`Cookie`
Georg Brandl116aa622007-08-15 14:28:22 +0000111 instances. Instead, if necessary, call :meth:`make_cookies` on a
112 :class:`CookieJar` instance.
113
114
115.. seealso::
116
Georg Brandl029986a2008-06-23 11:44:14 +0000117 Module :mod:`urllib.request`
Georg Brandl116aa622007-08-15 14:28:22 +0000118 URL opening with automatic cookie handling.
119
Georg Brandl24420152008-05-26 16:32:26 +0000120 Module :mod:`http.cookies`
Georg Brandl116aa622007-08-15 14:28:22 +0000121 HTTP cookie classes, principally useful for server-side code. The
Georg Brandl24420152008-05-26 16:32:26 +0000122 :mod:`http.cookiejar` and :mod:`http.cookies` modules do not depend on each
123 other.
Georg Brandl116aa622007-08-15 14:28:22 +0000124
Serhiy Storchaka6dff0202016-05-07 10:49:07 +0300125 https://curl.haxx.se/rfc/cookie_spec.html
Georg Brandl116aa622007-08-15 14:28:22 +0000126 The specification of the original Netscape cookie protocol. Though this is
127 still the dominant protocol, the 'Netscape cookie protocol' implemented by all
Georg Brandl24420152008-05-26 16:32:26 +0000128 the major browsers (and :mod:`http.cookiejar`) only bears a passing resemblance to
Georg Brandl116aa622007-08-15 14:28:22 +0000129 the one sketched out in ``cookie_spec.html``.
130
131 :rfc:`2109` - HTTP State Management Mechanism
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300132 Obsoleted by :rfc:`2965`. Uses :mailheader:`Set-Cookie` with version=1.
Georg Brandl116aa622007-08-15 14:28:22 +0000133
134 :rfc:`2965` - HTTP State Management Mechanism
135 The Netscape protocol with the bugs fixed. Uses :mailheader:`Set-Cookie2` in
136 place of :mailheader:`Set-Cookie`. Not widely used.
137
138 http://kristol.org/cookie/errata.html
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300139 Unfinished errata to :rfc:`2965`.
Georg Brandl116aa622007-08-15 14:28:22 +0000140
141 :rfc:`2964` - Use of HTTP State Management
142
143.. _cookie-jar-objects:
144
145CookieJar and FileCookieJar Objects
146-----------------------------------
147
Georg Brandl9afde1c2007-11-01 20:32:30 +0000148:class:`CookieJar` objects support the :term:`iterator` protocol for iterating over
Georg Brandl116aa622007-08-15 14:28:22 +0000149contained :class:`Cookie` objects.
150
151:class:`CookieJar` has the following methods:
152
153
154.. method:: CookieJar.add_cookie_header(request)
155
156 Add correct :mailheader:`Cookie` header to *request*.
157
158 If policy allows (ie. the :attr:`rfc2965` and :attr:`hide_cookie2` attributes of
159 the :class:`CookieJar`'s :class:`CookiePolicy` instance are true and false
160 respectively), the :mailheader:`Cookie2` header is also added when appropriate.
161
Milan Oberkirchb5bbb8a2019-07-13 12:17:17 +0200162 The *request* object (usually a :class:`urllib.request.Request` instance)
Georg Brandl029986a2008-06-23 11:44:14 +0000163 must support the methods :meth:`get_full_url`, :meth:`get_host`,
Senthil Kumaran05ec6ac2013-05-23 05:27:38 -0700164 :meth:`get_type`, :meth:`unverifiable`, :meth:`has_header`,
165 :meth:`get_header`, :meth:`header_items`, :meth:`add_unredirected_header`
166 and :attr:`origin_req_host` attribute as documented by
167 :mod:`urllib.request`.
168
169 .. versionchanged:: 3.3
170
171 *request* object needs :attr:`origin_req_host` attribute. Dependency on a
172 deprecated method :meth:`get_origin_req_host` has been removed.
Georg Brandl116aa622007-08-15 14:28:22 +0000173
174
175.. method:: CookieJar.extract_cookies(response, request)
176
177 Extract cookies from HTTP *response* and store them in the :class:`CookieJar`,
178 where allowed by policy.
179
180 The :class:`CookieJar` will look for allowable :mailheader:`Set-Cookie` and
181 :mailheader:`Set-Cookie2` headers in the *response* argument, and store cookies
182 as appropriate (subject to the :meth:`CookiePolicy.set_ok` method's approval).
183
Georg Brandl83e9f4c2008-06-12 18:52:31 +0000184 The *response* object (usually the result of a call to
Georg Brandl029986a2008-06-23 11:44:14 +0000185 :meth:`urllib.request.urlopen`, or similar) should support an :meth:`info`
Martin Panter7462b6492015-11-02 03:37:02 +0000186 method, which returns an :class:`email.message.Message` instance.
Georg Brandl116aa622007-08-15 14:28:22 +0000187
Georg Brandl029986a2008-06-23 11:44:14 +0000188 The *request* object (usually a :class:`urllib.request.Request` instance)
189 must support the methods :meth:`get_full_url`, :meth:`get_host`,
Senthil Kumaran05ec6ac2013-05-23 05:27:38 -0700190 :meth:`unverifiable`, and :attr:`origin_req_host` attribute, as documented
191 by :mod:`urllib.request`. The request is used to set default values for
Georg Brandl029986a2008-06-23 11:44:14 +0000192 cookie-attributes as well as for checking that the cookie is allowed to be
193 set.
Georg Brandl116aa622007-08-15 14:28:22 +0000194
Senthil Kumaran05ec6ac2013-05-23 05:27:38 -0700195 .. versionchanged:: 3.3
196
197 *request* object needs :attr:`origin_req_host` attribute. Dependency on a
198 deprecated method :meth:`get_origin_req_host` has been removed.
Georg Brandl116aa622007-08-15 14:28:22 +0000199
200.. method:: CookieJar.set_policy(policy)
201
202 Set the :class:`CookiePolicy` instance to be used.
203
204
205.. method:: CookieJar.make_cookies(response, request)
206
207 Return sequence of :class:`Cookie` objects extracted from *response* object.
208
209 See the documentation for :meth:`extract_cookies` for the interfaces required of
210 the *response* and *request* arguments.
211
212
213.. method:: CookieJar.set_cookie_if_ok(cookie, request)
214
215 Set a :class:`Cookie` if policy says it's OK to do so.
216
217
218.. method:: CookieJar.set_cookie(cookie)
219
220 Set a :class:`Cookie`, without checking with policy to see whether or not it
221 should be set.
222
223
224.. method:: CookieJar.clear([domain[, path[, name]]])
225
226 Clear some cookies.
227
228 If invoked without arguments, clear all cookies. If given a single argument,
229 only cookies belonging to that *domain* will be removed. If given two arguments,
230 cookies belonging to the specified *domain* and URL *path* are removed. If
231 given three arguments, then the cookie with the specified *domain*, *path* and
232 *name* is removed.
233
234 Raises :exc:`KeyError` if no matching cookie exists.
235
236
237.. method:: CookieJar.clear_session_cookies()
238
239 Discard all session cookies.
240
241 Discards all contained cookies that have a true :attr:`discard` attribute
242 (usually because they had either no ``max-age`` or ``expires`` cookie-attribute,
243 or an explicit ``discard`` cookie-attribute). For interactive browsers, the end
244 of a session usually corresponds to closing the browser window.
245
246 Note that the :meth:`save` method won't save session cookies anyway, unless you
247 ask otherwise by passing a true *ignore_discard* argument.
248
249:class:`FileCookieJar` implements the following additional methods:
250
251
252.. method:: FileCookieJar.save(filename=None, ignore_discard=False, ignore_expires=False)
253
254 Save cookies to a file.
255
256 This base class raises :exc:`NotImplementedError`. Subclasses may leave this
257 method unimplemented.
258
259 *filename* is the name of file in which to save cookies. If *filename* is not
260 specified, :attr:`self.filename` is used (whose default is the value passed to
261 the constructor, if any); if :attr:`self.filename` is :const:`None`,
262 :exc:`ValueError` is raised.
263
264 *ignore_discard*: save even cookies set to be discarded. *ignore_expires*: save
265 even cookies that have expired
266
267 The file is overwritten if it already exists, thus wiping all the cookies it
268 contains. Saved cookies can be restored later using the :meth:`load` or
269 :meth:`revert` methods.
270
271
272.. method:: FileCookieJar.load(filename=None, ignore_discard=False, ignore_expires=False)
273
274 Load cookies from a file.
275
276 Old cookies are kept unless overwritten by newly loaded ones.
277
278 Arguments are as for :meth:`save`.
279
280 The named file must be in the format understood by the class, or
Antoine Pitrou62ab10a02011-10-12 20:10:51 +0200281 :exc:`LoadError` will be raised. Also, :exc:`OSError` may be raised, for
Georg Brandl116aa622007-08-15 14:28:22 +0000282 example if the file does not exist.
283
Antoine Pitrou62ab10a02011-10-12 20:10:51 +0200284 .. versionchanged:: 3.3
285 :exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`.
286
Georg Brandl116aa622007-08-15 14:28:22 +0000287
288.. method:: FileCookieJar.revert(filename=None, ignore_discard=False, ignore_expires=False)
289
290 Clear all cookies and reload cookies from a saved file.
291
292 :meth:`revert` can raise the same exceptions as :meth:`load`. If there is a
293 failure, the object's state will not be altered.
294
295:class:`FileCookieJar` instances have the following public attributes:
296
297
298.. attribute:: FileCookieJar.filename
299
300 Filename of default file in which to keep cookies. This attribute may be
301 assigned to.
302
303
304.. attribute:: FileCookieJar.delayload
305
306 If true, load cookies lazily from disk. This attribute should not be assigned
307 to. This is only a hint, since this only affects performance, not behaviour
308 (unless the cookies on disk are changing). A :class:`CookieJar` object may
309 ignore it. None of the :class:`FileCookieJar` classes included in the standard
310 library lazily loads cookies.
311
312
313.. _file-cookie-jar-classes:
314
315FileCookieJar subclasses and co-operation with web browsers
316-----------------------------------------------------------
317
Senthil Kumaranaba088e2010-07-11 05:01:52 +0000318The following :class:`CookieJar` subclasses are provided for reading and
Serhiy Storchakaa4d170d2013-12-23 18:20:51 +0200319writing.
Georg Brandl116aa622007-08-15 14:28:22 +0000320
321.. class:: MozillaCookieJar(filename, delayload=None, policy=None)
322
323 A :class:`FileCookieJar` that can load from and save cookies to disk in the
324 Mozilla ``cookies.txt`` file format (which is also used by the Lynx and Netscape
325 browsers).
326
327 .. note::
328
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300329 This loses information about :rfc:`2965` cookies, and also about newer or
Georg Brandl116aa622007-08-15 14:28:22 +0000330 non-standard cookie-attributes such as ``port``.
331
332 .. warning::
333
334 Back up your cookies before saving if you have cookies whose loss / corruption
335 would be inconvenient (there are some subtleties which may lead to slight
336 changes in the file over a load / save round-trip).
337
338 Also note that cookies saved while Mozilla is running will get clobbered by
339 Mozilla.
340
341
342.. class:: LWPCookieJar(filename, delayload=None, policy=None)
343
344 A :class:`FileCookieJar` that can load from and save cookies to disk in format
345 compatible with the libwww-perl library's ``Set-Cookie3`` file format. This is
346 convenient if you want to store cookies in a human-readable file.
347
Stéphane Wirtel4b219ce2019-03-01 21:40:54 +0100348 .. versionchanged:: 3.8
349
350 The filename parameter supports a :term:`path-like object`.
Georg Brandl116aa622007-08-15 14:28:22 +0000351
352.. _cookie-policy-objects:
353
354CookiePolicy Objects
355--------------------
356
357Objects implementing the :class:`CookiePolicy` interface have the following
358methods:
359
360
361.. method:: CookiePolicy.set_ok(cookie, request)
362
363 Return boolean value indicating whether cookie should be accepted from server.
364
Georg Brandl24420152008-05-26 16:32:26 +0000365 *cookie* is a :class:`Cookie` instance. *request* is an object
Georg Brandl116aa622007-08-15 14:28:22 +0000366 implementing the interface defined by the documentation for
367 :meth:`CookieJar.extract_cookies`.
368
369
370.. method:: CookiePolicy.return_ok(cookie, request)
371
372 Return boolean value indicating whether cookie should be returned to server.
373
Georg Brandl24420152008-05-26 16:32:26 +0000374 *cookie* is a :class:`Cookie` instance. *request* is an object
Georg Brandl116aa622007-08-15 14:28:22 +0000375 implementing the interface defined by the documentation for
376 :meth:`CookieJar.add_cookie_header`.
377
378
379.. method:: CookiePolicy.domain_return_ok(domain, request)
380
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200381 Return ``False`` if cookies should not be returned, given cookie domain.
Georg Brandl116aa622007-08-15 14:28:22 +0000382
383 This method is an optimization. It removes the need for checking every cookie
384 with a particular domain (which might involve reading many files). Returning
385 true from :meth:`domain_return_ok` and :meth:`path_return_ok` leaves all the
386 work to :meth:`return_ok`.
387
388 If :meth:`domain_return_ok` returns true for the cookie domain,
389 :meth:`path_return_ok` is called for the cookie path. Otherwise,
390 :meth:`path_return_ok` and :meth:`return_ok` are never called for that cookie
391 domain. If :meth:`path_return_ok` returns true, :meth:`return_ok` is called
392 with the :class:`Cookie` object itself for a full check. Otherwise,
393 :meth:`return_ok` is never called for that cookie path.
394
395 Note that :meth:`domain_return_ok` is called for every *cookie* domain, not just
396 for the *request* domain. For example, the function might be called with both
397 ``".example.com"`` and ``"www.example.com"`` if the request domain is
398 ``"www.example.com"``. The same goes for :meth:`path_return_ok`.
399
400 The *request* argument is as documented for :meth:`return_ok`.
401
402
403.. method:: CookiePolicy.path_return_ok(path, request)
404
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200405 Return ``False`` if cookies should not be returned, given cookie path.
Georg Brandl116aa622007-08-15 14:28:22 +0000406
407 See the documentation for :meth:`domain_return_ok`.
408
409In addition to implementing the methods above, implementations of the
410:class:`CookiePolicy` interface must also supply the following attributes,
411indicating which protocols should be used, and how. All of these attributes may
412be assigned to.
413
414
415.. attribute:: CookiePolicy.netscape
416
417 Implement Netscape protocol.
418
419
420.. attribute:: CookiePolicy.rfc2965
421
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300422 Implement :rfc:`2965` protocol.
Georg Brandl116aa622007-08-15 14:28:22 +0000423
424
425.. attribute:: CookiePolicy.hide_cookie2
426
427 Don't add :mailheader:`Cookie2` header to requests (the presence of this header
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300428 indicates to the server that we understand :rfc:`2965` cookies).
Georg Brandl116aa622007-08-15 14:28:22 +0000429
430The most useful way to define a :class:`CookiePolicy` class is by subclassing
431from :class:`DefaultCookiePolicy` and overriding some or all of the methods
432above. :class:`CookiePolicy` itself may be used as a 'null policy' to allow
433setting and receiving any and all cookies (this is unlikely to be useful).
434
435
436.. _default-cookie-policy-objects:
437
438DefaultCookiePolicy Objects
439---------------------------
440
441Implements the standard rules for accepting and returning cookies.
442
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300443Both :rfc:`2965` and Netscape cookies are covered. RFC 2965 handling is switched
Georg Brandl116aa622007-08-15 14:28:22 +0000444off by default.
445
446The easiest way to provide your own policy is to override this class and call
447its methods in your overridden implementations before adding your own additional
448checks::
449
Georg Brandl24420152008-05-26 16:32:26 +0000450 import http.cookiejar
451 class MyCookiePolicy(http.cookiejar.DefaultCookiePolicy):
Georg Brandl116aa622007-08-15 14:28:22 +0000452 def set_ok(self, cookie, request):
Georg Brandl24420152008-05-26 16:32:26 +0000453 if not http.cookiejar.DefaultCookiePolicy.set_ok(self, cookie, request):
Georg Brandl116aa622007-08-15 14:28:22 +0000454 return False
455 if i_dont_want_to_store_this_cookie(cookie):
456 return False
457 return True
458
459In addition to the features required to implement the :class:`CookiePolicy`
460interface, this class allows you to block and allow domains from setting and
461receiving cookies. There are also some strictness switches that allow you to
462tighten up the rather loose Netscape protocol rules a little bit (at the cost of
463blocking some benign cookies).
464
Victor Stinnerfabd7bb2020-08-11 15:26:59 +0200465A domain denylist and allowlist is provided (both off by default). Only domains
466not in the denylist and present in the allowlist (if the allowlist is active)
Georg Brandl116aa622007-08-15 14:28:22 +0000467participate in cookie setting and returning. Use the *blocked_domains*
468constructor argument, and :meth:`blocked_domains` and
469:meth:`set_blocked_domains` methods (and the corresponding argument and methods
Victor Stinnerfabd7bb2020-08-11 15:26:59 +0200470for *allowed_domains*). If you set an allowlist, you can turn it off again by
Georg Brandl116aa622007-08-15 14:28:22 +0000471setting it to :const:`None`.
472
473Domains in block or allow lists that do not start with a dot must equal the
Victor Stinnerfabd7bb2020-08-11 15:26:59 +0200474cookie domain to be matched. For example, ``"example.com"`` matches a denylist
Georg Brandl116aa622007-08-15 14:28:22 +0000475entry of ``"example.com"``, but ``"www.example.com"`` does not. Domains that do
476start with a dot are matched by more specific domains too. For example, both
477``"www.example.com"`` and ``"www.coyote.example.com"`` match ``".example.com"``
478(but ``"example.com"`` itself does not). IP addresses are an exception, and
479must match exactly. For example, if blocked_domains contains ``"192.168.1.2"``
480and ``".168.1.2"``, 192.168.1.2 is blocked, but 193.168.1.2 is not.
481
482:class:`DefaultCookiePolicy` implements the following additional methods:
483
484
485.. method:: DefaultCookiePolicy.blocked_domains()
486
487 Return the sequence of blocked domains (as a tuple).
488
489
490.. method:: DefaultCookiePolicy.set_blocked_domains(blocked_domains)
491
492 Set the sequence of blocked domains.
493
494
495.. method:: DefaultCookiePolicy.is_blocked(domain)
496
Victor Stinnerfabd7bb2020-08-11 15:26:59 +0200497 Return whether *domain* is on the denylist for setting or receiving cookies.
Georg Brandl116aa622007-08-15 14:28:22 +0000498
499
500.. method:: DefaultCookiePolicy.allowed_domains()
501
502 Return :const:`None`, or the sequence of allowed domains (as a tuple).
503
504
505.. method:: DefaultCookiePolicy.set_allowed_domains(allowed_domains)
506
507 Set the sequence of allowed domains, or :const:`None`.
508
509
510.. method:: DefaultCookiePolicy.is_not_allowed(domain)
511
Victor Stinnerfabd7bb2020-08-11 15:26:59 +0200512 Return whether *domain* is not on the allowlist for setting or receiving
Georg Brandl116aa622007-08-15 14:28:22 +0000513 cookies.
514
515:class:`DefaultCookiePolicy` instances have the following attributes, which are
516all initialised from the constructor arguments of the same name, and which may
517all be assigned to.
518
519
520.. attribute:: DefaultCookiePolicy.rfc2109_as_netscape
521
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300522 If true, request that the :class:`CookieJar` instance downgrade :rfc:`2109` cookies
Georg Brandl116aa622007-08-15 14:28:22 +0000523 (ie. cookies received in a :mailheader:`Set-Cookie` header with a version
524 cookie-attribute of 1) to Netscape cookies by setting the version attribute of
525 the :class:`Cookie` instance to 0. The default value is :const:`None`, in which
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300526 case RFC 2109 cookies are downgraded if and only if :rfc:`2965` handling is turned
Georg Brandl116aa622007-08-15 14:28:22 +0000527 off. Therefore, RFC 2109 cookies are downgraded by default.
528
Georg Brandl116aa622007-08-15 14:28:22 +0000529
530General strictness switches:
531
Georg Brandl116aa622007-08-15 14:28:22 +0000532.. attribute:: DefaultCookiePolicy.strict_domain
533
534 Don't allow sites to set two-component domains with country-code top-level
535 domains like ``.co.uk``, ``.gov.uk``, ``.co.nz``.etc. This is far from perfect
536 and isn't guaranteed to work!
537
Georg Brandl116aa622007-08-15 14:28:22 +0000538
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300539:rfc:`2965` protocol strictness switches:
Georg Brandl116aa622007-08-15 14:28:22 +0000540
541.. attribute:: DefaultCookiePolicy.strict_rfc2965_unverifiable
542
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300543 Follow :rfc:`2965` rules on unverifiable transactions (usually, an unverifiable
Georg Brandl116aa622007-08-15 14:28:22 +0000544 transaction is one resulting from a redirect or a request for an image hosted on
545 another site). If this is false, cookies are *never* blocked on the basis of
546 verifiability
547
Georg Brandl116aa622007-08-15 14:28:22 +0000548
Georg Brandl55ac8f02007-09-01 13:51:09 +0000549Netscape protocol strictness switches:
Georg Brandl116aa622007-08-15 14:28:22 +0000550
551.. attribute:: DefaultCookiePolicy.strict_ns_unverifiable
552
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300553 Apply :rfc:`2965` rules on unverifiable transactions even to Netscape cookies.
Georg Brandl116aa622007-08-15 14:28:22 +0000554
555
556.. attribute:: DefaultCookiePolicy.strict_ns_domain
557
558 Flags indicating how strict to be with domain-matching rules for Netscape
559 cookies. See below for acceptable values.
560
561
562.. attribute:: DefaultCookiePolicy.strict_ns_set_initial_dollar
563
564 Ignore cookies in Set-Cookie: headers that have names starting with ``'$'``.
565
566
567.. attribute:: DefaultCookiePolicy.strict_ns_set_path
568
569 Don't allow setting cookies whose path doesn't path-match request URI.
570
571:attr:`strict_ns_domain` is a collection of flags. Its value is constructed by
572or-ing together (for example, ``DomainStrictNoDots|DomainStrictNonDomain`` means
573both flags are set).
574
575
576.. attribute:: DefaultCookiePolicy.DomainStrictNoDots
577
578 When setting cookies, the 'host prefix' must not contain a dot (eg.
579 ``www.foo.bar.com`` can't set a cookie for ``.bar.com``, because ``www.foo``
580 contains a dot).
581
582
583.. attribute:: DefaultCookiePolicy.DomainStrictNonDomain
584
585 Cookies that did not explicitly specify a ``domain`` cookie-attribute can only
586 be returned to a domain equal to the domain that set the cookie (eg.
587 ``spam.example.com`` won't be returned cookies from ``example.com`` that had no
588 ``domain`` cookie-attribute).
589
590
591.. attribute:: DefaultCookiePolicy.DomainRFC2965Match
592
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300593 When setting cookies, require a full :rfc:`2965` domain-match.
Georg Brandl116aa622007-08-15 14:28:22 +0000594
595The following attributes are provided for convenience, and are the most useful
596combinations of the above flags:
597
598
599.. attribute:: DefaultCookiePolicy.DomainLiberal
600
601 Equivalent to 0 (ie. all of the above Netscape domain strictness flags switched
602 off).
603
604
605.. attribute:: DefaultCookiePolicy.DomainStrict
606
607 Equivalent to ``DomainStrictNoDots|DomainStrictNonDomain``.
608
609
Georg Brandl116aa622007-08-15 14:28:22 +0000610Cookie Objects
611--------------
612
613:class:`Cookie` instances have Python attributes roughly corresponding to the
614standard cookie-attributes specified in the various cookie standards. The
615correspondence is not one-to-one, because there are complicated rules for
616assigning default values, because the ``max-age`` and ``expires``
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300617cookie-attributes contain equivalent information, and because :rfc:`2109` cookies
Georg Brandl24420152008-05-26 16:32:26 +0000618may be 'downgraded' by :mod:`http.cookiejar` from version 1 to version 0 (Netscape)
Georg Brandl116aa622007-08-15 14:28:22 +0000619cookies.
620
621Assignment to these attributes should not be necessary other than in rare
622circumstances in a :class:`CookiePolicy` method. The class does not enforce
623internal consistency, so you should know what you're doing if you do that.
624
625
626.. attribute:: Cookie.version
627
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300628 Integer or :const:`None`. Netscape cookies have :attr:`version` 0. :rfc:`2965` and
629 :rfc:`2109` cookies have a ``version`` cookie-attribute of 1. However, note that
Georg Brandl24420152008-05-26 16:32:26 +0000630 :mod:`http.cookiejar` may 'downgrade' RFC 2109 cookies to Netscape cookies, in which
Georg Brandl116aa622007-08-15 14:28:22 +0000631 case :attr:`version` is 0.
632
633
634.. attribute:: Cookie.name
635
636 Cookie name (a string).
637
638
639.. attribute:: Cookie.value
640
641 Cookie value (a string), or :const:`None`.
642
643
644.. attribute:: Cookie.port
645
646 String representing a port or a set of ports (eg. '80', or '80,8080'), or
647 :const:`None`.
648
649
650.. attribute:: Cookie.path
651
652 Cookie path (a string, eg. ``'/acme/rocket_launchers'``).
653
654
655.. attribute:: Cookie.secure
656
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200657 ``True`` if cookie should only be returned over a secure connection.
Georg Brandl116aa622007-08-15 14:28:22 +0000658
659
660.. attribute:: Cookie.expires
661
662 Integer expiry date in seconds since epoch, or :const:`None`. See also the
663 :meth:`is_expired` method.
664
665
666.. attribute:: Cookie.discard
667
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200668 ``True`` if this is a session cookie.
Georg Brandl116aa622007-08-15 14:28:22 +0000669
670
671.. attribute:: Cookie.comment
672
673 String comment from the server explaining the function of this cookie, or
674 :const:`None`.
675
676
677.. attribute:: Cookie.comment_url
678
679 URL linking to a comment from the server explaining the function of this cookie,
680 or :const:`None`.
681
682
683.. attribute:: Cookie.rfc2109
684
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300685 ``True`` if this cookie was received as an :rfc:`2109` cookie (ie. the cookie
Georg Brandl116aa622007-08-15 14:28:22 +0000686 arrived in a :mailheader:`Set-Cookie` header, and the value of the Version
687 cookie-attribute in that header was 1). This attribute is provided because
Georg Brandl24420152008-05-26 16:32:26 +0000688 :mod:`http.cookiejar` may 'downgrade' RFC 2109 cookies to Netscape cookies, in
Georg Brandl116aa622007-08-15 14:28:22 +0000689 which case :attr:`version` is 0.
690
Georg Brandl116aa622007-08-15 14:28:22 +0000691
692.. attribute:: Cookie.port_specified
693
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200694 ``True`` if a port or set of ports was explicitly specified by the server (in the
Georg Brandl116aa622007-08-15 14:28:22 +0000695 :mailheader:`Set-Cookie` / :mailheader:`Set-Cookie2` header).
696
697
698.. attribute:: Cookie.domain_specified
699
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200700 ``True`` if a domain was explicitly specified by the server.
Georg Brandl116aa622007-08-15 14:28:22 +0000701
702
703.. attribute:: Cookie.domain_initial_dot
704
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200705 ``True`` if the domain explicitly specified by the server began with a dot
Georg Brandl116aa622007-08-15 14:28:22 +0000706 (``'.'``).
707
708Cookies may have additional non-standard cookie-attributes. These may be
709accessed using the following methods:
710
711
712.. method:: Cookie.has_nonstandard_attr(name)
713
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200714 Return ``True`` if cookie has the named cookie-attribute.
Georg Brandl116aa622007-08-15 14:28:22 +0000715
716
717.. method:: Cookie.get_nonstandard_attr(name, default=None)
718
719 If cookie has the named cookie-attribute, return its value. Otherwise, return
720 *default*.
721
722
723.. method:: Cookie.set_nonstandard_attr(name, value)
724
725 Set the value of the named cookie-attribute.
726
727The :class:`Cookie` class also defines the following method:
728
729
Hynek Schlawackdfa46522012-05-21 11:01:54 +0200730.. method:: Cookie.is_expired(now=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000731
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200732 ``True`` if cookie has passed the time at which the server requested it should
Georg Brandl116aa622007-08-15 14:28:22 +0000733 expire. If *now* is given (in seconds since the epoch), return whether the
734 cookie has expired at the specified time.
735
736
Georg Brandl116aa622007-08-15 14:28:22 +0000737Examples
738--------
739
Georg Brandl24420152008-05-26 16:32:26 +0000740The first example shows the most common usage of :mod:`http.cookiejar`::
Georg Brandl116aa622007-08-15 14:28:22 +0000741
Georg Brandl029986a2008-06-23 11:44:14 +0000742 import http.cookiejar, urllib.request
Georg Brandl24420152008-05-26 16:32:26 +0000743 cj = http.cookiejar.CookieJar()
Georg Brandl029986a2008-06-23 11:44:14 +0000744 opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
Georg Brandl116aa622007-08-15 14:28:22 +0000745 r = opener.open("http://example.com/")
746
747This example illustrates how to open a URL using your Netscape, Mozilla, or Lynx
748cookies (assumes Unix/Netscape convention for location of the cookies file)::
749
Georg Brandl029986a2008-06-23 11:44:14 +0000750 import os, http.cookiejar, urllib.request
Georg Brandl24420152008-05-26 16:32:26 +0000751 cj = http.cookiejar.MozillaCookieJar()
Éric Araujo4dcf5022011-03-25 20:31:50 +0100752 cj.load(os.path.join(os.path.expanduser("~"), ".netscape", "cookies.txt"))
Georg Brandl029986a2008-06-23 11:44:14 +0000753 opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
Georg Brandl116aa622007-08-15 14:28:22 +0000754 r = opener.open("http://example.com/")
755
756The next example illustrates the use of :class:`DefaultCookiePolicy`. Turn on
Serhiy Storchaka0a36ac12018-05-31 07:39:00 +0300757:rfc:`2965` cookies, be more strict about domains when setting and returning
Georg Brandl116aa622007-08-15 14:28:22 +0000758Netscape cookies, and block some domains from setting cookies or having them
759returned::
760
Georg Brandl029986a2008-06-23 11:44:14 +0000761 import urllib.request
Georg Brandl24420152008-05-26 16:32:26 +0000762 from http.cookiejar import CookieJar, DefaultCookiePolicy
Georg Brandl116aa622007-08-15 14:28:22 +0000763 policy = DefaultCookiePolicy(
764 rfc2965=True, strict_ns_domain=Policy.DomainStrict,
765 blocked_domains=["ads.net", ".ads.net"])
766 cj = CookieJar(policy)
Georg Brandl029986a2008-06-23 11:44:14 +0000767 opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
Georg Brandl116aa622007-08-15 14:28:22 +0000768 r = opener.open("http://example.com/")
769