blob: 0c8f0f607314a0c558f918d2d7b903184d5acf3a [file] [log] [blame]
Senthil Kumaranaca8fd72008-06-23 04:41:59 +00001:mod:`urllib.parse` --- Parse URLs into components
2==================================================
Georg Brandl116aa622007-08-15 14:28:22 +00003
Senthil Kumaranaca8fd72008-06-23 04:41:59 +00004.. module:: urllib.parse
Georg Brandl116aa622007-08-15 14:28:22 +00005 :synopsis: Parse URLs into or assemble them from components.
6
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04007**Source code:** :source:`Lib/urllib/parse.py`
Georg Brandl116aa622007-08-15 14:28:22 +00008
9.. index::
10 single: WWW
11 single: World Wide Web
12 single: URL
13 pair: URL; parsing
14 pair: relative; URL
15
Éric Araujo19f9b712011-08-19 00:49:18 +020016--------------
17
Georg Brandl116aa622007-08-15 14:28:22 +000018This module defines a standard interface to break Uniform Resource Locator (URL)
19strings up in components (addressing scheme, network location, path etc.), to
20combine the components back into a URL string, and to convert a "relative URL"
21to an absolute URL given a "base URL."
22
23The module has been designed to match the Internet RFC on Relative Uniform
Senthil Kumaran4a27d9f2012-06-28 21:07:58 -070024Resource Locators. It supports the following URL schemes: ``file``, ``ftp``,
25``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``,
26``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtspu``, ``sftp``,
27``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, ``telnet``,
Berker Peksagf6767482016-09-16 14:43:58 +030028``wais``, ``ws``, ``wss``.
Georg Brandl116aa622007-08-15 14:28:22 +000029
Nick Coghlan9fc443c2010-11-30 15:48:08 +000030The :mod:`urllib.parse` module defines functions that fall into two broad
31categories: URL parsing and URL quoting. These are covered in detail in
32the following sections.
33
34URL Parsing
35-----------
36
37The URL parsing functions focus on splitting a URL string into its components,
38or on combining URL components into a URL string.
Georg Brandl116aa622007-08-15 14:28:22 +000039
R. David Murrayf5077aa2010-05-25 15:36:46 +000040.. function:: urlparse(urlstring, scheme='', allow_fragments=True)
Georg Brandl116aa622007-08-15 14:28:22 +000041
42 Parse a URL into six components, returning a 6-tuple. This corresponds to the
43 general structure of a URL: ``scheme://netloc/path;parameters?query#fragment``.
44 Each tuple item is a string, possibly empty. The components are not broken up in
45 smaller parts (for example, the network location is a single string), and %
46 escapes are not expanded. The delimiters as shown above are not part of the
47 result, except for a leading slash in the *path* component, which is retained if
Christian Heimesfe337bf2008-03-23 21:54:12 +000048 present. For example:
Georg Brandl116aa622007-08-15 14:28:22 +000049
Senthil Kumaranaca8fd72008-06-23 04:41:59 +000050 >>> from urllib.parse import urlparse
Georg Brandl116aa622007-08-15 14:28:22 +000051 >>> o = urlparse('http://www.cwi.nl:80/%7Eguido/Python.html')
Christian Heimesfe337bf2008-03-23 21:54:12 +000052 >>> o # doctest: +NORMALIZE_WHITESPACE
53 ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
54 params='', query='', fragment='')
Georg Brandl116aa622007-08-15 14:28:22 +000055 >>> o.scheme
56 'http'
57 >>> o.port
58 80
59 >>> o.geturl()
60 'http://www.cwi.nl:80/%7Eguido/Python.html'
61
Senthil Kumaran7089a4e2010-11-07 12:57:04 +000062 Following the syntax specifications in :rfc:`1808`, urlparse recognizes
63 a netloc only if it is properly introduced by '//'. Otherwise the
64 input is presumed to be a relative URL and thus to start with
65 a path component.
Senthil Kumaran84c7d9f2010-08-04 04:50:44 +000066
Marco Buttue65fcde2017-04-27 14:23:34 +020067 .. doctest::
68 :options: +NORMALIZE_WHITESPACE
69
Senthil Kumaranfe9230a2011-06-19 13:52:49 -070070 >>> from urllib.parse import urlparse
Senthil Kumaran84c7d9f2010-08-04 04:50:44 +000071 >>> urlparse('//www.cwi.nl:80/%7Eguido/Python.html')
72 ParseResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
73 params='', query='', fragment='')
Senthil Kumaran8fd36692013-02-26 01:02:58 -080074 >>> urlparse('www.cwi.nl/%7Eguido/Python.html')
Senthil Kumaran21b29332013-09-30 22:12:16 -070075 ParseResult(scheme='', netloc='', path='www.cwi.nl/%7Eguido/Python.html',
Senthil Kumaran84c7d9f2010-08-04 04:50:44 +000076 params='', query='', fragment='')
77 >>> urlparse('help/Python.html')
78 ParseResult(scheme='', netloc='', path='help/Python.html', params='',
79 query='', fragment='')
80
Berker Peksag89584c92015-06-25 23:38:48 +030081 The *scheme* argument gives the default addressing scheme, to be
82 used only if the URL does not specify one. It should be the same type
83 (text or bytes) as *urlstring*, except that the default value ``''`` is
84 always allowed, and is automatically converted to ``b''`` if appropriate.
Georg Brandl116aa622007-08-15 14:28:22 +000085
86 If the *allow_fragments* argument is false, fragment identifiers are not
Berker Peksag89584c92015-06-25 23:38:48 +030087 recognized. Instead, they are parsed as part of the path, parameters
88 or query component, and :attr:`fragment` is set to the empty string in
89 the return value.
Georg Brandl116aa622007-08-15 14:28:22 +000090
91 The return value is actually an instance of a subclass of :class:`tuple`. This
92 class has the following additional read-only convenience attributes:
93
94 +------------------+-------+--------------------------+----------------------+
95 | Attribute | Index | Value | Value if not present |
96 +==================+=======+==========================+======================+
Berker Peksag89584c92015-06-25 23:38:48 +030097 | :attr:`scheme` | 0 | URL scheme specifier | *scheme* parameter |
Georg Brandl116aa622007-08-15 14:28:22 +000098 +------------------+-------+--------------------------+----------------------+
99 | :attr:`netloc` | 1 | Network location part | empty string |
100 +------------------+-------+--------------------------+----------------------+
101 | :attr:`path` | 2 | Hierarchical path | empty string |
102 +------------------+-------+--------------------------+----------------------+
103 | :attr:`params` | 3 | Parameters for last path | empty string |
104 | | | element | |
105 +------------------+-------+--------------------------+----------------------+
106 | :attr:`query` | 4 | Query component | empty string |
107 +------------------+-------+--------------------------+----------------------+
108 | :attr:`fragment` | 5 | Fragment identifier | empty string |
109 +------------------+-------+--------------------------+----------------------+
110 | :attr:`username` | | User name | :const:`None` |
111 +------------------+-------+--------------------------+----------------------+
112 | :attr:`password` | | Password | :const:`None` |
113 +------------------+-------+--------------------------+----------------------+
114 | :attr:`hostname` | | Host name (lower case) | :const:`None` |
115 +------------------+-------+--------------------------+----------------------+
116 | :attr:`port` | | Port number as integer, | :const:`None` |
117 | | | if present | |
118 +------------------+-------+--------------------------+----------------------+
119
Robert Collinsdfa95c92015-08-10 09:53:30 +1200120 Reading the :attr:`port` attribute will raise a :exc:`ValueError` if
121 an invalid port is specified in the URL. See section
122 :ref:`urlparse-result-object` for more information on the result object.
Georg Brandl116aa622007-08-15 14:28:22 +0000123
Howie Benefielf6e863d2017-05-15 23:48:16 -0500124 Unmatched square brackets in the :attr:`netloc` attribute will raise a
125 :exc:`ValueError`.
126
Senthil Kumaran7a1e09f2010-04-22 12:19:46 +0000127 .. versionchanged:: 3.2
128 Added IPv6 URL parsing capabilities.
129
Georg Brandla79b8dc2012-09-29 08:59:23 +0200130 .. versionchanged:: 3.3
131 The fragment is now parsed for all URL schemes (unless *allow_fragment* is
132 false), in accordance with :rfc:`3986`. Previously, a whitelist of
133 schemes that support fragments existed.
134
Robert Collinsdfa95c92015-08-10 09:53:30 +1200135 .. versionchanged:: 3.6
136 Out-of-range port numbers now raise :exc:`ValueError`, instead of
137 returning :const:`None`.
138
Georg Brandl116aa622007-08-15 14:28:22 +0000139
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500140.. function:: parse_qs(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace', max_num_fields=None)
Facundo Batistac469d4c2008-09-03 22:49:01 +0000141
142 Parse a query string given as a string argument (data of type
143 :mimetype:`application/x-www-form-urlencoded`). Data are returned as a
144 dictionary. The dictionary keys are the unique query variable names and the
145 values are lists of values for each name.
146
147 The optional argument *keep_blank_values* is a flag indicating whether blank
Senthil Kumaranf0769e82010-08-09 19:53:52 +0000148 values in percent-encoded queries should be treated as blank strings. A true value
Facundo Batistac469d4c2008-09-03 22:49:01 +0000149 indicates that blanks should be retained as blank strings. The default false
150 value indicates that blank values are to be ignored and treated as if they were
151 not included.
152
153 The optional argument *strict_parsing* is a flag indicating what to do with
154 parsing errors. If false (the default), errors are silently ignored. If true,
155 errors raise a :exc:`ValueError` exception.
156
Victor Stinnerac71c542011-01-14 12:52:12 +0000157 The optional *encoding* and *errors* parameters specify how to decode
158 percent-encoded sequences into Unicode characters, as accepted by the
159 :meth:`bytes.decode` method.
160
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500161 The optional argument *max_num_fields* is the maximum number of fields to
162 read. If set, then throws a :exc:`ValueError` if there are more than
163 *max_num_fields* fields read.
164
Michael Foord207d2292012-09-28 14:40:44 +0100165 Use the :func:`urllib.parse.urlencode` function (with the ``doseq``
166 parameter set to ``True``) to convert such dictionaries into query
167 strings.
Facundo Batistac469d4c2008-09-03 22:49:01 +0000168
Victor Stinnerc58be2d2011-01-14 13:31:45 +0000169 .. versionchanged:: 3.2
170 Add *encoding* and *errors* parameters.
171
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500172 .. versionchanged:: 3.7.2
173 Added *max_num_fields* parameter.
Facundo Batistac469d4c2008-09-03 22:49:01 +0000174
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500175
176.. function:: parse_qsl(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace', max_num_fields=None)
Facundo Batistac469d4c2008-09-03 22:49:01 +0000177
178 Parse a query string given as a string argument (data of type
179 :mimetype:`application/x-www-form-urlencoded`). Data are returned as a list of
180 name, value pairs.
181
182 The optional argument *keep_blank_values* is a flag indicating whether blank
Senthil Kumaranf0769e82010-08-09 19:53:52 +0000183 values in percent-encoded queries should be treated as blank strings. A true value
Facundo Batistac469d4c2008-09-03 22:49:01 +0000184 indicates that blanks should be retained as blank strings. The default false
185 value indicates that blank values are to be ignored and treated as if they were
186 not included.
187
188 The optional argument *strict_parsing* is a flag indicating what to do with
189 parsing errors. If false (the default), errors are silently ignored. If true,
190 errors raise a :exc:`ValueError` exception.
191
Victor Stinnerac71c542011-01-14 12:52:12 +0000192 The optional *encoding* and *errors* parameters specify how to decode
193 percent-encoded sequences into Unicode characters, as accepted by the
194 :meth:`bytes.decode` method.
195
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500196 The optional argument *max_num_fields* is the maximum number of fields to
197 read. If set, then throws a :exc:`ValueError` if there are more than
198 *max_num_fields* fields read.
199
Facundo Batistac469d4c2008-09-03 22:49:01 +0000200 Use the :func:`urllib.parse.urlencode` function to convert such lists of pairs into
201 query strings.
202
Victor Stinnerc58be2d2011-01-14 13:31:45 +0000203 .. versionchanged:: 3.2
204 Add *encoding* and *errors* parameters.
205
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500206 .. versionchanged:: 3.7.2
207 Added *max_num_fields* parameter.
Facundo Batistac469d4c2008-09-03 22:49:01 +0000208
Georg Brandl116aa622007-08-15 14:28:22 +0000209.. function:: urlunparse(parts)
210
Georg Brandl0f7ede42008-06-23 11:23:31 +0000211 Construct a URL from a tuple as returned by ``urlparse()``. The *parts*
212 argument can be any six-item iterable. This may result in a slightly
213 different, but equivalent URL, if the URL that was parsed originally had
214 unnecessary delimiters (for example, a ``?`` with an empty query; the RFC
215 states that these are equivalent).
Georg Brandl116aa622007-08-15 14:28:22 +0000216
217
R. David Murrayf5077aa2010-05-25 15:36:46 +0000218.. function:: urlsplit(urlstring, scheme='', allow_fragments=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000219
220 This is similar to :func:`urlparse`, but does not split the params from the URL.
221 This should generally be used instead of :func:`urlparse` if the more recent URL
222 syntax allowing parameters to be applied to each segment of the *path* portion
223 of the URL (see :rfc:`2396`) is wanted. A separate function is needed to
224 separate the path segments and parameters. This function returns a 5-tuple:
225 (addressing scheme, network location, path, query, fragment identifier).
226
227 The return value is actually an instance of a subclass of :class:`tuple`. This
228 class has the following additional read-only convenience attributes:
229
230 +------------------+-------+-------------------------+----------------------+
231 | Attribute | Index | Value | Value if not present |
232 +==================+=======+=========================+======================+
Berker Peksag89584c92015-06-25 23:38:48 +0300233 | :attr:`scheme` | 0 | URL scheme specifier | *scheme* parameter |
Georg Brandl116aa622007-08-15 14:28:22 +0000234 +------------------+-------+-------------------------+----------------------+
235 | :attr:`netloc` | 1 | Network location part | empty string |
236 +------------------+-------+-------------------------+----------------------+
237 | :attr:`path` | 2 | Hierarchical path | empty string |
238 +------------------+-------+-------------------------+----------------------+
239 | :attr:`query` | 3 | Query component | empty string |
240 +------------------+-------+-------------------------+----------------------+
241 | :attr:`fragment` | 4 | Fragment identifier | empty string |
242 +------------------+-------+-------------------------+----------------------+
243 | :attr:`username` | | User name | :const:`None` |
244 +------------------+-------+-------------------------+----------------------+
245 | :attr:`password` | | Password | :const:`None` |
246 +------------------+-------+-------------------------+----------------------+
247 | :attr:`hostname` | | Host name (lower case) | :const:`None` |
248 +------------------+-------+-------------------------+----------------------+
249 | :attr:`port` | | Port number as integer, | :const:`None` |
250 | | | if present | |
251 +------------------+-------+-------------------------+----------------------+
252
Robert Collinsdfa95c92015-08-10 09:53:30 +1200253 Reading the :attr:`port` attribute will raise a :exc:`ValueError` if
254 an invalid port is specified in the URL. See section
255 :ref:`urlparse-result-object` for more information on the result object.
256
Howie Benefielf6e863d2017-05-15 23:48:16 -0500257 Unmatched square brackets in the :attr:`netloc` attribute will raise a
258 :exc:`ValueError`.
259
Robert Collinsdfa95c92015-08-10 09:53:30 +1200260 .. versionchanged:: 3.6
261 Out-of-range port numbers now raise :exc:`ValueError`, instead of
262 returning :const:`None`.
Georg Brandl116aa622007-08-15 14:28:22 +0000263
Georg Brandl116aa622007-08-15 14:28:22 +0000264
265.. function:: urlunsplit(parts)
266
Georg Brandl0f7ede42008-06-23 11:23:31 +0000267 Combine the elements of a tuple as returned by :func:`urlsplit` into a
268 complete URL as a string. The *parts* argument can be any five-item
269 iterable. This may result in a slightly different, but equivalent URL, if the
270 URL that was parsed originally had unnecessary delimiters (for example, a ?
271 with an empty query; the RFC states that these are equivalent).
Georg Brandl116aa622007-08-15 14:28:22 +0000272
Georg Brandl116aa622007-08-15 14:28:22 +0000273
Georg Brandl7f01a132009-09-16 15:58:14 +0000274.. function:: urljoin(base, url, allow_fragments=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000275
276 Construct a full ("absolute") URL by combining a "base URL" (*base*) with
277 another URL (*url*). Informally, this uses components of the base URL, in
Georg Brandl0f7ede42008-06-23 11:23:31 +0000278 particular the addressing scheme, the network location and (part of) the
279 path, to provide missing components in the relative URL. For example:
Georg Brandl116aa622007-08-15 14:28:22 +0000280
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000281 >>> from urllib.parse import urljoin
Georg Brandl116aa622007-08-15 14:28:22 +0000282 >>> urljoin('http://www.cwi.nl/%7Eguido/Python.html', 'FAQ.html')
283 'http://www.cwi.nl/%7Eguido/FAQ.html'
284
285 The *allow_fragments* argument has the same meaning and default as for
286 :func:`urlparse`.
287
288 .. note::
289
290 If *url* is an absolute URL (that is, starting with ``//`` or ``scheme://``),
291 the *url*'s host name and/or scheme will be present in the result. For example:
292
Christian Heimesfe337bf2008-03-23 21:54:12 +0000293 .. doctest::
Georg Brandl116aa622007-08-15 14:28:22 +0000294
295 >>> urljoin('http://www.cwi.nl/%7Eguido/Python.html',
296 ... '//www.python.org/%7Eguido')
297 'http://www.python.org/%7Eguido'
298
299 If you do not want that behavior, preprocess the *url* with :func:`urlsplit` and
300 :func:`urlunsplit`, removing possible *scheme* and *netloc* parts.
301
302
Antoine Pitrou55ac5b32014-08-21 19:16:17 -0400303 .. versionchanged:: 3.5
304
305 Behaviour updated to match the semantics defined in :rfc:`3986`.
306
307
Georg Brandl116aa622007-08-15 14:28:22 +0000308.. function:: urldefrag(url)
309
Georg Brandl0f7ede42008-06-23 11:23:31 +0000310 If *url* contains a fragment identifier, return a modified version of *url*
311 with no fragment identifier, and the fragment identifier as a separate
312 string. If there is no fragment identifier in *url*, return *url* unmodified
313 and an empty string.
Georg Brandl116aa622007-08-15 14:28:22 +0000314
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000315 The return value is actually an instance of a subclass of :class:`tuple`. This
316 class has the following additional read-only convenience attributes:
317
318 +------------------+-------+-------------------------+----------------------+
319 | Attribute | Index | Value | Value if not present |
320 +==================+=======+=========================+======================+
321 | :attr:`url` | 0 | URL with no fragment | empty string |
322 +------------------+-------+-------------------------+----------------------+
323 | :attr:`fragment` | 1 | Fragment identifier | empty string |
324 +------------------+-------+-------------------------+----------------------+
325
326 See section :ref:`urlparse-result-object` for more information on the result
327 object.
328
329 .. versionchanged:: 3.2
Raymond Hettinger9a236b02011-01-24 09:01:27 +0000330 Result is a structured object rather than a simple 2-tuple.
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000331
Georg Brandl009a6bd2011-01-24 19:59:08 +0000332.. _parsing-ascii-encoded-bytes:
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000333
334Parsing ASCII Encoded Bytes
335---------------------------
336
337The URL parsing functions were originally designed to operate on character
338strings only. In practice, it is useful to be able to manipulate properly
339quoted and encoded URLs as sequences of ASCII bytes. Accordingly, the
340URL parsing functions in this module all operate on :class:`bytes` and
341:class:`bytearray` objects in addition to :class:`str` objects.
342
343If :class:`str` data is passed in, the result will also contain only
344:class:`str` data. If :class:`bytes` or :class:`bytearray` data is
345passed in, the result will contain only :class:`bytes` data.
346
347Attempting to mix :class:`str` data with :class:`bytes` or
348:class:`bytearray` in a single function call will result in a
Éric Araujoff2a4ba2010-11-30 17:20:31 +0000349:exc:`TypeError` being raised, while attempting to pass in non-ASCII
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000350byte values will trigger :exc:`UnicodeDecodeError`.
351
352To support easier conversion of result objects between :class:`str` and
353:class:`bytes`, all return values from URL parsing functions provide
354either an :meth:`encode` method (when the result contains :class:`str`
355data) or a :meth:`decode` method (when the result contains :class:`bytes`
356data). The signatures of these methods match those of the corresponding
357:class:`str` and :class:`bytes` methods (except that the default encoding
358is ``'ascii'`` rather than ``'utf-8'``). Each produces a value of a
359corresponding type that contains either :class:`bytes` data (for
360:meth:`encode` methods) or :class:`str` data (for
361:meth:`decode` methods).
362
363Applications that need to operate on potentially improperly quoted URLs
364that may contain non-ASCII data will need to do their own decoding from
365bytes to characters before invoking the URL parsing methods.
366
367The behaviour described in this section applies only to the URL parsing
368functions. The URL quoting functions use their own rules when producing
369or consuming byte sequences as detailed in the documentation of the
370individual URL quoting functions.
371
372.. versionchanged:: 3.2
373 URL parsing functions now accept ASCII encoded byte sequences
374
375
376.. _urlparse-result-object:
377
378Structured Parse Results
379------------------------
380
381The result objects from the :func:`urlparse`, :func:`urlsplit` and
Georg Brandl46402372010-12-04 19:06:18 +0000382:func:`urldefrag` functions are subclasses of the :class:`tuple` type.
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000383These subclasses add the attributes listed in the documentation for
384those functions, the encoding and decoding support described in the
385previous section, as well as an additional method:
386
387.. method:: urllib.parse.SplitResult.geturl()
388
389 Return the re-combined version of the original URL as a string. This may
390 differ from the original URL in that the scheme may be normalized to lower
391 case and empty components may be dropped. Specifically, empty parameters,
392 queries, and fragment identifiers will be removed.
393
394 For :func:`urldefrag` results, only empty fragment identifiers will be removed.
395 For :func:`urlsplit` and :func:`urlparse` results, all noted changes will be
396 made to the URL returned by this method.
397
398 The result of this method remains unchanged if passed back through the original
399 parsing function:
400
401 >>> from urllib.parse import urlsplit
402 >>> url = 'HTTP://www.Python.org/doc/#'
403 >>> r1 = urlsplit(url)
404 >>> r1.geturl()
405 'http://www.Python.org/doc/'
406 >>> r2 = urlsplit(r1.geturl())
407 >>> r2.geturl()
408 'http://www.Python.org/doc/'
409
410
411The following classes provide the implementations of the structured parse
412results when operating on :class:`str` objects:
413
414.. class:: DefragResult(url, fragment)
415
416 Concrete class for :func:`urldefrag` results containing :class:`str`
417 data. The :meth:`encode` method returns a :class:`DefragResultBytes`
418 instance.
419
420 .. versionadded:: 3.2
421
422.. class:: ParseResult(scheme, netloc, path, params, query, fragment)
423
424 Concrete class for :func:`urlparse` results containing :class:`str`
425 data. The :meth:`encode` method returns a :class:`ParseResultBytes`
426 instance.
427
428.. class:: SplitResult(scheme, netloc, path, query, fragment)
429
430 Concrete class for :func:`urlsplit` results containing :class:`str`
431 data. The :meth:`encode` method returns a :class:`SplitResultBytes`
432 instance.
433
434
435The following classes provide the implementations of the parse results when
436operating on :class:`bytes` or :class:`bytearray` objects:
437
438.. class:: DefragResultBytes(url, fragment)
439
440 Concrete class for :func:`urldefrag` results containing :class:`bytes`
441 data. The :meth:`decode` method returns a :class:`DefragResult`
442 instance.
443
444 .. versionadded:: 3.2
445
446.. class:: ParseResultBytes(scheme, netloc, path, params, query, fragment)
447
448 Concrete class for :func:`urlparse` results containing :class:`bytes`
449 data. The :meth:`decode` method returns a :class:`ParseResult`
450 instance.
451
452 .. versionadded:: 3.2
453
454.. class:: SplitResultBytes(scheme, netloc, path, query, fragment)
455
456 Concrete class for :func:`urlsplit` results containing :class:`bytes`
457 data. The :meth:`decode` method returns a :class:`SplitResult`
458 instance.
459
460 .. versionadded:: 3.2
461
462
463URL Quoting
464-----------
465
466The URL quoting functions focus on taking program data and making it safe
467for use as URL components by quoting special characters and appropriately
468encoding non-ASCII text. They also support reversing these operations to
469recreate the original data from the contents of a URL component if that
470task isn't already covered by the URL parsing functions above.
Georg Brandl7f01a132009-09-16 15:58:14 +0000471
472.. function:: quote(string, safe='/', encoding=None, errors=None)
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000473
474 Replace special characters in *string* using the ``%xx`` escape. Letters,
Ratnadeep Debnath21024f02017-02-25 14:30:28 +0530475 digits, and the characters ``'_.-~'`` are never quoted. By default, this
Senthil Kumaran8aa8bbe2009-08-31 16:43:45 +0000476 function is intended for quoting the path section of URL. The optional *safe*
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000477 parameter specifies additional ASCII characters that should not be quoted
478 --- its default value is ``'/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000479
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000480 *string* may be either a :class:`str` or a :class:`bytes`.
481
Ratnadeep Debnath21024f02017-02-25 14:30:28 +0530482 .. versionchanged:: 3.7
Miss Islington (bot)0f1a1832018-05-30 22:33:23 -0700483 Moved from :rfc:`2396` to :rfc:`3986` for quoting URL strings. "~" is now
Ratnadeep Debnath21024f02017-02-25 14:30:28 +0530484 included in the set of reserved characters.
485
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000486 The optional *encoding* and *errors* parameters specify how to deal with
487 non-ASCII characters, as accepted by the :meth:`str.encode` method.
488 *encoding* defaults to ``'utf-8'``.
489 *errors* defaults to ``'strict'``, meaning unsupported characters raise a
490 :class:`UnicodeEncodeError`.
491 *encoding* and *errors* must not be supplied if *string* is a
492 :class:`bytes`, or a :class:`TypeError` is raised.
493
494 Note that ``quote(string, safe, encoding, errors)`` is equivalent to
495 ``quote_from_bytes(string.encode(encoding, errors), safe)``.
496
497 Example: ``quote('/El Niño/')`` yields ``'/El%20Ni%C3%B1o/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000498
499
Georg Brandl7f01a132009-09-16 15:58:14 +0000500.. function:: quote_plus(string, safe='', encoding=None, errors=None)
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000501
Georg Brandl0f7ede42008-06-23 11:23:31 +0000502 Like :func:`quote`, but also replace spaces by plus signs, as required for
Georg Brandl81c09db2009-07-29 07:27:08 +0000503 quoting HTML form values when building up a query string to go into a URL.
504 Plus signs in the original string are escaped unless they are included in
505 *safe*. It also does not have *safe* default to ``'/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000506
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000507 Example: ``quote_plus('/El Niño/')`` yields ``'%2FEl+Ni%C3%B1o%2F'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000508
Georg Brandl7f01a132009-09-16 15:58:14 +0000509
510.. function:: quote_from_bytes(bytes, safe='/')
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000511
512 Like :func:`quote`, but accepts a :class:`bytes` object rather than a
513 :class:`str`, and does not perform string-to-bytes encoding.
514
515 Example: ``quote_from_bytes(b'a&\xef')`` yields
516 ``'a%26%EF'``.
517
Georg Brandl7f01a132009-09-16 15:58:14 +0000518
519.. function:: unquote(string, encoding='utf-8', errors='replace')
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000520
521 Replace ``%xx`` escapes by their single-character equivalent.
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000522 The optional *encoding* and *errors* parameters specify how to decode
523 percent-encoded sequences into Unicode characters, as accepted by the
524 :meth:`bytes.decode` method.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000525
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000526 *string* must be a :class:`str`.
527
528 *encoding* defaults to ``'utf-8'``.
529 *errors* defaults to ``'replace'``, meaning invalid sequences are replaced
530 by a placeholder character.
531
532 Example: ``unquote('/El%20Ni%C3%B1o/')`` yields ``'/El Niño/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000533
534
Georg Brandl7f01a132009-09-16 15:58:14 +0000535.. function:: unquote_plus(string, encoding='utf-8', errors='replace')
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000536
Georg Brandl0f7ede42008-06-23 11:23:31 +0000537 Like :func:`unquote`, but also replace plus signs by spaces, as required for
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000538 unquoting HTML form values.
539
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000540 *string* must be a :class:`str`.
541
542 Example: ``unquote_plus('/El+Ni%C3%B1o/')`` yields ``'/El Niño/'``.
543
Georg Brandl7f01a132009-09-16 15:58:14 +0000544
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000545.. function:: unquote_to_bytes(string)
546
547 Replace ``%xx`` escapes by their single-octet equivalent, and return a
548 :class:`bytes` object.
549
550 *string* may be either a :class:`str` or a :class:`bytes`.
551
552 If it is a :class:`str`, unescaped non-ASCII characters in *string*
553 are encoded into UTF-8 bytes.
554
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000555 Example: ``unquote_to_bytes('a%26%EF')`` yields ``b'a&\xef'``.
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000556
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000557
R David Murrayc17686f2015-05-17 20:44:50 -0400558.. function:: urlencode(query, doseq=False, safe='', encoding=None, \
559 errors=None, quote_via=quote_plus)
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000560
Senthil Kumarandf022da2010-07-03 17:48:22 +0000561 Convert a mapping object or a sequence of two-element tuples, which may
Martin Pantercda85a02015-11-24 22:33:18 +0000562 contain :class:`str` or :class:`bytes` objects, to a percent-encoded ASCII
563 text string. If the resultant string is to be used as a *data* for POST
564 operation with the :func:`~urllib.request.urlopen` function, then
565 it should be encoded to bytes, otherwise it would result in a
566 :exc:`TypeError`.
Senthil Kumaran6b3434a2012-03-15 18:11:16 -0700567
Senthil Kumarandf022da2010-07-03 17:48:22 +0000568 The resulting string is a series of ``key=value`` pairs separated by ``'&'``
R David Murrayc17686f2015-05-17 20:44:50 -0400569 characters, where both *key* and *value* are quoted using the *quote_via*
570 function. By default, :func:`quote_plus` is used to quote the values, which
571 means spaces are quoted as a ``'+'`` character and '/' characters are
572 encoded as ``%2F``, which follows the standard for GET requests
573 (``application/x-www-form-urlencoded``). An alternate function that can be
574 passed as *quote_via* is :func:`quote`, which will encode spaces as ``%20``
575 and not encode '/' characters. For maximum control of what is quoted, use
576 ``quote`` and specify a value for *safe*.
577
578 When a sequence of two-element tuples is used as the *query*
Senthil Kumarandf022da2010-07-03 17:48:22 +0000579 argument, the first element of each tuple is a key and the second is a
580 value. The value element in itself can be a sequence and in that case, if
Serhiy Storchakaa97cd2e2016-10-19 16:43:42 +0300581 the optional parameter *doseq* is evaluates to ``True``, individual
Senthil Kumarandf022da2010-07-03 17:48:22 +0000582 ``key=value`` pairs separated by ``'&'`` are generated for each element of
583 the value sequence for the key. The order of parameters in the encoded
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000584 string will match the order of parameter tuples in the sequence.
Senthil Kumarandf022da2010-07-03 17:48:22 +0000585
R David Murray8c4e1122014-12-24 21:23:18 -0500586 The *safe*, *encoding*, and *errors* parameters are passed down to
R David Murrayc17686f2015-05-17 20:44:50 -0400587 *quote_via* (the *encoding* and *errors* parameters are only passed
R David Murray8c4e1122014-12-24 21:23:18 -0500588 when a query element is a :class:`str`).
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000589
590 To reverse this encoding process, :func:`parse_qs` and :func:`parse_qsl` are
591 provided in this module to parse query strings into Python data structures.
Senthil Kumarandf022da2010-07-03 17:48:22 +0000592
Senthil Kumaran29333122011-02-11 11:25:47 +0000593 Refer to :ref:`urllib examples <urllib-examples>` to find out how urlencode
594 method can be used for generating query string for a URL or data for POST.
595
Senthil Kumarandf022da2010-07-03 17:48:22 +0000596 .. versionchanged:: 3.2
Georg Brandl67b21b72010-08-17 15:07:14 +0000597 Query parameter supports bytes and string objects.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000598
R David Murrayc17686f2015-05-17 20:44:50 -0400599 .. versionadded:: 3.5
600 *quote_via* parameter.
601
Georg Brandl116aa622007-08-15 14:28:22 +0000602
603.. seealso::
604
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000605 :rfc:`3986` - Uniform Resource Identifiers
Senthil Kumaranfe9230a2011-06-19 13:52:49 -0700606 This is the current standard (STD66). Any changes to urllib.parse module
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000607 should conform to this. Certain deviations could be observed, which are
Georg Brandl6faee4e2010-09-21 14:48:28 +0000608 mostly for backward compatibility purposes and for certain de-facto
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000609 parsing requirements as commonly observed in major browsers.
610
611 :rfc:`2732` - Format for Literal IPv6 Addresses in URL's.
612 This specifies the parsing requirements of IPv6 URLs.
613
614 :rfc:`2396` - Uniform Resource Identifiers (URI): Generic Syntax
615 Document describing the generic syntactic requirements for both Uniform Resource
616 Names (URNs) and Uniform Resource Locators (URLs).
617
618 :rfc:`2368` - The mailto URL scheme.
Martin Panterfe289c02016-05-28 02:20:39 +0000619 Parsing requirements for mailto URL schemes.
Georg Brandl116aa622007-08-15 14:28:22 +0000620
621 :rfc:`1808` - Relative Uniform Resource Locators
622 This Request For Comments includes the rules for joining an absolute and a
623 relative URL, including a fair number of "Abnormal Examples" which govern the
624 treatment of border cases.
625
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000626 :rfc:`1738` - Uniform Resource Locators (URL)
627 This specifies the formal syntax and semantics of absolute URLs.