blob: b565e1edd32192fe7fc35d24b4e093bda9f182bb [file] [log] [blame]
Senthil Kumaranaca8fd72008-06-23 04:41:59 +00001:mod:`urllib.parse` --- Parse URLs into components
2==================================================
Georg Brandl116aa622007-08-15 14:28:22 +00003
Senthil Kumaranaca8fd72008-06-23 04:41:59 +00004.. module:: urllib.parse
Georg Brandl116aa622007-08-15 14:28:22 +00005 :synopsis: Parse URLs into or assemble them from components.
6
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04007**Source code:** :source:`Lib/urllib/parse.py`
Georg Brandl116aa622007-08-15 14:28:22 +00008
9.. index::
10 single: WWW
11 single: World Wide Web
12 single: URL
13 pair: URL; parsing
14 pair: relative; URL
15
Éric Araujo19f9b712011-08-19 00:49:18 +020016--------------
17
Georg Brandl116aa622007-08-15 14:28:22 +000018This module defines a standard interface to break Uniform Resource Locator (URL)
19strings up in components (addressing scheme, network location, path etc.), to
20combine the components back into a URL string, and to convert a "relative URL"
21to an absolute URL given a "base URL."
22
23The module has been designed to match the Internet RFC on Relative Uniform
Senthil Kumaran4a27d9f2012-06-28 21:07:58 -070024Resource Locators. It supports the following URL schemes: ``file``, ``ftp``,
25``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``,
26``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtspu``, ``sftp``,
27``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, ``telnet``,
Berker Peksagf6767482016-09-16 14:43:58 +030028``wais``, ``ws``, ``wss``.
Georg Brandl116aa622007-08-15 14:28:22 +000029
Nick Coghlan9fc443c2010-11-30 15:48:08 +000030The :mod:`urllib.parse` module defines functions that fall into two broad
31categories: URL parsing and URL quoting. These are covered in detail in
32the following sections.
33
34URL Parsing
35-----------
36
37The URL parsing functions focus on splitting a URL string into its components,
38or on combining URL components into a URL string.
Georg Brandl116aa622007-08-15 14:28:22 +000039
R. David Murrayf5077aa2010-05-25 15:36:46 +000040.. function:: urlparse(urlstring, scheme='', allow_fragments=True)
Georg Brandl116aa622007-08-15 14:28:22 +000041
42 Parse a URL into six components, returning a 6-tuple. This corresponds to the
43 general structure of a URL: ``scheme://netloc/path;parameters?query#fragment``.
44 Each tuple item is a string, possibly empty. The components are not broken up in
45 smaller parts (for example, the network location is a single string), and %
46 escapes are not expanded. The delimiters as shown above are not part of the
47 result, except for a leading slash in the *path* component, which is retained if
Christian Heimesfe337bf2008-03-23 21:54:12 +000048 present. For example:
Georg Brandl116aa622007-08-15 14:28:22 +000049
Senthil Kumaranaca8fd72008-06-23 04:41:59 +000050 >>> from urllib.parse import urlparse
Georg Brandl116aa622007-08-15 14:28:22 +000051 >>> o = urlparse('http://www.cwi.nl:80/%7Eguido/Python.html')
Christian Heimesfe337bf2008-03-23 21:54:12 +000052 >>> o # doctest: +NORMALIZE_WHITESPACE
53 ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
54 params='', query='', fragment='')
Georg Brandl116aa622007-08-15 14:28:22 +000055 >>> o.scheme
56 'http'
57 >>> o.port
58 80
59 >>> o.geturl()
60 'http://www.cwi.nl:80/%7Eguido/Python.html'
61
Senthil Kumaran7089a4e2010-11-07 12:57:04 +000062 Following the syntax specifications in :rfc:`1808`, urlparse recognizes
63 a netloc only if it is properly introduced by '//'. Otherwise the
64 input is presumed to be a relative URL and thus to start with
65 a path component.
Senthil Kumaran84c7d9f2010-08-04 04:50:44 +000066
Marco Buttue65fcde2017-04-27 14:23:34 +020067 .. doctest::
68 :options: +NORMALIZE_WHITESPACE
69
Senthil Kumaranfe9230a2011-06-19 13:52:49 -070070 >>> from urllib.parse import urlparse
Senthil Kumaran84c7d9f2010-08-04 04:50:44 +000071 >>> urlparse('//www.cwi.nl:80/%7Eguido/Python.html')
72 ParseResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
73 params='', query='', fragment='')
Senthil Kumaran8fd36692013-02-26 01:02:58 -080074 >>> urlparse('www.cwi.nl/%7Eguido/Python.html')
Senthil Kumaran21b29332013-09-30 22:12:16 -070075 ParseResult(scheme='', netloc='', path='www.cwi.nl/%7Eguido/Python.html',
Senthil Kumaran84c7d9f2010-08-04 04:50:44 +000076 params='', query='', fragment='')
77 >>> urlparse('help/Python.html')
78 ParseResult(scheme='', netloc='', path='help/Python.html', params='',
79 query='', fragment='')
80
Berker Peksag89584c92015-06-25 23:38:48 +030081 The *scheme* argument gives the default addressing scheme, to be
82 used only if the URL does not specify one. It should be the same type
83 (text or bytes) as *urlstring*, except that the default value ``''`` is
84 always allowed, and is automatically converted to ``b''`` if appropriate.
Georg Brandl116aa622007-08-15 14:28:22 +000085
86 If the *allow_fragments* argument is false, fragment identifiers are not
Berker Peksag89584c92015-06-25 23:38:48 +030087 recognized. Instead, they are parsed as part of the path, parameters
88 or query component, and :attr:`fragment` is set to the empty string in
89 the return value.
Georg Brandl116aa622007-08-15 14:28:22 +000090
91 The return value is actually an instance of a subclass of :class:`tuple`. This
92 class has the following additional read-only convenience attributes:
93
94 +------------------+-------+--------------------------+----------------------+
95 | Attribute | Index | Value | Value if not present |
96 +==================+=======+==========================+======================+
Berker Peksag89584c92015-06-25 23:38:48 +030097 | :attr:`scheme` | 0 | URL scheme specifier | *scheme* parameter |
Georg Brandl116aa622007-08-15 14:28:22 +000098 +------------------+-------+--------------------------+----------------------+
99 | :attr:`netloc` | 1 | Network location part | empty string |
100 +------------------+-------+--------------------------+----------------------+
101 | :attr:`path` | 2 | Hierarchical path | empty string |
102 +------------------+-------+--------------------------+----------------------+
103 | :attr:`params` | 3 | Parameters for last path | empty string |
104 | | | element | |
105 +------------------+-------+--------------------------+----------------------+
106 | :attr:`query` | 4 | Query component | empty string |
107 +------------------+-------+--------------------------+----------------------+
108 | :attr:`fragment` | 5 | Fragment identifier | empty string |
109 +------------------+-------+--------------------------+----------------------+
110 | :attr:`username` | | User name | :const:`None` |
111 +------------------+-------+--------------------------+----------------------+
112 | :attr:`password` | | Password | :const:`None` |
113 +------------------+-------+--------------------------+----------------------+
114 | :attr:`hostname` | | Host name (lower case) | :const:`None` |
115 +------------------+-------+--------------------------+----------------------+
116 | :attr:`port` | | Port number as integer, | :const:`None` |
117 | | | if present | |
118 +------------------+-------+--------------------------+----------------------+
119
Robert Collinsdfa95c92015-08-10 09:53:30 +1200120 Reading the :attr:`port` attribute will raise a :exc:`ValueError` if
121 an invalid port is specified in the URL. See section
122 :ref:`urlparse-result-object` for more information on the result object.
Georg Brandl116aa622007-08-15 14:28:22 +0000123
Howie Benefielf6e863d2017-05-15 23:48:16 -0500124 Unmatched square brackets in the :attr:`netloc` attribute will raise a
125 :exc:`ValueError`.
126
Steve Dowerdaad2c42019-03-07 09:08:18 -0800127 Characters in the :attr:`netloc` attribute that decompose under NFKC
128 normalization (as used by the IDNA encoding) into any of ``/``, ``?``,
129 ``#``, ``@``, or ``:`` will raise a :exc:`ValueError`. If the URL is
130 decomposed before parsing, no error will be raised.
131
Senthil Kumaran7a1e09f2010-04-22 12:19:46 +0000132 .. versionchanged:: 3.2
133 Added IPv6 URL parsing capabilities.
134
Georg Brandla79b8dc2012-09-29 08:59:23 +0200135 .. versionchanged:: 3.3
136 The fragment is now parsed for all URL schemes (unless *allow_fragment* is
137 false), in accordance with :rfc:`3986`. Previously, a whitelist of
138 schemes that support fragments existed.
139
Robert Collinsdfa95c92015-08-10 09:53:30 +1200140 .. versionchanged:: 3.6
141 Out-of-range port numbers now raise :exc:`ValueError`, instead of
142 returning :const:`None`.
143
Steve Dowerdaad2c42019-03-07 09:08:18 -0800144 .. versionchanged:: 3.7.3
145 Characters that affect netloc parsing under NFKC normalization will
146 now raise :exc:`ValueError`.
147
Georg Brandl116aa622007-08-15 14:28:22 +0000148
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500149.. function:: parse_qs(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace', max_num_fields=None)
Facundo Batistac469d4c2008-09-03 22:49:01 +0000150
151 Parse a query string given as a string argument (data of type
152 :mimetype:`application/x-www-form-urlencoded`). Data are returned as a
153 dictionary. The dictionary keys are the unique query variable names and the
154 values are lists of values for each name.
155
156 The optional argument *keep_blank_values* is a flag indicating whether blank
Senthil Kumaranf0769e82010-08-09 19:53:52 +0000157 values in percent-encoded queries should be treated as blank strings. A true value
Facundo Batistac469d4c2008-09-03 22:49:01 +0000158 indicates that blanks should be retained as blank strings. The default false
159 value indicates that blank values are to be ignored and treated as if they were
160 not included.
161
162 The optional argument *strict_parsing* is a flag indicating what to do with
163 parsing errors. If false (the default), errors are silently ignored. If true,
164 errors raise a :exc:`ValueError` exception.
165
Victor Stinnerac71c542011-01-14 12:52:12 +0000166 The optional *encoding* and *errors* parameters specify how to decode
167 percent-encoded sequences into Unicode characters, as accepted by the
168 :meth:`bytes.decode` method.
169
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500170 The optional argument *max_num_fields* is the maximum number of fields to
171 read. If set, then throws a :exc:`ValueError` if there are more than
172 *max_num_fields* fields read.
173
Michael Foord207d2292012-09-28 14:40:44 +0100174 Use the :func:`urllib.parse.urlencode` function (with the ``doseq``
175 parameter set to ``True``) to convert such dictionaries into query
176 strings.
Facundo Batistac469d4c2008-09-03 22:49:01 +0000177
Victor Stinnerc58be2d2011-01-14 13:31:45 +0000178 .. versionchanged:: 3.2
179 Add *encoding* and *errors* parameters.
180
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500181 .. versionchanged:: 3.7.2
182 Added *max_num_fields* parameter.
Facundo Batistac469d4c2008-09-03 22:49:01 +0000183
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500184
185.. function:: parse_qsl(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace', max_num_fields=None)
Facundo Batistac469d4c2008-09-03 22:49:01 +0000186
187 Parse a query string given as a string argument (data of type
188 :mimetype:`application/x-www-form-urlencoded`). Data are returned as a list of
189 name, value pairs.
190
191 The optional argument *keep_blank_values* is a flag indicating whether blank
Senthil Kumaranf0769e82010-08-09 19:53:52 +0000192 values in percent-encoded queries should be treated as blank strings. A true value
Facundo Batistac469d4c2008-09-03 22:49:01 +0000193 indicates that blanks should be retained as blank strings. The default false
194 value indicates that blank values are to be ignored and treated as if they were
195 not included.
196
197 The optional argument *strict_parsing* is a flag indicating what to do with
198 parsing errors. If false (the default), errors are silently ignored. If true,
199 errors raise a :exc:`ValueError` exception.
200
Victor Stinnerac71c542011-01-14 12:52:12 +0000201 The optional *encoding* and *errors* parameters specify how to decode
202 percent-encoded sequences into Unicode characters, as accepted by the
203 :meth:`bytes.decode` method.
204
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500205 The optional argument *max_num_fields* is the maximum number of fields to
206 read. If set, then throws a :exc:`ValueError` if there are more than
207 *max_num_fields* fields read.
208
Facundo Batistac469d4c2008-09-03 22:49:01 +0000209 Use the :func:`urllib.parse.urlencode` function to convert such lists of pairs into
210 query strings.
211
Victor Stinnerc58be2d2011-01-14 13:31:45 +0000212 .. versionchanged:: 3.2
213 Add *encoding* and *errors* parameters.
214
matthewbelisle-wfaa0a21a2018-10-30 16:15:24 -0500215 .. versionchanged:: 3.7.2
216 Added *max_num_fields* parameter.
Facundo Batistac469d4c2008-09-03 22:49:01 +0000217
Georg Brandl116aa622007-08-15 14:28:22 +0000218.. function:: urlunparse(parts)
219
Georg Brandl0f7ede42008-06-23 11:23:31 +0000220 Construct a URL from a tuple as returned by ``urlparse()``. The *parts*
221 argument can be any six-item iterable. This may result in a slightly
222 different, but equivalent URL, if the URL that was parsed originally had
223 unnecessary delimiters (for example, a ``?`` with an empty query; the RFC
224 states that these are equivalent).
Georg Brandl116aa622007-08-15 14:28:22 +0000225
226
R. David Murrayf5077aa2010-05-25 15:36:46 +0000227.. function:: urlsplit(urlstring, scheme='', allow_fragments=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000228
229 This is similar to :func:`urlparse`, but does not split the params from the URL.
230 This should generally be used instead of :func:`urlparse` if the more recent URL
231 syntax allowing parameters to be applied to each segment of the *path* portion
232 of the URL (see :rfc:`2396`) is wanted. A separate function is needed to
233 separate the path segments and parameters. This function returns a 5-tuple:
234 (addressing scheme, network location, path, query, fragment identifier).
235
236 The return value is actually an instance of a subclass of :class:`tuple`. This
237 class has the following additional read-only convenience attributes:
238
239 +------------------+-------+-------------------------+----------------------+
240 | Attribute | Index | Value | Value if not present |
241 +==================+=======+=========================+======================+
Berker Peksag89584c92015-06-25 23:38:48 +0300242 | :attr:`scheme` | 0 | URL scheme specifier | *scheme* parameter |
Georg Brandl116aa622007-08-15 14:28:22 +0000243 +------------------+-------+-------------------------+----------------------+
244 | :attr:`netloc` | 1 | Network location part | empty string |
245 +------------------+-------+-------------------------+----------------------+
246 | :attr:`path` | 2 | Hierarchical path | empty string |
247 +------------------+-------+-------------------------+----------------------+
248 | :attr:`query` | 3 | Query component | empty string |
249 +------------------+-------+-------------------------+----------------------+
250 | :attr:`fragment` | 4 | Fragment identifier | empty string |
251 +------------------+-------+-------------------------+----------------------+
252 | :attr:`username` | | User name | :const:`None` |
253 +------------------+-------+-------------------------+----------------------+
254 | :attr:`password` | | Password | :const:`None` |
255 +------------------+-------+-------------------------+----------------------+
256 | :attr:`hostname` | | Host name (lower case) | :const:`None` |
257 +------------------+-------+-------------------------+----------------------+
258 | :attr:`port` | | Port number as integer, | :const:`None` |
259 | | | if present | |
260 +------------------+-------+-------------------------+----------------------+
261
Robert Collinsdfa95c92015-08-10 09:53:30 +1200262 Reading the :attr:`port` attribute will raise a :exc:`ValueError` if
263 an invalid port is specified in the URL. See section
264 :ref:`urlparse-result-object` for more information on the result object.
265
Howie Benefielf6e863d2017-05-15 23:48:16 -0500266 Unmatched square brackets in the :attr:`netloc` attribute will raise a
267 :exc:`ValueError`.
268
Steve Dowerdaad2c42019-03-07 09:08:18 -0800269 Characters in the :attr:`netloc` attribute that decompose under NFKC
270 normalization (as used by the IDNA encoding) into any of ``/``, ``?``,
271 ``#``, ``@``, or ``:`` will raise a :exc:`ValueError`. If the URL is
272 decomposed before parsing, no error will be raised.
273
Robert Collinsdfa95c92015-08-10 09:53:30 +1200274 .. versionchanged:: 3.6
275 Out-of-range port numbers now raise :exc:`ValueError`, instead of
276 returning :const:`None`.
Georg Brandl116aa622007-08-15 14:28:22 +0000277
Steve Dowerdaad2c42019-03-07 09:08:18 -0800278 .. versionchanged:: 3.7.3
279 Characters that affect netloc parsing under NFKC normalization will
280 now raise :exc:`ValueError`.
281
Georg Brandl116aa622007-08-15 14:28:22 +0000282
283.. function:: urlunsplit(parts)
284
Georg Brandl0f7ede42008-06-23 11:23:31 +0000285 Combine the elements of a tuple as returned by :func:`urlsplit` into a
286 complete URL as a string. The *parts* argument can be any five-item
287 iterable. This may result in a slightly different, but equivalent URL, if the
288 URL that was parsed originally had unnecessary delimiters (for example, a ?
289 with an empty query; the RFC states that these are equivalent).
Georg Brandl116aa622007-08-15 14:28:22 +0000290
Georg Brandl116aa622007-08-15 14:28:22 +0000291
Georg Brandl7f01a132009-09-16 15:58:14 +0000292.. function:: urljoin(base, url, allow_fragments=True)
Georg Brandl116aa622007-08-15 14:28:22 +0000293
294 Construct a full ("absolute") URL by combining a "base URL" (*base*) with
295 another URL (*url*). Informally, this uses components of the base URL, in
Georg Brandl0f7ede42008-06-23 11:23:31 +0000296 particular the addressing scheme, the network location and (part of) the
297 path, to provide missing components in the relative URL. For example:
Georg Brandl116aa622007-08-15 14:28:22 +0000298
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000299 >>> from urllib.parse import urljoin
Georg Brandl116aa622007-08-15 14:28:22 +0000300 >>> urljoin('http://www.cwi.nl/%7Eguido/Python.html', 'FAQ.html')
301 'http://www.cwi.nl/%7Eguido/FAQ.html'
302
303 The *allow_fragments* argument has the same meaning and default as for
304 :func:`urlparse`.
305
306 .. note::
307
308 If *url* is an absolute URL (that is, starting with ``//`` or ``scheme://``),
309 the *url*'s host name and/or scheme will be present in the result. For example:
310
Christian Heimesfe337bf2008-03-23 21:54:12 +0000311 .. doctest::
Georg Brandl116aa622007-08-15 14:28:22 +0000312
313 >>> urljoin('http://www.cwi.nl/%7Eguido/Python.html',
314 ... '//www.python.org/%7Eguido')
315 'http://www.python.org/%7Eguido'
316
317 If you do not want that behavior, preprocess the *url* with :func:`urlsplit` and
318 :func:`urlunsplit`, removing possible *scheme* and *netloc* parts.
319
320
Antoine Pitrou55ac5b32014-08-21 19:16:17 -0400321 .. versionchanged:: 3.5
322
323 Behaviour updated to match the semantics defined in :rfc:`3986`.
324
325
Georg Brandl116aa622007-08-15 14:28:22 +0000326.. function:: urldefrag(url)
327
Georg Brandl0f7ede42008-06-23 11:23:31 +0000328 If *url* contains a fragment identifier, return a modified version of *url*
329 with no fragment identifier, and the fragment identifier as a separate
330 string. If there is no fragment identifier in *url*, return *url* unmodified
331 and an empty string.
Georg Brandl116aa622007-08-15 14:28:22 +0000332
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000333 The return value is actually an instance of a subclass of :class:`tuple`. This
334 class has the following additional read-only convenience attributes:
335
336 +------------------+-------+-------------------------+----------------------+
337 | Attribute | Index | Value | Value if not present |
338 +==================+=======+=========================+======================+
339 | :attr:`url` | 0 | URL with no fragment | empty string |
340 +------------------+-------+-------------------------+----------------------+
341 | :attr:`fragment` | 1 | Fragment identifier | empty string |
342 +------------------+-------+-------------------------+----------------------+
343
344 See section :ref:`urlparse-result-object` for more information on the result
345 object.
346
347 .. versionchanged:: 3.2
Raymond Hettinger9a236b02011-01-24 09:01:27 +0000348 Result is a structured object rather than a simple 2-tuple.
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000349
Georg Brandl009a6bd2011-01-24 19:59:08 +0000350.. _parsing-ascii-encoded-bytes:
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000351
352Parsing ASCII Encoded Bytes
353---------------------------
354
355The URL parsing functions were originally designed to operate on character
356strings only. In practice, it is useful to be able to manipulate properly
357quoted and encoded URLs as sequences of ASCII bytes. Accordingly, the
358URL parsing functions in this module all operate on :class:`bytes` and
359:class:`bytearray` objects in addition to :class:`str` objects.
360
361If :class:`str` data is passed in, the result will also contain only
362:class:`str` data. If :class:`bytes` or :class:`bytearray` data is
363passed in, the result will contain only :class:`bytes` data.
364
365Attempting to mix :class:`str` data with :class:`bytes` or
366:class:`bytearray` in a single function call will result in a
Éric Araujoff2a4ba2010-11-30 17:20:31 +0000367:exc:`TypeError` being raised, while attempting to pass in non-ASCII
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000368byte values will trigger :exc:`UnicodeDecodeError`.
369
370To support easier conversion of result objects between :class:`str` and
371:class:`bytes`, all return values from URL parsing functions provide
372either an :meth:`encode` method (when the result contains :class:`str`
373data) or a :meth:`decode` method (when the result contains :class:`bytes`
374data). The signatures of these methods match those of the corresponding
375:class:`str` and :class:`bytes` methods (except that the default encoding
376is ``'ascii'`` rather than ``'utf-8'``). Each produces a value of a
377corresponding type that contains either :class:`bytes` data (for
378:meth:`encode` methods) or :class:`str` data (for
379:meth:`decode` methods).
380
381Applications that need to operate on potentially improperly quoted URLs
382that may contain non-ASCII data will need to do their own decoding from
383bytes to characters before invoking the URL parsing methods.
384
385The behaviour described in this section applies only to the URL parsing
386functions. The URL quoting functions use their own rules when producing
387or consuming byte sequences as detailed in the documentation of the
388individual URL quoting functions.
389
390.. versionchanged:: 3.2
391 URL parsing functions now accept ASCII encoded byte sequences
392
393
394.. _urlparse-result-object:
395
396Structured Parse Results
397------------------------
398
399The result objects from the :func:`urlparse`, :func:`urlsplit` and
Georg Brandl46402372010-12-04 19:06:18 +0000400:func:`urldefrag` functions are subclasses of the :class:`tuple` type.
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000401These subclasses add the attributes listed in the documentation for
402those functions, the encoding and decoding support described in the
403previous section, as well as an additional method:
404
405.. method:: urllib.parse.SplitResult.geturl()
406
407 Return the re-combined version of the original URL as a string. This may
408 differ from the original URL in that the scheme may be normalized to lower
409 case and empty components may be dropped. Specifically, empty parameters,
410 queries, and fragment identifiers will be removed.
411
412 For :func:`urldefrag` results, only empty fragment identifiers will be removed.
413 For :func:`urlsplit` and :func:`urlparse` results, all noted changes will be
414 made to the URL returned by this method.
415
416 The result of this method remains unchanged if passed back through the original
417 parsing function:
418
419 >>> from urllib.parse import urlsplit
420 >>> url = 'HTTP://www.Python.org/doc/#'
421 >>> r1 = urlsplit(url)
422 >>> r1.geturl()
423 'http://www.Python.org/doc/'
424 >>> r2 = urlsplit(r1.geturl())
425 >>> r2.geturl()
426 'http://www.Python.org/doc/'
427
428
429The following classes provide the implementations of the structured parse
430results when operating on :class:`str` objects:
431
432.. class:: DefragResult(url, fragment)
433
434 Concrete class for :func:`urldefrag` results containing :class:`str`
435 data. The :meth:`encode` method returns a :class:`DefragResultBytes`
436 instance.
437
438 .. versionadded:: 3.2
439
440.. class:: ParseResult(scheme, netloc, path, params, query, fragment)
441
442 Concrete class for :func:`urlparse` results containing :class:`str`
443 data. The :meth:`encode` method returns a :class:`ParseResultBytes`
444 instance.
445
446.. class:: SplitResult(scheme, netloc, path, query, fragment)
447
448 Concrete class for :func:`urlsplit` results containing :class:`str`
449 data. The :meth:`encode` method returns a :class:`SplitResultBytes`
450 instance.
451
452
453The following classes provide the implementations of the parse results when
454operating on :class:`bytes` or :class:`bytearray` objects:
455
456.. class:: DefragResultBytes(url, fragment)
457
458 Concrete class for :func:`urldefrag` results containing :class:`bytes`
459 data. The :meth:`decode` method returns a :class:`DefragResult`
460 instance.
461
462 .. versionadded:: 3.2
463
464.. class:: ParseResultBytes(scheme, netloc, path, params, query, fragment)
465
466 Concrete class for :func:`urlparse` results containing :class:`bytes`
467 data. The :meth:`decode` method returns a :class:`ParseResult`
468 instance.
469
470 .. versionadded:: 3.2
471
472.. class:: SplitResultBytes(scheme, netloc, path, query, fragment)
473
474 Concrete class for :func:`urlsplit` results containing :class:`bytes`
475 data. The :meth:`decode` method returns a :class:`SplitResult`
476 instance.
477
478 .. versionadded:: 3.2
479
480
481URL Quoting
482-----------
483
484The URL quoting functions focus on taking program data and making it safe
485for use as URL components by quoting special characters and appropriately
486encoding non-ASCII text. They also support reversing these operations to
487recreate the original data from the contents of a URL component if that
488task isn't already covered by the URL parsing functions above.
Georg Brandl7f01a132009-09-16 15:58:14 +0000489
490.. function:: quote(string, safe='/', encoding=None, errors=None)
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000491
492 Replace special characters in *string* using the ``%xx`` escape. Letters,
Ratnadeep Debnath21024f02017-02-25 14:30:28 +0530493 digits, and the characters ``'_.-~'`` are never quoted. By default, this
Senthil Kumaran8aa8bbe2009-08-31 16:43:45 +0000494 function is intended for quoting the path section of URL. The optional *safe*
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000495 parameter specifies additional ASCII characters that should not be quoted
496 --- its default value is ``'/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000497
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000498 *string* may be either a :class:`str` or a :class:`bytes`.
499
Ratnadeep Debnath21024f02017-02-25 14:30:28 +0530500 .. versionchanged:: 3.7
Miss Islington (bot)0f1a1832018-05-30 22:33:23 -0700501 Moved from :rfc:`2396` to :rfc:`3986` for quoting URL strings. "~" is now
Ratnadeep Debnath21024f02017-02-25 14:30:28 +0530502 included in the set of reserved characters.
503
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000504 The optional *encoding* and *errors* parameters specify how to deal with
505 non-ASCII characters, as accepted by the :meth:`str.encode` method.
506 *encoding* defaults to ``'utf-8'``.
507 *errors* defaults to ``'strict'``, meaning unsupported characters raise a
508 :class:`UnicodeEncodeError`.
509 *encoding* and *errors* must not be supplied if *string* is a
510 :class:`bytes`, or a :class:`TypeError` is raised.
511
512 Note that ``quote(string, safe, encoding, errors)`` is equivalent to
513 ``quote_from_bytes(string.encode(encoding, errors), safe)``.
514
515 Example: ``quote('/El Niño/')`` yields ``'/El%20Ni%C3%B1o/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000516
517
Georg Brandl7f01a132009-09-16 15:58:14 +0000518.. function:: quote_plus(string, safe='', encoding=None, errors=None)
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000519
Georg Brandl0f7ede42008-06-23 11:23:31 +0000520 Like :func:`quote`, but also replace spaces by plus signs, as required for
Georg Brandl81c09db2009-07-29 07:27:08 +0000521 quoting HTML form values when building up a query string to go into a URL.
522 Plus signs in the original string are escaped unless they are included in
523 *safe*. It also does not have *safe* default to ``'/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000524
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000525 Example: ``quote_plus('/El Niño/')`` yields ``'%2FEl+Ni%C3%B1o%2F'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000526
Georg Brandl7f01a132009-09-16 15:58:14 +0000527
528.. function:: quote_from_bytes(bytes, safe='/')
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000529
530 Like :func:`quote`, but accepts a :class:`bytes` object rather than a
531 :class:`str`, and does not perform string-to-bytes encoding.
532
533 Example: ``quote_from_bytes(b'a&\xef')`` yields
534 ``'a%26%EF'``.
535
Georg Brandl7f01a132009-09-16 15:58:14 +0000536
537.. function:: unquote(string, encoding='utf-8', errors='replace')
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000538
539 Replace ``%xx`` escapes by their single-character equivalent.
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000540 The optional *encoding* and *errors* parameters specify how to decode
541 percent-encoded sequences into Unicode characters, as accepted by the
542 :meth:`bytes.decode` method.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000543
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000544 *string* must be a :class:`str`.
545
546 *encoding* defaults to ``'utf-8'``.
547 *errors* defaults to ``'replace'``, meaning invalid sequences are replaced
548 by a placeholder character.
549
550 Example: ``unquote('/El%20Ni%C3%B1o/')`` yields ``'/El Niño/'``.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000551
552
Georg Brandl7f01a132009-09-16 15:58:14 +0000553.. function:: unquote_plus(string, encoding='utf-8', errors='replace')
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000554
Georg Brandl0f7ede42008-06-23 11:23:31 +0000555 Like :func:`unquote`, but also replace plus signs by spaces, as required for
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000556 unquoting HTML form values.
557
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000558 *string* must be a :class:`str`.
559
560 Example: ``unquote_plus('/El+Ni%C3%B1o/')`` yields ``'/El Niño/'``.
561
Georg Brandl7f01a132009-09-16 15:58:14 +0000562
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000563.. function:: unquote_to_bytes(string)
564
565 Replace ``%xx`` escapes by their single-octet equivalent, and return a
566 :class:`bytes` object.
567
568 *string* may be either a :class:`str` or a :class:`bytes`.
569
570 If it is a :class:`str`, unescaped non-ASCII characters in *string*
571 are encoded into UTF-8 bytes.
572
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000573 Example: ``unquote_to_bytes('a%26%EF')`` yields ``b'a&\xef'``.
Guido van Rossum52dbbb92008-08-18 21:44:30 +0000574
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000575
R David Murrayc17686f2015-05-17 20:44:50 -0400576.. function:: urlencode(query, doseq=False, safe='', encoding=None, \
577 errors=None, quote_via=quote_plus)
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000578
Senthil Kumarandf022da2010-07-03 17:48:22 +0000579 Convert a mapping object or a sequence of two-element tuples, which may
Martin Pantercda85a02015-11-24 22:33:18 +0000580 contain :class:`str` or :class:`bytes` objects, to a percent-encoded ASCII
581 text string. If the resultant string is to be used as a *data* for POST
582 operation with the :func:`~urllib.request.urlopen` function, then
583 it should be encoded to bytes, otherwise it would result in a
584 :exc:`TypeError`.
Senthil Kumaran6b3434a2012-03-15 18:11:16 -0700585
Senthil Kumarandf022da2010-07-03 17:48:22 +0000586 The resulting string is a series of ``key=value`` pairs separated by ``'&'``
R David Murrayc17686f2015-05-17 20:44:50 -0400587 characters, where both *key* and *value* are quoted using the *quote_via*
588 function. By default, :func:`quote_plus` is used to quote the values, which
589 means spaces are quoted as a ``'+'`` character and '/' characters are
590 encoded as ``%2F``, which follows the standard for GET requests
591 (``application/x-www-form-urlencoded``). An alternate function that can be
592 passed as *quote_via* is :func:`quote`, which will encode spaces as ``%20``
593 and not encode '/' characters. For maximum control of what is quoted, use
594 ``quote`` and specify a value for *safe*.
595
596 When a sequence of two-element tuples is used as the *query*
Senthil Kumarandf022da2010-07-03 17:48:22 +0000597 argument, the first element of each tuple is a key and the second is a
598 value. The value element in itself can be a sequence and in that case, if
Serhiy Storchakaa97cd2e2016-10-19 16:43:42 +0300599 the optional parameter *doseq* is evaluates to ``True``, individual
Senthil Kumarandf022da2010-07-03 17:48:22 +0000600 ``key=value`` pairs separated by ``'&'`` are generated for each element of
601 the value sequence for the key. The order of parameters in the encoded
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000602 string will match the order of parameter tuples in the sequence.
Senthil Kumarandf022da2010-07-03 17:48:22 +0000603
R David Murray8c4e1122014-12-24 21:23:18 -0500604 The *safe*, *encoding*, and *errors* parameters are passed down to
R David Murrayc17686f2015-05-17 20:44:50 -0400605 *quote_via* (the *encoding* and *errors* parameters are only passed
R David Murray8c4e1122014-12-24 21:23:18 -0500606 when a query element is a :class:`str`).
Nick Coghlan9fc443c2010-11-30 15:48:08 +0000607
608 To reverse this encoding process, :func:`parse_qs` and :func:`parse_qsl` are
609 provided in this module to parse query strings into Python data structures.
Senthil Kumarandf022da2010-07-03 17:48:22 +0000610
Senthil Kumaran29333122011-02-11 11:25:47 +0000611 Refer to :ref:`urllib examples <urllib-examples>` to find out how urlencode
612 method can be used for generating query string for a URL or data for POST.
613
Senthil Kumarandf022da2010-07-03 17:48:22 +0000614 .. versionchanged:: 3.2
Georg Brandl67b21b72010-08-17 15:07:14 +0000615 Query parameter supports bytes and string objects.
Senthil Kumaranaca8fd72008-06-23 04:41:59 +0000616
R David Murrayc17686f2015-05-17 20:44:50 -0400617 .. versionadded:: 3.5
618 *quote_via* parameter.
619
Georg Brandl116aa622007-08-15 14:28:22 +0000620
621.. seealso::
622
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000623 :rfc:`3986` - Uniform Resource Identifiers
Senthil Kumaranfe9230a2011-06-19 13:52:49 -0700624 This is the current standard (STD66). Any changes to urllib.parse module
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000625 should conform to this. Certain deviations could be observed, which are
Georg Brandl6faee4e2010-09-21 14:48:28 +0000626 mostly for backward compatibility purposes and for certain de-facto
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000627 parsing requirements as commonly observed in major browsers.
628
629 :rfc:`2732` - Format for Literal IPv6 Addresses in URL's.
630 This specifies the parsing requirements of IPv6 URLs.
631
632 :rfc:`2396` - Uniform Resource Identifiers (URI): Generic Syntax
633 Document describing the generic syntactic requirements for both Uniform Resource
634 Names (URNs) and Uniform Resource Locators (URLs).
635
636 :rfc:`2368` - The mailto URL scheme.
Martin Panterfe289c02016-05-28 02:20:39 +0000637 Parsing requirements for mailto URL schemes.
Georg Brandl116aa622007-08-15 14:28:22 +0000638
639 :rfc:`1808` - Relative Uniform Resource Locators
640 This Request For Comments includes the rules for joining an absolute and a
641 relative URL, including a fair number of "Abnormal Examples" which govern the
642 treatment of border cases.
643
Senthil Kumaran6257bdd2010-04-22 05:53:18 +0000644 :rfc:`1738` - Uniform Resource Locators (URL)
645 This specifies the formal syntax and semantics of absolute URLs.