blob: 5a54b3f4e091e21835bfe93017cd297a2a8d485c [file] [log] [blame]
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001****************************
Raymond Hettingerf558ddd2009-06-28 21:37:08 +00002 What's New In Python 3.2
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00003****************************
4
5:Author: Raymond Hettinger
6:Release: |release|
7:Date: |today|
8
9.. $Id$
10 Rules for maintenance:
11
12 * Anyone can add text to this document. Do not spend very much time
13 on the wording of your changes, because your text will probably
Raymond Hettinger6f04adc2010-12-04 22:56:25 +000014 get rewritten.
Raymond Hettinger6e6565b2009-06-28 20:56:11 +000015
16 * The maintainer will go through Misc/NEWS periodically and add
17 changes; it's therefore more important to add your changes to
18 Misc/NEWS than to this file.
19
20 * This is not a complete list of every single change; completeness
21 is the purpose of Misc/NEWS. Some changes I consider too small
22 or esoteric to include. If such a change is added to the text,
23 I'll just remove it. (This is another reason you shouldn't spend
24 too much time on writing your addition.)
25
26 * If you want to draw your new text to the attention of the
27 maintainer, add 'XXX' to the beginning of the paragraph or
28 section.
29
30 * It's OK to just add a fragmentary note about a change. For
31 example: "XXX Describe the transmogrify() function added to the
32 socket module." The maintainer will research the change and
33 write the necessary text.
34
35 * You can comment out your additions if you like, but it's not
36 necessary (especially when a final release is some months away).
37
38 * Credit the author of a patch or bugfix. Just the name is
Georg Brandlda0a2112010-09-05 11:28:33 +000039 sufficient; the e-mail address isn't necessary. It's helpful to
40 add the issue number:
Raymond Hettinger6e6565b2009-06-28 20:56:11 +000041
Éric Araujo4234ad42010-09-05 17:32:25 +000042 XXX Describe the transmogrify() function added to the socket
43 module.
44
45 (Contributed by P.Y. Developer; :issue:`12345`.)
Raymond Hettinger6e6565b2009-06-28 20:56:11 +000046
47 This saves the maintainer the effort of going through the SVN log
48 when researching a change.
49
Raymond Hettingerffad35e2010-12-14 21:12:03 +000050This article explains the new features in Python 3.2 as compared to 3.1. It
51focuses on a few highlights and gives a few examples. For full details, see the
52:source:`Misc/NEWS <Misc/NEWS>` file.
Raymond Hettinger2c1ecc32010-12-07 09:55:02 +000053
Raymond Hettinger6e6565b2009-06-28 20:56:11 +000054
Martin v. Löwis932e49e2010-12-04 13:49:32 +000055PEP 384: Defining a Stable ABI
Martin v. Löwis4d0d4712010-12-03 20:14:31 +000056==============================
57
58In the past, extension modules built for one Python version were often
59not usable with other Python versions. Particularly on Windows, every
60feature release of Python required rebuilding all extension modules that
61one wanted to use. This requirement was the result of the free access to
62Python interpreter internals that extension modules could use.
63
64With Python 3.2, an alternative approach becomes available: extension
Raymond Hettinger6f04adc2010-12-04 22:56:25 +000065modules which restrict themselves to a limited API (by defining
Martin v. Löwis4d0d4712010-12-03 20:14:31 +000066Py_LIMITED_API) cannot use many of the internals, but are constrained
67to a set of API functions that are promised to be stable for several
68releases. As a consequence, extension modules built for 3.2 in that
69mode will also work with 3.3, 3.4, and so on. Extension modules that
70make use of details of memory structures can still be built, but will
71need to be recompiled for every feature release.
72
Raymond Hettinger6f04adc2010-12-04 22:56:25 +000073.. seealso::
74
Georg Brandl65b2eb92010-12-05 11:42:38 +000075 :pep:`384` - Defining a Stable ABI
Raymond Hettinger2c1ecc32010-12-07 09:55:02 +000076 PEP written by Martin von Löwis.
Raymond Hettinger6f04adc2010-12-04 22:56:25 +000077
Raymond Hettingera5a35542010-12-05 00:39:18 +000078PEP 389: Argparse Command Line Parsing Module
79=============================================
80
81A new module for command line parsing, :mod:`argparse`, was introduced to
82overcome the limitations of :mod:`optparse` which did not provide support for
Raymond Hettinger677e10a2010-12-07 06:45:30 +000083positional arguments (not just options), subcommands, required options and other
Raymond Hettinger413abbc2010-12-05 07:06:47 +000084common patterns of specifying and validating options.
Raymond Hettingera5a35542010-12-05 00:39:18 +000085
86This module has already has wide-spread success in the community as a
Raymond Hettingerb1ff4022010-12-08 11:19:45 +000087third-party module. Being more fully featured than its predecessor, the
88:mod:`argparse` module is now the preferred module for command-line processing.
89The older module is still being kept available because of the substantial amount
90of legacy code that depends on it.
Raymond Hettingera5a35542010-12-05 00:39:18 +000091
Raymond Hettinger677e10a2010-12-07 06:45:30 +000092Here's an annotated example parser showing features like limiting results to a
93set of choices, specifying a *metavar* in the help screen, validating that one
Raymond Hettinger68f1e8d2010-12-07 09:24:30 +000094or more positional arguments is present, and making a required option::
Raymond Hettinger677e10a2010-12-07 06:45:30 +000095
96 import argparse
97 parser = argparse.ArgumentParser(
98 description = 'Manage servers', # main description for help
99 epilog = 'Tested on Solaris and Linux') # displayed after help
100 parser.add_argument('action', # argument name
101 choices = ['deploy', 'start', 'stop'], # one of four allowed values
102 help = 'action on each target') # help msg
103 parser.add_argument('targets',
104 metavar = 'HOSTNAME', # var name used in help msg
105 nargs = '+', # require 1 or more targets
106 help = 'url for target machines') # help msg explanation
107 parser.add_argument('-u', '--user', # -u or --user option
108 required = True, # make this a required argument
109 help = 'login as user')
110
111Example of calling the parser on a command string::
112
113 >>> cmd = 'deploy sneezy.example.com sleepy.example.com -u skycaptain'
114 >>> result = parser.parse_args(cmd.split())
115
Raymond Hettingerb1ff4022010-12-08 11:19:45 +0000116 >>> # parsed variables are stored in the attributes
Raymond Hettinger677e10a2010-12-07 06:45:30 +0000117 >>> result.action
118 'deploy'
119 >>> result.targets
120 ['sneezy.example.com', 'sleepy.example.com']
121 >>> result.user
122 'skycaptain'
123
124Example of the parser's automatically generated help::
125
126 >>> parser.parse_args('-h'.split())
127
Raymond Hettinger3fcf0022010-12-08 01:13:53 +0000128 usage: manage_cloud.py [-h] -u USER
129 {deploy,start,stop} HOSTNAME [HOSTNAME ...]
Raymond Hettinger677e10a2010-12-07 06:45:30 +0000130
131 Manage servers
132
133 positional arguments:
134 {deploy,start,stop} action on each target
135 HOSTNAME url for target machines
136
137 optional arguments:
138 -h, --help show this help message and exit
139 -u USER, --user USER login as user
140
141 Tested on Solaris and Linux
142
Raymond Hettingerb1ff4022010-12-08 11:19:45 +0000143An especially nice :mod:`argparse` feature is the ability to define subparsers,
144each with their own argument patterns and help displays::
145
146 import argparse
147 parser = argparse.ArgumentParser(prog='HELM')
148 subparsers = parser.add_subparsers()
149
150 parser_l = subparsers.add_parser('launch', help='Launch Control') # first subgroup
151 parser_l.add_argument('-m', '--missles', action='store_true')
152 parser_l.add_argument('-t', '--torpedos', action='store_true')
153
154 parser_m = subparsers.add_parser('move', help='Move Vessel') # second subgroup
155 parser_m.add_argument('-c', '--course', type=int, required=True)
156 parser_m.add_argument('-s', '--speed', type=int, default=0)
157
158 $ ./helm.py --help # top level help (launch and move)
159 $ ./helm.py launch --help # help for launch options
160 $ ./helm.py launch --missiles # set missiles=True and torpedos=False
161 $ ./helm.py move --course 180 --speed 5 # set movement parameters
Raymond Hettingera5a35542010-12-05 00:39:18 +0000162
163.. seealso::
164
165 :pep:`389` - New Command Line Parsing Module
166 PEP written by Steven Bethard.
167
Raymond Hettinger677e10a2010-12-07 06:45:30 +0000168 :ref:`upgrading-optparse-code` for details on the differences from
169 :mod:`optparse`.
170
Raymond Hettinger6e6565b2009-06-28 20:56:11 +0000171
Éric Araujo4234ad42010-09-05 17:32:25 +0000172PEP 391: Dictionary Based Configuration for Logging
173====================================================
Raymond Hettingeref2335c2010-09-05 08:35:38 +0000174
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000175The :mod:`logging` module provided two kinds of configuration, one style with
176function calls for each option or another style driven by an external file saved
177in a :mod:`ConfigParser` format. Those options did not provide the flexibility
Georg Brandl9e75cad2010-09-06 06:45:47 +0000178to create configurations from JSON or YAML files, nor did they support
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000179incremental configuration, which is needed for specifying logger options from a
180command line.
Raymond Hettingeref2335c2010-09-05 08:35:38 +0000181
182To support a more flexible style, the module now offers
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000183:func:`logging.config.dictConfig` for specifying logging configuration with
184plain Python dictionaries. The configuration options include formatters,
185handlers, filters, and loggers. Here's a working example of a configuration
186dictionary::
Raymond Hettingeref2335c2010-09-05 08:35:38 +0000187
Georg Brandlda0a2112010-09-05 11:28:33 +0000188 {"version": 1,
189 "formatters": {"brief": {"format": "%(levelname)-8s: %(name)-15s: %(message)s"},
190 "full": {"format": "%(asctime)s %(name)-15s %(levelname)-8s %(message)s"},
191 },
192 "handlers": {"console": {
193 "class": "logging.StreamHandler",
194 "formatter": "brief",
195 "level": "INFO",
196 "stream": "ext://sys.stdout"},
197 "console_priority": {
198 "class": "logging.StreamHandler",
199 "formatter": "full",
200 "level": "ERROR",
201 "stream": "ext://sys.stderr"},
202 },
203 "root": {"level": "DEBUG", "handlers": ["console", "console_priority"]}}
Raymond Hettingeref2335c2010-09-05 08:35:38 +0000204
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000205
Raymond Hettinger6f04adc2010-12-04 22:56:25 +0000206If that dictionary is stored in a file called :file:`conf.json`, it can loaded
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000207and called with code like this::
208
209 >>> import logging.config
210 >>> logging.config.dictConfig(json.load(open('conf.json', 'rb')))
211 >>> logging.info("Transaction completed normally")
212 >>> logging.critical("Abnormal termination")
213
Raymond Hettingeref2335c2010-09-05 08:35:38 +0000214.. seealso::
215
216 :pep:`391` - Dictionary Based Configuration for Logging
217 PEP written by Vinay Sajip.
218
Georg Brandl97b20da2010-11-16 15:15:29 +0000219PEP 3148: The ``concurrent.futures`` module
220============================================
221
Raymond Hettinger6f04adc2010-12-04 22:56:25 +0000222Code for creating and managing concurrency is being collected in a new toplevel
223namespace, *concurrent*. Its first member is a *futures* package which provides
224a uniform high level interface for managing threads and processes.
225
226The design for :mod:`concurrent.futures` was inspired by
227*java.util.concurrent.package*. In that model, a running call and its result
228are represented by a :class:`~concurrent.futures.Future` object which abstracts
229features common to threads, processes, and remote procedure calls. That object
230supports status checks (running or done), timeouts, cancellations, adding
Raymond Hettinger24a09412010-12-08 06:50:02 +0000231callbacks, and access to results or exceptions.
Raymond Hettinger6f04adc2010-12-04 22:56:25 +0000232
233The primary offering of the new module is a pair of executor classes for
234launching and managing calls. The goal of the executors is to make it easier to
235use existing tools for making parallel calls. They save the effort needed to
236setup a pool of resources, launch the calls, create a results queue, add
237time-out handling, and limit the total number of threads, processes, or remote
Raymond Hettingerc269ae82010-12-05 01:01:52 +0000238procedure calls.
Raymond Hettinger6f04adc2010-12-04 22:56:25 +0000239
240Ideally, each application should share a single executor across multiple
241components so that process and thread limits can be centrally managed. This
242solves the design challenge that arises when each component has its own
243competing strategy for resource management.
244
Raymond Hettingerb1055192010-12-08 06:42:41 +0000245Both classes share a common interface with three methods:
246:meth:`~concurrent.futures.Executor.submit` for scheduling a callable and
247returning a :class:`~concurrent.futures.Future` object;
248:meth:`~concurrent.futures.Executor.map` for scheduling many asynchronous calls
Raymond Hettinger83d80792010-12-08 06:48:33 +0000249at a time, and :meth:`~concurrent.futures.Executor.shutdown` for freeing
250resources. The class is a :term:`context manager` and can be used within a
251:keyword:`with` statement to assure that resources are automatically released
252when currently pending futures are done executing.
Raymond Hettinger6f04adc2010-12-04 22:56:25 +0000253
Raymond Hettingerb1055192010-12-08 06:42:41 +0000254A simple of example of :class:`~concurrent.futures.ThreadPoolExecutor` is a
Raymond Hettinger83d80792010-12-08 06:48:33 +0000255launch of four parallel threads for copying files::
Raymond Hettingerb1055192010-12-08 06:42:41 +0000256
257 import shutil
258 with ThreadPoolExecutor(max_workers=4) as e:
259 e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
260 e.submit(shutil.copy, 'src2.txt', 'dest2.txt')
261 e.submit(shutil.copy, 'src3.txt', 'dest3.txt')
262 e.submit(shutil.copy, 'src3.txt', 'dest4.txt')
263
Raymond Hettinger6f04adc2010-12-04 22:56:25 +0000264.. seealso::
265
Raymond Hettingera5a35542010-12-05 00:39:18 +0000266 :pep:`3148` - Futures -- Execute Computations Asynchronously
Andrew M. Kuchling42877fe2010-12-15 02:37:01 +0000267 PEP written by Brian Quinlan.
Georg Brandl97b20da2010-11-16 15:15:29 +0000268
Raymond Hettinger83d80792010-12-08 06:48:33 +0000269 :ref:`Code for Threaded Parallel URL reads<threadpoolexecutor-example>`, an
270 example using threads to fetch multiple web pages in parallel.
271
272 :ref:`Code for computing prime numbers in
273 parallel<processpoolexecutor-example>`, an example demonstrating
274 :class:`~concurrent.futures.ProcessPoolExecutor`.
275
276
Georg Brandlda0a2112010-09-05 11:28:33 +0000277
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000278PEP 3147: PYC Repository Directories
279=====================================
280
David Malcolm778645a2010-12-07 00:32:04 +0000281Python's scheme for caching bytecode in *.pyc* files did not work well in
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000282environments with multiple python interpreters. If one interpreter encountered
283a cached file created by another interpreter, it would recompile the source and
284overwrite the cached file, thus losing the benefits of caching.
285
286The issue of "pyc fights" has become more pronounced as it has become
Éric Araujo4234ad42010-09-05 17:32:25 +0000287commonplace for Linux distributions to ship with multiple versions of Python.
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000288These conflicts also arise with CPython alternatives such as Unladen Swallow.
289
290To solve this problem, Python's import machinery has been extended to use
Éric Araujo4234ad42010-09-05 17:32:25 +0000291distinct filenames for each interpreter. Instead of Python 3.2 and Python 3.3 and
292Unladen Swallow each competing for a file called "mymodule.pyc", they will now
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000293look for "mymodule.cpython-32.pyc", "mymodule.cpython-33.pyc", and
Éric Araujo4234ad42010-09-05 17:32:25 +0000294"mymodule.unladen10.pyc". And to prevent all of these new files from
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000295cluttering source directories, the *pyc* files are now collected in a
296"__pycache__" directory stored under the package directory.
297
298Aside from the filenames and target directories, the new scheme has a few
299aspects that are visible to the programmer:
300
Georg Brandlda0a2112010-09-05 11:28:33 +0000301* Imported modules now have a :attr:`__cached__` attribute which stores the name
302 of the actual file that was imported:
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000303
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000304 >>> import collections
305 >>> collections.__cached__
306 'c:/py32/lib/__pycache__/collections.cpython-32.pyc'
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000307
308* The tag that is unique to each interpreter is accessible from the :mod:`imp`
Georg Brandlda0a2112010-09-05 11:28:33 +0000309 module:
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000310
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000311 >>> import imp
312 >>> imp.get_tag()
313 'cpython-32'
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000314
315* Scripts that try to deduce source filename from the imported file now need to
316 be smarter. It is no longer sufficient to simply strip the "c" from a ".pyc"
317 filename. Instead, use the new functions in the :mod:`imp` module:
318
Georg Brandlda0a2112010-09-05 11:28:33 +0000319 >>> imp.source_from_cache('c:/py32/lib/__pycache__/collections.cpython-32.pyc')
320 'c:/py32/lib/collections.py'
321 >>> imp.cache_from_source('c:/py32/lib/collections.py')
322 'c:/py32/lib/__pycache__/collections.cpython-32.pyc'
Raymond Hettingerf95b1992010-09-04 23:53:24 +0000323
324* The :mod:`py_compile` and :mod:`compileall` modules have been updated to
325 reflect the new naming convention and target directory.
326
327.. seealso::
328
329 :pep:`3147` - PYC Repository Directories
330 PEP written by Barry Warsaw.
331
Georg Brandlda0a2112010-09-05 11:28:33 +0000332
Georg Brandl3ad46752010-12-05 07:59:29 +0000333PEP 3149: ABI Version Tagged .so Files
334======================================
Georg Brandlf11c6c42010-09-03 22:20:58 +0000335
Raymond Hettingerebea6fa2010-09-05 00:27:25 +0000336The PYC repository directory allows multiple bytecode cache files to be
337co-located. This PEP implements a similar mechanism for shared object files by
338giving them a common directory and distinct names for each version.
Georg Brandlf11c6c42010-09-03 22:20:58 +0000339
Raymond Hettingerebea6fa2010-09-05 00:27:25 +0000340The common directory is "pyshared" and the file names are made distinct by
341identifying the Python implementation (such as CPython, PyPy, Jython, etc.), the
342major and minor version numbers, and optional build flags (such as "d" for
Éric Araujo4234ad42010-09-05 17:32:25 +0000343debug, "m" for pymalloc, "u" for wide-unicode). For an arbitrary package "foo",
Raymond Hettingerebea6fa2010-09-05 00:27:25 +0000344you may see these files when the distribution package is installed::
345
346 /usr/share/pyshared/foo.cpython-32m.so
347 /usr/share/pyshared/foo.cpython-33md.so
348
349In Python itself, the tags are accessible from functions in the :mod:`sysconfig`
350module::
351
352 >>> import sysconfig
353 >>> sysconfig.get_config_var('SOABI') # find the version tag
354 'cpython-32mu'
355 >>> sysconfig.get_config_var('SO') # find the full filename extension
356 'cpython-32mu.so'
357
358.. seealso::
359
360 :pep:`3149` - ABI Version Tagged .so Files
361 PEP written by Barry Warsaw.
Raymond Hettinger6e6565b2009-06-28 20:56:11 +0000362
363
364Other Language Changes
365======================
366
367Some smaller changes made to the core Python language are:
368
Raymond Hettingere5e1a982010-12-05 08:35:21 +0000369* String formatting for :func:`format` and :meth:`str.format` gained new
370 capabilities for the format character **#**. Previously, for integers in
371 binary, octal, or hexadecimal, it caused the output to be prefixed with '0b',
372 '0o', or '0x' respectively. Now it can also handle floats, complex, and
373 Decimal, causing the output to always have a decimal point even when no digits
374 follow it.
Raymond Hettingere5e728b2010-12-05 06:35:16 +0000375
376 >>> format(20, '#o')
377 '0o24'
378 >>> format(12.34, '#5.0f')
379 ' 12.'
380
381 (Suggested by Mark Dickinson and implemented by Eric Smith in :issue:`7094`.)
Raymond Hettinger43b5a852010-12-05 04:04:21 +0000382
Raymond Hettingerc269ae82010-12-05 01:01:52 +0000383* The interpreter can now be started with a quiet option, ``-q``, to suppress
384 the copyright and version information in an interactive mode.
385
386 (Contributed by Marcin Wojdyr in issue:`1772833`).
387
Georg Brandlda0a2112010-09-05 11:28:33 +0000388* The :func:`hasattr` function used to catch and suppress any Exception. Now,
389 it only catches :exc:`AttributeError`. Under the hood, :func:`hasattr` works
390 by calling :func:`getattr` and throwing away the results. This is necessary
391 because dynamic attribute creation is possible using :meth:`__getattribute__`
Éric Araujo4234ad42010-09-05 17:32:25 +0000392 or :meth:`__getattr__`. If :func:`hasattr` were to just scan instance and class
Éric Araujocc6aac62010-09-07 21:35:35 +0000393 dictionaries it would miss the dynamic methods and make it difficult to
Georg Brandlda0a2112010-09-05 11:28:33 +0000394 implement proxy objects.
Raymond Hettinger1784ff02010-09-05 01:00:19 +0000395
Raymond Hettingera55ffbc2010-12-15 18:31:57 +0000396 (Discovered by Yury Selivanov and fixed by Benjamin Peterson; :issue:`9666`.)
Raymond Hettinger1784ff02010-09-05 01:00:19 +0000397
Éric Araujo4234ad42010-09-05 17:32:25 +0000398* The :func:`str` of a float or complex number is now the same as its
Raymond Hettinger1784ff02010-09-05 01:00:19 +0000399 :func:`repr`. Previously, the :func:`str` form was shorter but that just
Éric Araujo4234ad42010-09-05 17:32:25 +0000400 caused confusion and is no longer needed now that the shortest possible
Georg Brandlda0a2112010-09-05 11:28:33 +0000401 :func:`repr` is displayed by default:
Raymond Hettingerbb734c62010-09-05 05:56:44 +0000402
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000403 >>> repr(math.pi)
404 '3.141592653589793'
405 >>> str(math.pi)
406 '3.141592653589793'
Raymond Hettinger1784ff02010-09-05 01:00:19 +0000407
Georg Brandlda0a2112010-09-05 11:28:33 +0000408 (Proposed and implemented by Mark Dickinson; :issue:`9337`.)
Raymond Hettinger6e6565b2009-06-28 20:56:11 +0000409
Raymond Hettinger21ec4bc2010-12-10 01:09:01 +0000410* :class:`memoryview` objects now have a :meth:`~memoryview.release()` method
411 and they also now support the context manager protocol. This allows timely
412 release of any resources that were acquired when requesting a buffer from the
413 original object.
Antoine Pitroud3052002010-09-15 15:09:40 +0000414
Raymond Hettingerd8fae4e2010-12-05 05:39:54 +0000415 >>> with memoryview(b'abcdefgh') as v:
416 ... print(v.tolist())
417 ...
418 [97, 98, 99, 100, 101, 102, 103, 104]
419
Antoine Pitroud3052002010-09-15 15:09:40 +0000420 (Added by Antoine Pitrou; :issue:`9757`.)
421
Raymond Hettinger92ba2862010-09-06 01:16:46 +0000422* Mark Dickinson crafted an elegant and efficient scheme for assuring that
423 different numeric datatypes will have the same hash value whenever their
424 actual values are equal::
425
426 >>> assert hash(Fraction(3, 2)) == hash(1.5) == \
427 hash(Decimal("1.5")) == hash(complex(1.5, 0))
428
429 (See :issue:`8188`.)
Raymond Hettinger6e6565b2009-06-28 20:56:11 +0000430
Amaury Forgeot d'Arcba117ef2010-09-10 21:39:53 +0000431* Previously it was illegal to delete a name from the local namespace if it
432 occurs as a free variable in a nested block::
433
434 >>> def outer(x):
435 ... def inner():
436 ... return x
437 ... inner()
438 ... del x
439
440 This is now allowed. Remember that the target of an :keyword:`except` clause
441 is cleared, so this code which used to work with Python 2.6, raised a
442 :exc:`SyntaxError` with Python 3.1 and now works again::
443
444 >>> def f():
445 ... def print_error():
446 ... print(e)
447 ... try:
448 ... something
449 ... except Exception as e:
450 ... print_error()
451 ... # implicit "del e" here
452
453 (See :issue:`4617`.)
454
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000455* A new warning category, :exc:`ResourceWarning`, has been added. It is
Raymond Hettingerc269ae82010-12-05 01:01:52 +0000456 emitted when potential issues with resource consumption or cleanup
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000457 are detected. It is silenced by default in normal release builds, but
Raymond Hettingerc269ae82010-12-05 01:01:52 +0000458 can be enabled through the means provided by the :mod:`warnings`
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000459 module, or on the command line.
460
Raymond Hettingerd8fae4e2010-12-05 05:39:54 +0000461 A :exc:`ResourceWarning` is issued at interpreter shutdown if the
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000462 :data:`gc.garbage` list isn't empty. This is meant to make the programmer
463 aware that their code contains object finalization issues.
464
Raymond Hettingerd8fae4e2010-12-05 05:39:54 +0000465 A :exc:`ResourceWarning` is also issued when a :term:`file object` is destroyed
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000466 without having been explicitly closed. While the deallocator for such
467 object ensures it closes the underlying operating system resource
468 (usually, a file descriptor), the delay in deallocating the object could
469 produce various issues, especially under Windows. Here is an example
470 of enabling the warning from the command line::
471
Raymond Hettinger673ccf22010-12-07 09:37:11 +0000472 $ ./python -q -Wdefault
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000473 >>> f = open("foo", "wb")
474 >>> del f
475 __main__:1: ResourceWarning: unclosed file <_io.BufferedWriter name='foo'>
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000476
Raymond Hettingerd8fae4e2010-12-05 05:39:54 +0000477 (Added by Antoine Pitrou and Georg Brandl in :issue:`10093` and :issue:`477863`.)
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000478
Raymond Hettingera0266332010-12-07 08:52:41 +0000479* :class:`range` objects now support *index* and *count* methods. This is part
480 of an effort to make more objects fully implement the
481 :class:`collections.Sequence` :term:`abstract base class`. As a result, the
482 language will have a more uniform API. In addition, :class:`range` objects
483 now support slicing and negative indices. This makes *range* more
Raymond Hettinger2ffa6712010-12-08 10:18:21 +0000484 interoperable with lists::
485
486 >>> range(0, 100, 2).count(10)
487 1
488 >>> range(0, 100, 2).index(10)
489 5
490 >>> range(0, 100, 2)[5]
491 10
492 >>> range(0, 100, 2)[0:5]
493 range(0, 10, 2)
Raymond Hettingerdadf93c2010-12-05 02:56:21 +0000494
495 (Contributed by Daniel Stuzback in :issue:`9213` and by Alexander Belopolsky
496 in :issue:`2690`.)
Nick Coghlan37ee8502010-12-03 14:26:13 +0000497
Raymond Hettingerd8fae4e2010-12-05 05:39:54 +0000498* The :func:`callable` builtin function from Py2.x was resurrected. It provides
Raymond Hettingerb87ba262010-12-06 04:31:40 +0000499 a concise, readable alternative to using an :term:`abstract base class` in an
Raymond Hettinger792c0762010-12-09 16:41:54 +0000500 expression like ``isinstance(x, collections.Callable)``:
501
502 >>> callable(max)
503 True
504 >>> callable(20)
505 False
Raymond Hettingerd8fae4e2010-12-05 05:39:54 +0000506
507 (See :issue:`10518`.)
Amaury Forgeot d'Arcba117ef2010-09-10 21:39:53 +0000508
Raymond Hettinger070ec702010-12-10 17:45:13 +0000509* Python's import mechanism can now load module installed in directories with
510 non-ASCII characters in the path name.
511
512 (Required extensive work by Victor Stinner in :issue:`9425`.)
513
514
Raymond Hettinger6e6565b2009-06-28 20:56:11 +0000515New, Improved, and Deprecated Modules
516=====================================
517
Raymond Hettinger99db3fd2010-12-15 19:33:49 +0000518Python's standard library has undergone significant maintenance efforts and
519quality improvements.
Raymond Hettingere434b3b2010-12-15 19:20:01 +0000520
521The biggest news for Python 3.2 is that the :mod:`email` package and
Raymond Hettinger99db3fd2010-12-15 19:33:49 +0000522:mod:`nntplib` modules now work correctly with the bytes/text model in Python 3.
Raymond Hettingere434b3b2010-12-15 19:20:01 +0000523For the first time, there is correct handling of inputs with mixed encodings.
524
525Another significant win is the addition of substantially better support for
526*SSL* connections and security certificates.
527
Raymond Hettinger99db3fd2010-12-15 19:33:49 +0000528In addition, more functions and classes now have a :term:`context manager` to
529support convenient and reliable resource clean-up using the
Raymond Hettingere434b3b2010-12-15 19:20:01 +0000530:keyword:`with`-statement.
531
Raymond Hettinger0358a172010-12-15 19:00:38 +0000532email
533-----
534
535The usability of the :mod:`email` package in Python 3 has been mostly fixed by
536the extensive efforts of R. David Murray. The problem was that emails are
537typically read and stored in the form of :class:`bytes` rather than :class:`str`
538text, and they may contain multiple encodings within a single email. So, the
539email package had to be extended to parse and generate email messages in bytes
540format.
541
542* New functions :func:`~email.message_from_bytes` and
543 :func:`~email.message_from_binary_file`, and new classes
544 :class:`~email.parser.BytesFeedParser` and :class:`~email.parser.BytesParser`
545 allow binary message data to be parsed into model objects.
546
547* Given bytes input to the model, :meth:`~email.message.Message.get_payload`
548 will by default decode a message body that has a
549 :mailheader:`Content-Transfer-Encoding` of *8bit* using the charset
550 specified in the MIME headers and return the resulting string.
551
552* Given bytes input to the model, :class:`~email.generator.Generator` will
553 convert message bodies that have a :mailheader:`Content-Transfer-Encoding` of
554 *8bit* to instead have a *7bit* :mailheader:`Content-Transfer-Encoding`.
555
556* A new class :class:`~email.generator.BytesGenerator` produces bytes as output,
557 preserving any unchanged non-ASCII data that was present in the input used to
558 build the model, including message bodies with a
559 :mailheader:`Content-Transfer-Encoding` of *8bit*.
560
561* The :mod:`smtplib` :class:`~smtplib.SMTP` class now accepts a byte string
562 for the *msg* argument to the :meth:`~smtplib.SMTP.sendmail` method,
563 and a new method, :meth:`~smtplib.SMTP.send_message` accepts a
564 :class:`~email.message.Message` object and can optionally obtain the
565 *from_addr* and *to_addrs* addresses directly from the object.
566
567.. XXX Update before 3.2rc1 to reflect all of the latest work and add examples.
568
569(Proposed and implemented by R. David Murray, :issue:`4661` and :issue:`10321`.)
570
571
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000572functools
573---------
574
Éric Araujo4234ad42010-09-05 17:32:25 +0000575* The :mod:`functools` module includes a new decorator for caching function
Georg Brandlda0a2112010-09-05 11:28:33 +0000576 calls. :func:`functools.lru_cache` can save repeated queries to an external
577 resource whenever the results are expected to be the same.
Raymond Hettingeraed05eb2010-08-02 01:43:41 +0000578
Raymond Hettinger86f96132010-08-06 23:23:49 +0000579 For example, adding a caching decorator to a database query function can save
580 database accesses for popular searches::
Raymond Hettingeraed05eb2010-08-02 01:43:41 +0000581
Georg Brandlda0a2112010-09-05 11:28:33 +0000582 @functools.lru_cache(maxsize=300)
583 def get_phone_number(name):
584 c = conn.cursor()
585 c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,))
586 return c.fetchone()[0]
Raymond Hettingeraed05eb2010-08-02 01:43:41 +0000587
Georg Brandlda0a2112010-09-05 11:28:33 +0000588 >>> for name in user_requests:
Raymond Hettinger7496b412010-11-30 19:15:45 +0000589 ... get_phone_number(name) # cached lookup
590
591 To help with choosing an effective cache size, the wrapped function is
592 instrumented for tracking cache statistics:
593
Raymond Hettinger5e20bab2010-11-30 07:13:04 +0000594 >>> get_phone_number.cache_info()
Raymond Hettinger7496b412010-11-30 19:15:45 +0000595 CacheInfo(hits=4805, misses=980, maxsize=300, currsize=300)
Raymond Hettingeraed05eb2010-08-02 01:43:41 +0000596
Raymond Hettingerf3098282010-08-15 03:30:45 +0000597 If the phonelist table gets updated, the outdated contents of the cache can be
Georg Brandlda0a2112010-09-05 11:28:33 +0000598 cleared with:
Raymond Hettingerf3098282010-08-15 03:30:45 +0000599
Georg Brandlda0a2112010-09-05 11:28:33 +0000600 >>> get_phone_number.cache_clear()
Raymond Hettingerf3098282010-08-15 03:30:45 +0000601
Raymond Hettinger6e353942010-12-04 23:42:12 +0000602 (Contributed by Raymond Hettinger and incorporating design ideas from
Raymond Hettingerb87ba262010-12-06 04:31:40 +0000603 Jim Baker, Miki Tebeka, and Nick Coghlan.)
Raymond Hettingeraed05eb2010-08-02 01:43:41 +0000604
Antoine Pitrou7d49bc92010-09-15 15:13:17 +0000605* The :func:`functools.wraps` decorator now adds a :attr:`__wrapped__` attribute
606 pointing to the original callable function. This allows wrapped functions to
607 be introspected. It also copies :attr:`__annotations__` if defined. And now
608 it also gracefully skips over missing attributes such as :attr:`__doc__` which
Raymond Hettinger5eb63902010-12-09 23:43:34 +0000609 might not be defined for the wrapped callable.
Antoine Pitrou7d49bc92010-09-15 15:13:17 +0000610
611 (By Nick Coghlan and Terrence Cole; :issue:`9567`, :issue:`3445`, and
612 :issue:`8814`.)
613
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000614itertools
615---------
616
Raymond Hettinger673ccf22010-12-07 09:37:11 +0000617* The :mod:`itertools` module has a new :func:`~itertools.accumulate` function
Raymond Hettingera5a35542010-12-05 00:39:18 +0000618 modeled on APL's *scan* operator and on Numpy's *accumulate* function:
Raymond Hettinger6e353942010-12-04 23:42:12 +0000619
620 >>> list(accumulate(8, 2, 50))
621 [8, 10, 60]
622
623 >>> prob_dist = [0.1, 0.4, 0.2, 0.3]
624 >>> list(accumulate(prob_dist)) # cumulative probability distribution
625 [0.1, 0.5, 0.7, 1.0]
626
627 For an example using :func:`~itertools.accumulate`, see the :ref:`examples for
628 the random module <random-examples>`.
629
630 (Contributed by Raymond Hettinger and incorporating design suggestions
631 from Mark Dickinson.)
632
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000633collections
634-----------
635
Raymond Hettinger792c0762010-12-09 16:41:54 +0000636* The :class:`collections.Counter` class now has two forms of in-place
637 subtraction, the existing *-=* operator for `saturating subtraction
638 <http://en.wikipedia.org/wiki/Saturation_arithmetic>`_ and the new
639 :meth:`~collections.Counter.subtract` method for regular subtraction. The
640 former is suitable for `multisets <http://en.wikipedia.org/wiki/Multiset>`_
Raymond Hettingerffad35e2010-12-14 21:12:03 +0000641 which only have positive counts, and the latter is more suitable for use cases
Raymond Hettinger792c0762010-12-09 16:41:54 +0000642 that allow negative counts:
Alexander Belopolsky72572312010-12-08 21:21:56 +0000643
Raymond Hettinger792c0762010-12-09 16:41:54 +0000644 >>> tally = Counter(dogs=5, cat=3)
645 >>> tally -= Counter(dogs=2, cats=8) # saturating subtraction
646 >>> tally
647 Counter({'dogs': 3})
Alexander Belopolsky72572312010-12-08 21:21:56 +0000648
Raymond Hettinger792c0762010-12-09 16:41:54 +0000649 >>> tally = Counter(dogs=5, cats=3)
650 >>> tally.subtract(dogs=2, cats=8) # regular subtraction
651 >>> tally
652 Counter({'dogs': 3, 'cats': -5})
Alexander Belopolsky72572312010-12-08 21:21:56 +0000653
Raymond Hettinger792c0762010-12-09 16:41:54 +0000654 (Contributed by Raymond Hettinger.)
Alexander Belopolsky72572312010-12-08 21:21:56 +0000655
Raymond Hettingere0a96002010-12-15 17:54:13 +0000656* The :class:`collections.OrderedDict` class has a new method
657 :meth:`~collections.OrderedDict.move_to_end` which takes an existing key and
658 moves it to either the beginning or end of an ordered sequence. When the
659 dictionary sequence is being used as a queue, these operations correspond to
660 "move to the front of the line" or "move to the back of the line":
661
662 >>> d = OrderedDict.fromkeys(['a', 'b', 'X', 'd', 'e'])
663 >>> list(d)
664 ['a', 'b', 'X', 'd', 'e']
665 >>> d.move_to_end('X', last=True)
666 >>> list(d)
667 ['a', 'b', 'd', 'e', 'X']
668 >>> d.move_to_end('X', last=False)
669 >>> list(d)
670 ['X', 'a', 'b', 'd', 'e']
671
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000672datetime
673--------
674
Raymond Hettinger792c0762010-12-09 16:41:54 +0000675* The :mod:`datetime` module has a new type :class:`~datetime.timezone` that
676 implements the :class:`~datetime.tzinfo` interface by returning a fixed UTC
677 offset and timezone name. This makes it easier to create timezone aware
678 datetime objects:
Alexander Belopolsky72572312010-12-08 21:21:56 +0000679
Raymond Hettinger792c0762010-12-09 16:41:54 +0000680 >>> datetime.now(timezone.utc)
681 datetime.datetime(2010, 12, 8, 21, 4, 2, 923754, tzinfo=datetime.timezone.utc)
Alexander Belopolsky72572312010-12-08 21:21:56 +0000682
Raymond Hettinger792c0762010-12-09 16:41:54 +0000683 >>> datetime.strptime("01/01/2000 12:00 +0000", "%m/%d/%Y %H:%M %z")
684 datetime.datetime(2000, 1, 1, 12, 0, tzinfo=datetime.timezone.utc)
Alexander Belopolsky72572312010-12-08 21:21:56 +0000685
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000686* Also, :class:`~datetime.timedelta` objects can now be multiplied by
Raymond Hettinger792c0762010-12-09 16:41:54 +0000687 :class:`float` and divided by :class:`float` and :class:`int` objects.
688
689 (Contributed by Alexander Belopolsky in :issue:`1289118`, :issue:`5094` and
690 :issue:`6641`.)
Alexander Belopolsky72572312010-12-08 21:21:56 +0000691
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000692abc
693---
Antoine Pitrou7d49bc92010-09-15 15:13:17 +0000694
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000695The :mod:`abc` module now supports :func:`~abc.abstractclassmethod` and
696:func:`~abc.abstractstaticmethod`.
Raymond Hettingera5a35542010-12-05 00:39:18 +0000697
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000698These tools make it possible to define an :term:`Abstract Base Class` that
699requires a particular :func:`classmethod` or :func:`staticmethod` to be
700implemented.
Antoine Pitrou7d49bc92010-09-15 15:13:17 +0000701
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000702(Patch submitted by Daniel Urban; :issue:`5867`.)
Raymond Hettingerbcbd6962010-09-05 08:46:36 +0000703
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000704ftp
705---
Raymond Hettingerbcbd6962010-09-05 08:46:36 +0000706
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000707The :class:`ftplib.FTP` class now supports the context manager protocol to
708unconditionally consume :exc:`socket.error` exceptions and to close the FTP
709connection when done::
Giampaolo Rodolàbd576b72010-05-10 14:53:29 +0000710
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000711 >>> from ftplib import FTP
712 >>> with FTP("ftp1.at.proftpd.org") as ftp:
713 ... ftp.login()
714 ... ftp.dir()
715 ...
716 '230 Anonymous login ok, restrictions apply.'
717 dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 .
718 dr-xr-xr-x 9 ftp ftp 154 May 6 10:43 ..
719 dr-xr-xr-x 5 ftp ftp 4096 May 6 10:43 CentOS
720 dr-xr-xr-x 3 ftp ftp 18 Jul 10 2008 Fedora
Georg Brandlda0a2112010-09-05 11:28:33 +0000721
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000722Other file-like objects such as :class:`mmap.mmap` and :func:`fileinput.input`
723also grew auto-closing context managers::
724
725 with fileinput.input(files=('log1.txt', 'log2.txt')) as f:
726 for line in f:
727 process(line)
728
729(Contributed by Tarek Ziadé and Giampaolo Rodolà in :issue:`4972`, and
730by Georg Brandl in :issue:`8046` and :issue:`1286`.)
Antoine Pitrou696e0352010-08-08 22:18:46 +0000731
Raymond Hettinger0358a172010-12-15 19:00:38 +0000732.. XXX mention os.popen and subprocess.Popen auto-closing of fds
Georg Brandl3ad46752010-12-05 07:59:29 +0000733
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000734gzip
735----
Antoine Pitroucd889af2010-10-06 21:13:56 +0000736
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000737:class:`gzip.GzipFile` now implements the :class:`io.BufferedIOBase`
738:term:`abstract base class` (except for ``truncate()``). It also has a
739:meth:`~gzip.GzipFile.peek` method and supports unseekable as well as
740zero-padded file objects.
Raymond Hettingera0266332010-12-07 08:52:41 +0000741
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000742The :mod:`gzip` module also gains the :func:`~gzip.compress` and
743:func:`~gzip.decompress` functions for easier in-memory compression and
744decompression. Keep in mind that text needs to be encoded in to :class:`bytes`
745before compressing and decompressing:
Raymond Hettingera0266332010-12-07 08:52:41 +0000746
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000747>>> s = 'Three shall be the number thou shalt count, '
748>>> s += 'and the number of the counting shall be three'
749>>> b = s.encode() # convert to utf-8
750>>> len(b)
75189
752>>> c = gzip.compress(b)
753>>> len(c)
75477
755>>> gzip.decompress(c).decode()[:42] # decompress and convert to text
756'Three shall be the number thou shalt count,'
Antoine Pitroucd889af2010-10-06 21:13:56 +0000757
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000758(Contributed by Anand B. Pillai in :issue:`3488`; and by Antoine Pitrou, Nir
759Aides and Brian Curtin in :issue:`9962`, :issue:`1675951`, :issue:`7471` and
760:issue:`2846`.)
761
762shutil
763------
764
765The :func:`shutil.copytree` function has two new options:
Antoine Pitroud67075e2010-07-31 22:48:02 +0000766
Raymond Hettingerdb9044e2010-09-06 01:29:23 +0000767 * *ignore_dangling_symlinks*: when ``symlinks=False`` so that the function
768 copies the file pointed to by the symlink, not the symlink itself. This
Georg Brandlda0a2112010-09-05 11:28:33 +0000769 option will silence the error raised if the file doesn't exist.
Antoine Pitroud67075e2010-07-31 22:48:02 +0000770
Raymond Hettingerdb9044e2010-09-06 01:29:23 +0000771 * *copy_function*: is a callable that will be used to copy files.
Antoine Pitroud67075e2010-07-31 22:48:02 +0000772 :func:`shutil.copy2` is used by default.
773
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000774(Contributed by Tarek Ziadé.)
Antoine Pitroud67075e2010-07-31 22:48:02 +0000775
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000776sqlite3
777-------
Antoine Pitroue43f9d02010-08-08 23:24:50 +0000778
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000779The :mod:`sqlite3` module has two new capabilities.
Antoine Pitroue43f9d02010-08-08 23:24:50 +0000780
Raymond Hettinger0358a172010-12-15 19:00:38 +0000781* The :attr:`sqlite3.Connection.in_transit` attribute is true if there is an
782 active transaction for uncommitted changes.
Antoine Pitroud67075e2010-07-31 22:48:02 +0000783
Raymond Hettinger0358a172010-12-15 19:00:38 +0000784* The :meth:`sqlite3.Connection.enable_load_extension` and
785 :meth:`sqlite3.Connection.load_extension` methods allows you to load SQLite
786 extensions from ".so" files. One well-known extension is the fulltext-search
787 extension distributed with SQLite.
Antoine Pitroud67075e2010-07-31 22:48:02 +0000788
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000789(Contributed by R. David Murray and Shashwat Anand; :issue:`8845`.)
790
791socket
792------
793
794The :mod:`socket` module has two new improvements.
795
796* Socket objects now have a :meth:`~socket.socket.detach()` method which puts
797 the socket into closed state without actually closing the underlying file
798 descriptor. The latter can then be reused for other purposes.
799 (Added by Antoine Pitrou; :issue:`8524`.)
800
801* :func:`socket.create_connection` now supports the context manager protocol
802 to unconditionally consume :exc:`socket.error` exceptions and to close the
803 socket when done.
804 (Contributed by Giampaolo Rodolà; :issue:`9794`.)
805
806ssl
807---
Antoine Pitroud67075e2010-07-31 22:48:02 +0000808
Georg Brandlda0a2112010-09-05 11:28:33 +0000809* The :mod:`ssl` module has a new class, :class:`~ssl.SSLContext` which serves
810 as a container for various persistent SSL data, such as protocol settings,
811 certificates, private keys, and various other options. The
812 :meth:`~ssl.SSLContext.wrap_socket` method allows to create an SSL socket from
813 such an SSL context. (Added by Antoine Pitrou; :issue:`8550`.)
Antoine Pitrou4f2a0a82010-07-31 18:08:33 +0000814
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000815* A new function, :func:`ssl.match_hostname`, helps implement server identity
Antoine Pitrou0ee4c9f2010-10-08 16:46:17 +0000816 verification for higher-level protocols by implementing the rules of
817 HTTPS (from :rfc:`2818`), which are also suitable for other protocols.
818 (Added by Antoine Pitrou, :issue:`1589`).
819
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000820* The :func:`ssl.wrap_socket` constructor function now takes a *ciphers*
Georg Brandlda0a2112010-09-05 11:28:33 +0000821 argument that's a string listing the encryption algorithms to be allowed; the
822 format of the string is described `in the OpenSSL documentation
823 <http://www.openssl.org/docs/apps/ciphers.html#CIPHER_LIST_FORMAT>`__. (Added
824 by Antoine Pitrou; :issue:`8322`.)
Antoine Pitrou4f2a0a82010-07-31 18:08:33 +0000825
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000826* When linked against a recent enough version of OpenSSL, the :mod:`ssl`
Antoine Pitrou7d15a722010-11-05 22:13:55 +0000827 module now supports the Server Name Indication extension to the TLS
828 protocol, allowing for several "virtual hosts" using different certificates
829 on a single IP/port. This extension is only supported in client mode,
830 and is activated by passing the *server_hostname* argument to
831 :meth:`SSLContext.wrap_socket`.
832 (Added by Antoine Pitrou, :issue:`5639`.)
833
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000834* Various options have been added to the :mod:`ssl` module, such as
Georg Brandlda0a2112010-09-05 11:28:33 +0000835 :data:`~ssl.OP_NO_SSLv2` which allows to force disabling of the insecure and
836 obsolete SSLv2 protocol. (Added by Antoine Pitrou; :issue:`4870`.)
Antoine Pitrou4f2a0a82010-07-31 18:08:33 +0000837
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000838* Another change makes the extension load all of OpenSSL's ciphers and digest
Georg Brandlda0a2112010-09-05 11:28:33 +0000839 algorithms so that they're all available. Some SSL certificates couldn't be
840 verified, reporting an "unknown algorithm" error. (Reported by Beda Kosata,
841 and fixed by Antoine Pitrou; :issue:`8484`.)
Antoine Pitrou4f2a0a82010-07-31 18:08:33 +0000842
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000843* The version of OpenSSL being used is now available as the module attributes
Georg Brandlda0a2112010-09-05 11:28:33 +0000844 :data:`ssl.OPENSSL_VERSION` (a string), :data:`ssl.OPENSSL_VERSION_INFO` (a
845 5-tuple), and :data:`ssl.OPENSSL_VERSION_NUMBER` (an integer). (Added by
846 Antoine Pitrou; :issue:`8321`.)
Antoine Pitrou4f2a0a82010-07-31 18:08:33 +0000847
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000848nntp
849----
Raymond Hettinger070ec702010-12-10 17:45:13 +0000850
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000851The :mod:`nntplib` module has a revamped implementation with better bytes and
852unicode semantics as well as more practical APIs. These improvements break
853compatibility with the nntplib version in Python 3.1, which was partly
854dysfunctional in itself.
Raymond Hettinger070ec702010-12-10 17:45:13 +0000855
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000856(Contributed by Antoine Pitrou in :issue:`9360`)
857
858certificates
859------------
860
861:class:`http.client.HTTPSConnection`, :class:`urllib.request.HTTPSHandler`
862and :func:`urllib.request.urlopen` now take optional arguments to allow for
863server certificate checking against a set of Certificate Authorities,
864as recommended in public uses of HTTPS.
865
866(Added by Antoine Pitrou, :issue:`9003`.)
867
868unittest
869--------
Antoine Pitrouafb078d2010-11-05 22:18:28 +0000870
Raymond Hettingera0266332010-12-07 08:52:41 +0000871* The command-line call, ``python -m unittest`` can now accept file paths
872 instead of module names for running specific tests (:issue:`10620`). The new
873 test discovery can find tests within packages, locating any test importable
874 from the top level directory. The top level directory can be specified with
875 the `-t` option, a pattern for matching files with ``-p``, and a directory to
876 start discovery with ``-s``::
877
878 $ python -m unittest discover -s my_proj_dir -p '_test.py'
879
880 (Contributed by Michael Foord.)
Antoine Pitroud3052002010-09-15 15:09:40 +0000881
Raymond Hettingerdc2f9b52010-12-05 07:02:45 +0000882* The :mod:`unittest` module has two new methods,
883 :meth:`~unittest.TestCase.assertWarns` and
884 :meth:`~unittest.TestCase.assertWarnsRegex` to check that a given warning type
Raymond Hettinger413abbc2010-12-05 07:06:47 +0000885 is triggered by the code under test:
Antoine Pitroud3052002010-09-15 15:09:40 +0000886
Raymond Hettingerdc2f9b52010-12-05 07:02:45 +0000887 >>> with self.assertWarns(DeprecationWarning):
888 ... legacy_function('XYZ')
Ezio Melotti2baf1a62010-11-22 12:56:58 +0000889
Raymond Hettinger21ec4bc2010-12-10 01:09:01 +0000890 Another new method, :meth:`~unittest.TestCase.assertCountEqual` is used to
Raymond Hettingerffad35e2010-12-14 21:12:03 +0000891 compare two iterables to determine if their element counts are equal (whether
892 the same elements are present with the same number of occurrences regardless
893 of order)::
Raymond Hettingera0266332010-12-07 08:52:41 +0000894
895 def test_anagram(self):
896 self.assertCountEqual('algorithm', 'logarithm')
897
898 A principal feature of the unittest module is an effort to produce meaningful
899 diagnostics when a test fails. When possible the failure is recorded along
900 with a diff of the output. This is especially helpful for analyzing log files
901 of failed test runs. However, since diffs can sometime be voluminous, there is
902 a new :attr:`~unittest.TestCase.maxDiff` attribute which sets maximum length of
903 diffs.
904
Raymond Hettinger68f1e8d2010-12-07 09:24:30 +0000905 In addition the naming in the module has undergone a number of clean-ups. For
Raymond Hettingera0266332010-12-07 08:52:41 +0000906 example, :meth:`~unittest.TestCase.assertRegex` is the new name for
907 :meth:`~unittest.TestCase.assertRegexpMatches` which was misnamed because the
Raymond Hettinger21ec4bc2010-12-10 01:09:01 +0000908 test uses :func:`re.search`, not :func:`re.match`. Other methods using
909 regular expressions are now named using short form "Regex" in preference
910 to "Regexp" -- this matches the names used in other unittest implementations,
911 matches Python's old name for the :mod:`re` module, and it has unambiguous
912 camel-casing.
Raymond Hettingerdc2f9b52010-12-05 07:02:45 +0000913
914 To improve consistency, some of long-standing method aliases are being
915 deprecated in favor of the preferred names:
916
917 - replace :meth:`assert_` with :meth:`.assertTrue`
918 - replace :meth:`assertEquals` with :meth:`.assertEqual`
919 - replace :meth:`assertNotEquals` with :meth:`.assertNotEqual`
920 - replace :meth:`assertAlmostEquals` with :meth:`.assertAlmostEqual`
921 - replace :meth:`assertNotAlmostEquals` with :meth:`.assertNotAlmostEqual`
922
923 Likewise, the ``TestCase.fail*`` methods deprecated in Python 3.1 are expected
924 to be removed in Python 3.3. See also the :ref:`deprecated-aliases` section in
925 the :mod:`unittest` documentation.
Ezio Melotti2baf1a62010-11-22 12:56:58 +0000926
927 (Contributed by Ezio Melotti; :issue:`9424`.)
Antoine Pitroud3052002010-09-15 15:09:40 +0000928
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000929random
930------
Raymond Hettingere0a96002010-12-15 17:54:13 +0000931
Raymond Hettinger0358a172010-12-15 19:00:38 +0000932The integer methods in the :mod:`random` module now do a better job of producing
Raymond Hettinger99db3fd2010-12-15 19:33:49 +0000933uniform distributions. Previously, they computed selections with
934``int(n*random())`` which had a slight bias whenever *n* was not a power of two.
935Now, multiple selections are made from a range upto the next power of two and a
936selection is kept only when it falls within the range ``0 <= x < n``. The
937functions and methods affected are :func:`~random.randrange`,
938:func:`~random.randint`, :func:`~random.choice`, :func:`~random.shuffle` and
939:func:`~random.sample`.
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000940
941(Contributed by Raymond Hettinger; :issue:`9025`.)
942
943poplib
944------
Raymond Hettingere0a96002010-12-15 17:54:13 +0000945
Giampaolo Rodolà42382fe2010-08-17 16:09:53 +0000946* :class:`~poplib.POP3_SSL` class now accepts a *context* parameter, which is a
947 :class:`ssl.SSLContext` object allowing bundling SSL configuration options,
948 certificates and private keys into a single (potentially long-lived)
949 structure.
950
951 (Contributed by Giampaolo Rodolà; :issue:`8807`.)
952
Giampaolo Rodolà977c7072010-10-04 21:08:36 +0000953* :class:`asyncore.dispatcher` now provides a
954 :meth:`~asyncore.dispatcher.handle_accepted()` method
955 returning a `(sock, addr)` pair which is called when a connection has actually
956 been established with a new remote endpoint. This is supposed to be used as a
957 replacement for old :meth:`~asyncore.dispatcher.handle_accept()` and avoids
958 the user to call :meth:`~asyncore.dispatcher.accept()` directly.
959
960 (Contributed by Giampaolo Rodolà; :issue:`6706`.)
Georg Brandlda0a2112010-09-05 11:28:33 +0000961
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000962tempfile
963--------
Raymond Hettingera0266332010-12-07 08:52:41 +0000964
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000965The :mod:`tempfile` module has a new context manager,
966:class:`~tempfile.TemporaryDirectory` which provides easy deterministic
967cleanup of temporary directories:
Nick Coghlan543af752010-10-24 11:23:25 +0000968
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000969>>> with tempfile.TemporaryDirectory() as tmpdirname:
970... print 'created temporary directory', tmpdirname
Nick Coghlan543af752010-10-24 11:23:25 +0000971
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000972(Contributed by Neil Schemenauer and Nick Coghlan; :issue:`5178`.)
Nick Coghlane0f04652010-11-21 03:44:04 +0000973
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000974inspect
975-------
976
Raymond Hettinger0358a172010-12-15 19:00:38 +0000977* The :mod:`inspect` module has a new function
978 :func:`~inspect.getgeneratorstate` to easily identify the current state of a
979 generator as one of ``GEN_CREATED``, ``GEN_RUNNING``, ``GEN_SUSPENDED`` or
980 ``GEN_CLOSED``. (Contributed by Rodolpho Eckhardt and Nick Coghlan,
981 :issue:`10220`.)
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000982
Raymond Hettingera55ffbc2010-12-15 18:31:57 +0000983* To support lookups without the possibility of activating a dynamic attribute,
984 the :mod:`inspect` module has a new function, :func:`~inspect.getattr_static`.
985 Unlike, :func:`hasattr`, this is a true read-only search, guaranteed not to
986 change state while it is searching. (Contributed by Michael Foord.)
Nick Coghlane0f04652010-11-21 03:44:04 +0000987
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000988pydoc
989-----
Nick Coghlan7bb30b72010-12-03 09:29:11 +0000990
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000991The :mod:`pydoc` module now provides a much improved Web server interface,
992as well as a new command-line option to automatically open a browser
993window to display that server.
Nick Coghlan7bb30b72010-12-03 09:29:11 +0000994
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000995(Contributed by Ron Adam; :issue:`2001`.)
Raymond Hettinger3f9734c2010-12-07 01:47:52 +0000996
Raymond Hettingerbba537b2010-12-15 18:20:19 +0000997sysconfig
998---------
Raymond Hettinger3f9734c2010-12-07 01:47:52 +0000999
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001000The new :mod:`sysconfig` module makes it straight-forward to discover
1001installation paths and configuration variables which vary across platforms and
1002installations.
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001003
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001004The module offers access simple access functions for platform and version
1005information:
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001006
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001007* :func:`~sysconfig.get_platform` returning values like *linux-i586* or
1008 *macosx-10.6-ppc*.
1009* :func:`~sysconfig.get_python_version` returns a Python version string in
1010 the form, "3.2".
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001011
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001012It also provides access to the paths and variables corresponding to one of
1013seven named schemes used by :mod:`distutils`. Those include *posix_prefix*,
1014*posix_home*, *posix_user*, *nt*, *nt_user*, *os2*, *os2_home*:
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001015
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001016* :func:`~sysconfig.get_paths` makes a dictionary containing installation paths
1017 for the current installation scheme.
1018* :func:`~sysconfig.get_config_vars` returns a dictionary of platform specific
1019 variables.
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001020
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001021There is also a convenient command-line interface::
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001022
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001023 C:\Python32>python -m sysconfig
1024 Platform: "win32"
1025 Python version: "3.2"
1026 Current installation scheme: "nt"
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001027
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001028 Paths:
1029 data = "C:\Python32"
1030 include = "C:\Python32\Include"
1031 platinclude = "C:\Python32\Include"
1032 platlib = "C:\Python32\Lib\site-packages"
1033 platstdlib = "C:\Python32\Lib"
1034 purelib = "C:\Python32\Lib\site-packages"
1035 scripts = "C:\Python32\Scripts"
1036 stdlib = "C:\Python32\Lib"
1037
1038 Variables:
1039 BINDIR = "C:\Python32"
1040 BINLIBDEST = "C:\Python32\Lib"
1041 EXE = ".exe"
1042 INCLUDEPY = "C:\Python32\Include"
1043 LIBDEST = "C:\Python32\Lib"
1044 SO = ".pyd"
1045 VERSION = "32"
1046 abiflags = ""
1047 base = "C:\Python32"
1048 exec_prefix = "C:\Python32"
1049 platbase = "C:\Python32"
1050 prefix = "C:\Python32"
1051 projectbase = "C:\Python32"
1052 py_version = "3.2"
1053 py_version_nodot = "32"
1054 py_version_short = "3.2"
1055 srcdir = "C:\Python32"
1056 userbase = "C:\Documents and Settings\Raymond\Application Data\Python"
1057
1058pdb
1059---
1060
1061The :mod:`pdb` debugger module gained a number of usability improvements:
Raymond Hettingerb5d79332010-12-07 02:04:56 +00001062
Raymond Hettinger99db3fd2010-12-15 19:33:49 +00001063* :file:`pdb.py` now has a ``-c`` option that executes commands as given in a
1064 :file:`.pdbrc` script file.
1065* A :file:`.pdbrc` script file can contain ``continue`` and ``next`` commands
1066 that continue debugging.
1067* The :class:`Pdb` class constructor now accepts a *nosigint* argument.
1068* new commands: ``l(list)``, ``ll(long list`` and ``source`` for
1069 listing source code.
1070* new commands: ``display`` and ``undisplay`` for showing or hiding
1071 the value of an expression if it has changed.
1072* new command: ``interact`` for starting an interactive interpreter containing
1073 the global and local names found in the current scope.
1074* breakpoints can be cleared by breakpoint number
Raymond Hettingerb5d79332010-12-07 02:04:56 +00001075
Raymond Hettinger3f9734c2010-12-07 01:47:52 +00001076
Raymond Hettingera55ffbc2010-12-15 18:31:57 +00001077.. XXX: Various ConfigParser changes
1078.. XXX: Mention urllib.parse changes
1079 Issue 9873 (Nick Coghlan):
1080 - ASCII byte sequence support in URL parsing
1081 - named tuple for urldefrag return value
1082 Issue 5468 (Dan Mahn) for urlencode:
1083 - bytes input support
1084 - non-UTF8 percent encoding of non-ASCII characters
1085 Issue 2987 for IPv6 (RFC2732) support in urlparse
1086
1087
Antoine Pitroud42bc512009-11-10 23:18:31 +00001088Multi-threading
1089===============
1090
Georg Brandlda0a2112010-09-05 11:28:33 +00001091* The mechanism for serializing execution of concurrently running Python threads
1092 (generally known as the GIL or Global Interpreter Lock) has been rewritten.
1093 Among the objectives were more predictable switching intervals and reduced
1094 overhead due to lock contention and the number of ensuing system calls. The
1095 notion of a "check interval" to allow thread switches has been abandoned and
1096 replaced by an absolute duration expressed in seconds. This parameter is
1097 tunable through :func:`sys.setswitchinterval()`. It currently defaults to 5
1098 milliseconds.
Antoine Pitroud42bc512009-11-10 23:18:31 +00001099
1100 Additional details about the implementation can be read from a `python-dev
1101 mailing-list message
1102 <http://mail.python.org/pipermail/python-dev/2009-October/093321.html>`_
Georg Brandlda0a2112010-09-05 11:28:33 +00001103 (however, "priority requests" as exposed in this message have not been kept
1104 for inclusion).
Antoine Pitroud42bc512009-11-10 23:18:31 +00001105
Georg Brandl5e73a812010-04-22 07:02:51 +00001106 (Contributed by Antoine Pitrou.)
Antoine Pitroud42bc512009-11-10 23:18:31 +00001107
Georg Brandlda0a2112010-09-05 11:28:33 +00001108* Regular and recursive locks now accept an optional *timeout* argument to their
Raymond Hettinger09e4ebb2010-09-06 19:55:51 +00001109 :meth:`acquire` method. (Contributed by Antoine Pitrou; :issue:`7316`.)
Georg Brandlda0a2112010-09-05 11:28:33 +00001110
Raymond Hettingerbba537b2010-12-15 18:20:19 +00001111* Similarly, :meth:`threading.Semaphore.acquire` also gained a *timeout*
Georg Brandlda0a2112010-09-05 11:28:33 +00001112 argument. (Contributed by Torsten Landschoff; :issue:`850728`.)
Antoine Pitroue95a9ff2010-05-04 23:31:41 +00001113
Antoine Pitroud42bc512009-11-10 23:18:31 +00001114
Raymond Hettinger92ba2862010-09-06 01:16:46 +00001115Optimizations
1116=============
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001117
Raymond Hettinger92ba2862010-09-06 01:16:46 +00001118A number of small performance enhancements have been added:
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001119
Antoine Pitroud3052002010-09-15 15:09:40 +00001120* Python's peephole optimizer now recognizes patterns such ``x in {1, 2, 3}`` as
Raymond Hettinger92ba2862010-09-06 01:16:46 +00001121 being a test for membership in a set of constants. The optimizer recasts the
1122 :class:`set` as a :class:`frozenset` and stores the pre-built constant.
1123
1124 Now that the speed penalty is gone, it is practical to start writing
1125 membership tests using set-notation. This style is both semantically clear
1126 and operationally fast::
1127
1128 extension = name.rpartition('.')[2]
1129 if extension in {'xml', 'html', 'xhtml', 'css'}:
1130 handle(name)
1131
1132 (Patch and additional tests by Dave Malcolm; :issue:`6690`).
1133
Antoine Pitroud3052002010-09-15 15:09:40 +00001134* Serializing and unserializing data using the :mod:`pickle` module is now
Raymond Hettingerdadf93c2010-12-05 02:56:21 +00001135 several times faster.
1136
1137 (Contributed by Alexandre Vassalotti, Antoine Pitrou
Antoine Pitrouff150f22010-10-22 21:41:05 +00001138 and the Unladen Swallow team in :issue:`9410` and :issue:`3873`.)
Antoine Pitroud3052002010-09-15 15:09:40 +00001139
Raymond Hettingerc269ae82010-12-05 01:01:52 +00001140* The `Timsort algorithm <http://en.wikipedia.org/wiki/Timsort>`_ used in
Raymond Hettingerffad35e2010-12-14 21:12:03 +00001141 :meth:`list.sort` and :func:`sorted` now runs faster and uses less memory
Raymond Hettingerc269ae82010-12-05 01:01:52 +00001142 when called with a :term:`key function`. Previously, every element of
1143 a list was wrapped with a temporary object that remembered the key value
1144 associated with each element. Now, an array of keys and values are
1145 sorted in parallel. This save the memory consumed by the sort wrappers,
1146 and it saves time lost from during comparisons which where delegated
1147 by the sort wrappers.
1148
1149 (Patch by Daniel Stuzback in :issue:`9915`.)
1150
Raymond Hettingerdadf93c2010-12-05 02:56:21 +00001151* JSON decoding performance is improved and memory consumption is reduced
Raymond Hettinger413abbc2010-12-05 07:06:47 +00001152 whenever the same string is repeated for multiple keys. Also, JSON encoding
Raymond Hettingerdadf93c2010-12-05 02:56:21 +00001153 now uses the C speedups when the ``sort_keys`` argument is true.
1154
1155 (Contributed by Antoine Pitrou in :issue:`7451` and by Raymond Hettinger and
1156 Antoine Pitrou in :issue:`10314`.)
1157
Raymond Hettinger21ec4bc2010-12-10 01:09:01 +00001158* Recursive locks (created with the :func:`threading.RLock` API) now benefit
1159 from a C implementation which makes them as fast as regular locks, and between
1160 10x and 15x faster than their previous pure Python implementation.
1161
1162 (Contributed by Antoine Pitrou; :issue:`3001`.)
1163
Raymond Hettingerdadf93c2010-12-05 02:56:21 +00001164* The fast-search algorithm in stringlib is now used by the :meth:`split`,
1165 :meth:`rsplit`, :meth:`splitlines` and :meth:`replace` methods on
1166 :class:`bytes`, :class:`bytearray` and :class:`str` objects. Likewise, the
1167 algorithm is also used by :meth:`rfind`, :meth:`rindex`, :meth:`rsplit` and
1168 :meth:`rpartition`.
1169
1170 (Patch by Florent Xicluna in :issue:`7622` and :issue:`7462`.)
1171
Raymond Hettingerd8fae4e2010-12-05 05:39:54 +00001172There were several other minor optimizations. Set differencing now runs faster
1173when one operand is much larger than the other (Patch by Andress Bennetts in
1174:issue:`8685`). The :meth:`array.repeat` method has a faster implementation
1175(:issue:`1569291` by Alexander Belopolsky). The :class:`BaseHTTPRequestHandler`
1176has more efficient buffering (:issue:`3709` by Andrew Schaaf). The
1177multi-argument form of :func:`operator.attrgetter` now function runs slightly
1178faster (:issue:`10160` by Christos Georgiou). And :class:`ConfigParser` loads
1179multi-line arguments a bit faster (:issue:`7113` by Łukasz Langa).
1180
Antoine Pitroud3052002010-09-15 15:09:40 +00001181
Victor Stinner47ce9652010-10-29 00:57:35 +00001182Unicode
1183=======
Victor Stinner94908bb2010-08-18 21:23:25 +00001184
Alexander Belopolsky507e3f82010-12-02 00:05:57 +00001185Python has been updated to Unicode 6.0.0. The new features of the
1186Unicode Standard that will affect Python users include:
1187
Alexander Belopolsky84cc0622010-12-08 21:38:46 +00001188* addition of 2,088 characters, including over 1,000 additional
1189 symbols—chief among them the additional emoji symbols, which are
1190 especially important for mobile phones;
Alexander Belopolsky507e3f82010-12-02 00:05:57 +00001191
Alexander Belopolsky84cc0622010-12-08 21:38:46 +00001192* changes to character properties for existing characters including
Alexander Belopolsky507e3f82010-12-02 00:05:57 +00001193
Raymond Hettingerc74d5182010-12-02 01:38:25 +00001194 - a general category change to two Kannada characters (U+0CF1,
1195 U+0CF2), which has the effect of making them newly eligible for
1196 inclusion in identifiers;
1197
1198 - a general category change to one New Tai Lue numeric character
Alexander Belopolsky84cc0622010-12-08 21:38:46 +00001199 (U+19DA), which has the effect of disqualifying it from
1200 inclusion in identifiers.
1201
1202 For more information, see `Unicode Character Database Changes
1203 <http://www.unicode.org/versions/Unicode6.0.0/#Database_Changes>`_
1204 at the `Unicode Consortium <http://www.unicode.org/>`_ web site.
Alexander Belopolsky507e3f82010-12-02 00:05:57 +00001205
Éric Araujo4234ad42010-09-05 17:32:25 +00001206The :mod:`os` module has two new functions: :func:`~os.fsencode` and
Victor Stinner47ce9652010-10-29 00:57:35 +00001207:func:`~os.fsdecode`. Add :data:`os.environb`: bytes version of
1208:data:`os.environ`, :func:`os.getenvb` function and
1209:data:`os.supports_bytes_environ` constant.
Victor Stinnere8d51452010-08-19 01:05:19 +00001210
Georg Brandl326c57d2010-11-26 12:10:06 +00001211``'mbcs'`` encoding doesn't ignore the error handler argument any more. By
Victor Stinner47ce9652010-10-29 00:57:35 +00001212default (strict mode), it raises an UnicodeDecodeError on undecodable byte
1213sequence and UnicodeEncodeError on unencodable character. To get the ``'mbcs'``
1214encoding of Python 3.1, use ``'ignore'`` error handler to decode and
1215``'replace'`` error handler to encode. ``'mbcs'`` supports ``'strict'`` and
1216``'ignore'`` error handlers for decoding, and ``'strict'`` and ``'replace'``
1217for encoding.
1218
1219On Mac OS X, Python uses ``'utf-8'`` to decode the command line arguments,
1220instead of the locale encoding (which is ISO-8859-1 if the ``LANG`` environment
1221variable is not set).
1222
1223By default, tarfile uses ``'utf-8'`` encoding on Windows (instead of
1224``'mbcs'``), and the ``'surrogateescape'`` error handler on all operating
1225systems.
Antoine Pitroud3052002010-09-15 15:09:40 +00001226
Victor Stinner94908bb2010-08-18 21:23:25 +00001227
Raymond Hettinger1fa76822010-12-06 23:31:36 +00001228Documentation
1229=============
1230
1231The documentation continues to be improved.
1232
1233A table of quick links has been added to the top of lengthy sections such as
1234:ref:`built-in-funcs`. In the case of :mod:`itertools`, the links are
1235accompanied by tables of cheatsheet-style summaries to provide an overview and
1236memory jog without having to read all of the docs.
1237
1238In some cases, the pure python source code can be helpful adjunct to the docs,
1239so now some modules feature quick links to the latest version of the source
1240code. For example, the :mod:`functools` module documentation has a quick link
1241at the top labeled :source:`functools Python source code <Lib/functools.py>`.
1242
1243The docs now contain more examples and recipes. In particular, :mod:`re` module
1244has an extensive section, :ref:`re-examples`. Likewise, the :mod:`itertools`
1245module continues to be updated with new :ref:`itertools-recipes`.
1246
Raymond Hettinger677e10a2010-12-07 06:45:30 +00001247The :mod:`datetime` module now has an auxiliary implementation in pure Python.
1248No functionality was changed. This just provides an easier-to-read
1249alternate implementation. (Contributed by Alexander Belopolsky.)
1250
Raymond Hettinger1fa76822010-12-06 23:31:36 +00001251
1252IDLE
1253====
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001254
Georg Brandlcc9d2372010-12-10 19:22:11 +00001255* The format menu now has an option to clean-up source files by stripping
1256 trailing whitespace (:issue:`5150`).
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001257
1258
1259Build and C API Changes
1260=======================
1261
1262Changes to Python's build process and to the C API include:
1263
Georg Brandlda0a2112010-09-05 11:28:33 +00001264* The C functions that access the Unicode Database now accept and return
1265 characters from the full Unicode range, even on narrow unicode builds
Raymond Hettinger1784ff02010-09-05 01:00:19 +00001266 (Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and others). A visible difference
Georg Brandlda0a2112010-09-05 11:28:33 +00001267 in Python is that :func:`unicodedata.numeric` now returns the correct value
1268 for large code points, and :func:`repr` may consider more characters as
1269 printable.
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001270
Raymond Hettinger1784ff02010-09-05 01:00:19 +00001271 (Reported by Bupjoe Lee and fixed by Amaury Forgeot D'Arc; :issue:`5127`.)
1272
Georg Brandlda0a2112010-09-05 11:28:33 +00001273* Computed gotos are now enabled by default on supported compilers (which are
Raymond Hettingerdb9044e2010-09-06 01:29:23 +00001274 detected by the configure script). They can still be disabled selectively by
Georg Brandlda0a2112010-09-05 11:28:33 +00001275 specifying ``--without-computed-gotos``.
Raymond Hettinger1784ff02010-09-05 01:00:19 +00001276
Georg Brandlda0a2112010-09-05 11:28:33 +00001277 (Contributed by Antoine Pitrou; :issue:`9203`.)
1278
Amaury Forgeot d'Arcfeb73072010-09-12 22:42:57 +00001279* The option ``--with-wctype-functions`` was removed. The built-in unicode
1280 database is now used for all functions.
1281
1282 (Contributed by Amaury Forgeot D'Arc; :issue:`9210`.)
1283
Skip Montanaro961aaf52010-10-17 22:22:24 +00001284* Hash values are now values of a new type, Py_hash_t, which is defined to
1285 be the same size as a pointer. Previously they were of type long, which
Raymond Hettinger1fa76822010-12-06 23:31:36 +00001286 on some 64-bit operating systems is still only 32 bits long. As a result
1287 of this fix, :class:`set` and :class:`dict` can now hold more than ``2**32``
1288 entries on builds with 64-bit pointers (previously, they could grow to
1289 that size but their performance degraded catastrophically).
Skip Montanaro961aaf52010-10-17 22:22:24 +00001290
1291 (Contributed by Benjamin Peterson; :issue:`9778`.)
1292
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001293
Raymond Hettingerf558ddd2009-06-28 21:37:08 +00001294Porting to Python 3.2
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001295=====================
1296
Georg Brandlda0a2112010-09-05 11:28:33 +00001297This section lists previously described changes and other bugfixes that may
1298require changes to your code:
Raymond Hettinger6e6565b2009-06-28 20:56:11 +00001299
Antoine Pitroucd889af2010-10-06 21:13:56 +00001300* The :mod:`nntplib` module was reworked extensively, meaning that its APIs
1301 are often incompatible with the 3.1 APIs.
1302
Raymond Hettinger1fa76822010-12-06 23:31:36 +00001303* :class:`bytearray` objects can no longer be used as filenames; instead,
1304 they should be converted to :class:`bytes`.
Victor Stinnerdcb24032010-04-22 12:08:36 +00001305
Victor Stinner25e8ec42010-06-25 00:02:38 +00001306* PyArg_Parse*() functions:
Victor Stinner3dcb5ac2010-06-08 22:54:19 +00001307
Victor Stinner25e8ec42010-06-25 00:02:38 +00001308 * "t#" format has been removed: use "s#" or "s*" instead
1309 * "w" and "w#" formats has been removed: use "w*" instead
1310
Georg Brandl60203b42010-10-06 10:11:56 +00001311* The :c:type:`PyCObject` type, deprecated in 3.1, has been removed. To wrap
1312 opaque C pointers in Python objects, the :c:type:`PyCapsule` API should be used
Éric Araujo4234ad42010-09-05 17:32:25 +00001313 instead; the new type has a well-defined interface for passing typing safety
Georg Brandlda0a2112010-09-05 11:28:33 +00001314 information and a less complicated signature for calling a destructor.
Victor Stinner0cbec572010-09-12 20:32:57 +00001315
Raymond Hettingere0a96002010-12-15 17:54:13 +00001316* The :func:`sys.setfilesystemencoding` function was removed because
1317 it had a flawed design.
Raymond Hettinger3fcf0022010-12-08 01:13:53 +00001318
Raymond Hettingere0a96002010-12-15 17:54:13 +00001319* The :func:`random.seed` function and method now salt string seeds with an
1320 sha512 hash function. To access the previous version of *seed* in order to
1321 reproduce Python 3.1 sequences, set the *version* argument to *1*,
1322 ``random.seed(s, version=1)``.
Raymond Hettinger21ec4bc2010-12-10 01:09:01 +00001323
Raymond Hettinger522cc0a2010-12-10 01:19:15 +00001324* The previously deprecated :func:`string.maketrans` function has been removed
1325 in favor of the static methods, :meth:`bytes.maketrans` and
1326 :meth:`bytearray.maketrans`. This change solves the confusion around which
1327 types were supported by the :mod:`string` module. Now, :class:`str`,
1328 :class:`bytes`, and :class:`bytearray` each have their own **maketrans** and
1329 **translate** methods with intermediate translation tables of the appropriate
1330 type.
1331
1332 (Contributed by Georg Brandl; :issue:`5675`.)
1333
1334* The previously deprecated :func:`contextlib.nested` function has been removed
1335 in favor of a plain :keyword:`with` statement which can accept multiple
1336 context managers. The latter technique is faster (because it is built-in),
1337 and it does a better job finalizing multiple context managers when one of them
1338 raises an exception::
1339
1340 >>> with open('mylog.txt') as infile, open('a.out', 'w') as outfile:
1341 ... for line in infile:
1342 ... if '<critical>' in line:
1343 ... outfile.write(line)
1344
1345 (Contributed by Georg Brandl and Mattias Brändström;
1346 `appspot issue 53094 <http://codereview.appspot.com/53094>`_.)