blob: 158b3be5bb3d601878bd0735319f7876cac0c934 [file] [log] [blame]
Benjamin Petersond6313712008-07-31 16:23:04 +00001.. _2to3-reference:
2
32to3 - Automated Python 2 to 3 code translation
4===============================================
5
Benjamin Peterson058e31e2009-01-16 03:54:08 +00006.. sectionauthor:: Benjamin Peterson <benjamin@python.org>
Benjamin Petersond6313712008-07-31 16:23:04 +00007
Benjamin Peterson8951b612008-09-03 02:27:16 +000082to3 is a Python program that reads Python 2.x source code and applies a series
9of *fixers* to transform it into valid Python 3.x code. The standard library
Benjamin Petersonae5360b2008-09-08 23:05:23 +000010contains a rich set of fixers that will handle almost all code. 2to3 supporting
11library :mod:`lib2to3` is, however, a flexible and generic library, so it is
12possible to write your own fixers for 2to3. :mod:`lib2to3` could also be
13adapted to custom applications in which Python code needs to be edited
14automatically.
Benjamin Petersond6313712008-07-31 16:23:04 +000015
16
Benjamin Petersonf91df042009-02-13 02:50:59 +000017.. _2to3-using:
18
Benjamin Petersond6313712008-07-31 16:23:04 +000019Using 2to3
20----------
21
Benjamin Petersonae5360b2008-09-08 23:05:23 +0000222to3 will usually be installed with the Python interpreter as a script. It is
23also located in the :file:`Tools/scripts` directory of the Python root.
24
252to3's basic arguments are a list of files or directories to transform. The
Chris Jerdonek46397492012-10-11 15:59:32 -070026directories are recursively traversed for Python sources.
Benjamin Petersond6313712008-07-31 16:23:04 +000027
28Here is a sample Python 2.x source file, :file:`example.py`::
29
30 def greet(name):
31 print "Hello, {0}!".format(name)
32 print "What's your name?"
33 name = raw_input()
34 greet(name)
35
36It can be converted to Python 3.x code via 2to3 on the command line::
37
38 $ 2to3 example.py
39
Benjamin Petersonae5360b2008-09-08 23:05:23 +000040A diff against the original source file is printed. 2to3 can also write the
Benjamin Petersonad3d5c22009-02-26 03:38:59 +000041needed modifications right back to the source file. (A backup of the original
42file is made unless :option:`-n` is also given.) Writing the changes back is
43enabled with the :option:`-w` flag::
Benjamin Petersond6313712008-07-31 16:23:04 +000044
45 $ 2to3 -w example.py
46
Benjamin Petersonae5360b2008-09-08 23:05:23 +000047After transformation, :file:`example.py` looks like this::
Benjamin Petersond6313712008-07-31 16:23:04 +000048
49 def greet(name):
50 print("Hello, {0}!".format(name))
51 print("What's your name?")
52 name = input()
53 greet(name)
54
Benjamin Peterson1a6e0d02008-10-25 15:49:17 +000055Comments and exact indentation are preserved throughout the translation process.
Benjamin Petersond6313712008-07-31 16:23:04 +000056
Benjamin Petersonf91df042009-02-13 02:50:59 +000057By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`. The
58:option:`-l` flag lists all available fixers. An explicit set of fixers to run
59can be given with :option:`-f`. Likewise the :option:`-x` explicitly disables a
60fixer. The following example runs only the ``imports`` and ``has_key`` fixers::
Benjamin Petersond6313712008-07-31 16:23:04 +000061
62 $ 2to3 -f imports -f has_key example.py
63
Benjamin Peterson206e3072008-10-19 14:07:49 +000064This command runs every fixer except the ``apply`` fixer::
65
66 $ 2to3 -x apply example.py
67
Benjamin Peterson1a6e0d02008-10-25 15:49:17 +000068Some fixers are *explicit*, meaning they aren't run by default and must be
Benjamin Petersonae5360b2008-09-08 23:05:23 +000069listed on the command line to be run. Here, in addition to the default fixers,
70the ``idioms`` fixer is run::
Benjamin Petersond6313712008-07-31 16:23:04 +000071
72 $ 2to3 -f all -f idioms example.py
73
Benjamin Petersonae5360b2008-09-08 23:05:23 +000074Notice how passing ``all`` enables all default fixers.
Benjamin Petersond6313712008-07-31 16:23:04 +000075
Benjamin Peterson1a6e0d02008-10-25 15:49:17 +000076Sometimes 2to3 will find a place in your source code that needs to be changed,
77but 2to3 cannot fix automatically. In this case, 2to3 will print a warning
78beneath the diff for a file. You should address the warning in order to have
79compliant 3.x code.
Benjamin Petersonae5360b2008-09-08 23:05:23 +000080
812to3 can also refactor doctests. To enable this mode, use the :option:`-d`
Benjamin Petersone9bbc8b2008-09-28 02:06:32 +000082flag. Note that *only* doctests will be refactored. This also doesn't require
83the module to be valid Python. For example, doctest like examples in a reST
84document could also be refactored with this option.
Benjamin Petersonae5360b2008-09-08 23:05:23 +000085
Benjamin Peterson206e3072008-10-19 14:07:49 +000086The :option:`-v` option enables output of more information on the translation
87process.
Benjamin Petersonae5360b2008-09-08 23:05:23 +000088
Benjamin Petersona0dfa822009-11-13 02:25:08 +000089Since some print statements can be parsed as function calls or statements, 2to3
90cannot always read files containing the print function. When 2to3 detects the
91presence of the ``from __future__ import print_function`` compiler directive, it
Georg Brandl6faee4e2010-09-21 14:48:28 +000092modifies its internal grammar to interpret :func:`print` as a function. This
Benjamin Petersona0dfa822009-11-13 02:25:08 +000093change can also be enabled manually with the :option:`-p` flag. Use
94:option:`-p` to run fixers on code that already has had its print statements
95converted.
96
Gregory P. Smith58f23ff2012-02-12 15:50:21 -080097The :option:`-o` or :option:`--output-dir` option allows specification of an
98alternate directory for processed output files to be written to. The
99:option:`-n` flag is required when using this as backup files do not make sense
100when not overwriting the input files.
101
102.. versionadded:: 3.2.3
103 The :option:`-o` option was added.
104
105The :option:`-W` or :option:`--write-unchanged-files` flag tells 2to3 to always
106write output files even if no changes were required to the file. This is most
107useful with :option:`-o` so that an entire Python source tree is copied with
108translation from one directory to another.
109This option implies the :option:`-w` flag as it would not make sense otherwise.
110
111.. versionadded:: 3.2.3
112 The :option:`-W` flag was added.
113
114The :option:`--add-suffix` option specifies a string to append to all output
115filenames. The :option:`-n` flag is required when specifying this as backups
116are not necessary when writing to different filenames. Example::
117
118 $ 2to3 -n -W --add-suffix=3 example.py
119
120Will cause a converted file named ``example.py3`` to be written.
121
122.. versionadded:: 3.2.3
123 The :option:`--add-suffix` option was added.
124
125To translate an entire project from one directory tree to another use::
126
127 $ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode
128
Benjamin Petersonf91df042009-02-13 02:50:59 +0000129
130.. _2to3-fixers:
131
132Fixers
133------
134
Georg Brandlae2dbe22009-03-13 19:04:40 +0000135Each step of transforming code is encapsulated in a fixer. The command ``2to3
Benjamin Petersonf91df042009-02-13 02:50:59 +0000136-l`` lists them. As :ref:`documented above <2to3-using>`, each can be turned on
137and off individually. They are described here in more detail.
138
139
140.. 2to3fixer:: apply
141
142 Removes usage of :func:`apply`. For example ``apply(function, *args,
143 **kwargs)`` is converted to ``function(*args, **kwargs)``.
144
145.. 2to3fixer:: basestring
146
147 Converts :class:`basestring` to :class:`str`.
148
149.. 2to3fixer:: buffer
150
151 Converts :class:`buffer` to :class:`memoryview`. This fixer is optional
152 because the :class:`memoryview` API is similar but not exactly the same as
153 that of :class:`buffer`.
154
155.. 2to3fixer:: callable
156
Benjamin Peterson1baf4652009-12-31 03:11:23 +0000157 Converts ``callable(x)`` to ``isinstance(x, collections.Callable)``, adding
Benjamin Petersond6ca6c22011-10-24 08:51:15 -0400158 an import to :mod:`collections` if needed. Note ``callable(x)`` has returned
159 in Python 3.2, so if you do not intend to support Python 3.1, you can disable
160 this fixer.
Benjamin Petersonf91df042009-02-13 02:50:59 +0000161
162.. 2to3fixer:: dict
163
164 Fixes dictionary iteration methods. :meth:`dict.iteritems` is converted to
165 :meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and
Alexandre Vassalottia515bab2010-01-12 18:38:14 +0000166 :meth:`dict.itervalues` to :meth:`dict.values`. Similarly,
Benjamin Petersone90f54e2010-03-21 23:17:57 +0000167 :meth:`dict.viewitems`, :meth:`dict.viewkeys` and :meth:`dict.viewvalues` are
168 converted respectively to :meth:`dict.items`, :meth:`dict.keys` and
Alexandre Vassalottia515bab2010-01-12 18:38:14 +0000169 :meth:`dict.values`. It also wraps existing usages of :meth:`dict.items`,
170 :meth:`dict.keys`, and :meth:`dict.values` in a call to :class:`list`.
Benjamin Petersonf91df042009-02-13 02:50:59 +0000171
172.. 2to3fixer:: except
173
174 Converts ``except X, T`` to ``except X as T``.
175
176.. 2to3fixer:: exec
177
Georg Brandl375aec22011-01-15 17:03:02 +0000178 Converts the ``exec`` statement to the :func:`exec` function.
Benjamin Petersonf91df042009-02-13 02:50:59 +0000179
180.. 2to3fixer:: execfile
181
182 Removes usage of :func:`execfile`. The argument to :func:`execfile` is
183 wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`.
184
Benjamin Petersone90f54e2010-03-21 23:17:57 +0000185.. 2to3fixer:: exitfunc
186
187 Changes assignment of :attr:`sys.exitfunc` to use of the :mod:`atexit`
188 module.
189
Benjamin Petersonf91df042009-02-13 02:50:59 +0000190.. 2to3fixer:: filter
191
192 Wraps :func:`filter` usage in a :class:`list` call.
193
194.. 2to3fixer:: funcattrs
195
196 Fixes function attributes that have been renamed. For example,
197 ``my_function.func_closure`` is converted to ``my_function.__closure__``.
198
199.. 2to3fixer:: future
200
201 Removes ``from __future__ import new_feature`` statements.
202
203.. 2to3fixer:: getcwdu
204
205 Renames :func:`os.getcwdu` to :func:`os.getcwd`.
206
207.. 2to3fixer:: has_key
208
209 Changes ``dict.has_key(key)`` to ``key in dict``.
210
211.. 2to3fixer:: idioms
212
Georg Brandlae2dbe22009-03-13 19:04:40 +0000213 This optional fixer performs several transformations that make Python code
214 more idiomatic. Type comparisons like ``type(x) is SomeClass`` and
Benjamin Petersonf91df042009-02-13 02:50:59 +0000215 ``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``.
216 ``while 1`` becomes ``while True``. This fixer also tries to make use of
Georg Brandlae2dbe22009-03-13 19:04:40 +0000217 :func:`sorted` in appropriate places. For example, this block ::
Benjamin Petersonf91df042009-02-13 02:50:59 +0000218
219 L = list(some_iterable)
220 L.sort()
221
222 is changed to ::
223
224 L = sorted(some_iterable)
225
226.. 2to3fixer:: import
227
228 Detects sibling imports and converts them to relative imports.
229
230.. 2to3fixer:: imports
231
232 Handles module renames in the standard library.
233
234.. 2to3fixer:: imports2
235
236 Handles other modules renames in the standard library. It is separate from
237 the :2to3fixer:`imports` fixer only because of technical limitations.
238
239.. 2to3fixer:: input
240
241 Converts ``input(prompt)`` to ``eval(input(prompt))``
242
243.. 2to3fixer:: intern
244
245 Converts :func:`intern` to :func:`sys.intern`.
246
247.. 2to3fixer:: isinstance
248
249 Fixes duplicate types in the second argument of :func:`isinstance`. For
250 example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x,
251 (int))``.
252
253.. 2to3fixer:: itertools_imports
254
255 Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and
256 :func:`itertools.imap`. Imports of :func:`itertools.ifilterfalse` are also
257 changed to :func:`itertools.filterfalse`.
258
259.. 2to3fixer:: itertools
260
261 Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and
Georg Brandl22b34312009-07-26 14:54:51 +0000262 :func:`itertools.imap` to their built-in equivalents.
Benjamin Petersonf91df042009-02-13 02:50:59 +0000263 :func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`.
264
265.. 2to3fixer:: long
266
Benjamin Peterson4dafd402013-01-28 18:28:38 -0500267 Renames :class:`long` to :class:`int`.
Benjamin Petersonf91df042009-02-13 02:50:59 +0000268
269.. 2to3fixer:: map
270
271 Wraps :func:`map` in a :class:`list` call. It also changes ``map(None, x)``
272 to ``list(x)``. Using ``from future_builtins import map`` disables this
273 fixer.
274
275.. 2to3fixer:: metaclass
276
277 Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class
278 body) to the new (``class X(metaclass=Meta)``).
279
280.. 2to3fixer:: methodattrs
281
282 Fixes old method attribute names. For example, ``meth.im_func`` is converted
283 to ``meth.__func__``.
284
285.. 2to3fixer:: ne
286
287 Converts the old not-equal syntax, ``<>``, to ``!=``.
288
289.. 2to3fixer:: next
290
Georg Brandl502d9a52009-07-26 15:02:41 +0000291 Converts the use of iterator's :meth:`~iterator.next` methods to the
292 :func:`next` function. It also renames :meth:`next` methods to
293 :meth:`~object.__next__`.
Benjamin Petersonf91df042009-02-13 02:50:59 +0000294
295.. 2to3fixer:: nonzero
296
297 Renames :meth:`~object.__nonzero__` to :meth:`~object.__bool__`.
298
299.. 2to3fixer:: numliterals
300
301 Converts octal literals into the new syntax.
302
Alexandre Vassalottiae780182010-08-05 07:12:18 +0000303.. 2to3fixer:: operator
304
305 Converts calls to various functions in the :mod:`operator` module to other,
306 but equivalent, function calls. When needed, the appropriate ``import``
307 statements are added, e.g. ``import collections``. The following mapping
308 are made:
309
310 ================================== ==========================================
311 From To
312 ================================== ==========================================
313 ``operator.isCallable(obj)`` ``hasattr(obj, '__call__')``
314 ``operator.sequenceIncludes(obj)`` ``operator.contains(obj)``
315 ``operator.isSequenceType(obj)`` ``isinstance(obj, collections.Sequence)``
316 ``operator.isMappingType(obj)`` ``isinstance(obj, collections.Mapping)``
317 ``operator.isNumberType(obj)`` ``isinstance(obj, numbers.Number)``
318 ``operator.repeat(obj, n)`` ``operator.mul(obj, n)``
319 ``operator.irepeat(obj, n)`` ``operator.imul(obj, n)``
320 ================================== ==========================================
321
Benjamin Petersonf91df042009-02-13 02:50:59 +0000322.. 2to3fixer:: paren
323
324 Add extra parenthesis where they are required in list comprehensions. For
325 example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``.
326
327.. 2to3fixer:: print
328
Georg Brandl375aec22011-01-15 17:03:02 +0000329 Converts the ``print`` statement to the :func:`print` function.
Benjamin Petersonf91df042009-02-13 02:50:59 +0000330
Benjamin Petersonb51b5c42010-07-01 17:49:01 +0000331.. 2to3fixer:: raise
Benjamin Petersonf91df042009-02-13 02:50:59 +0000332
333 Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise
334 E(V).with_traceback(T)``. If ``E`` is a tuple, the translation will be
335 incorrect because substituting tuples for exceptions has been removed in 3.0.
336
337.. 2to3fixer:: raw_input
338
339 Converts :func:`raw_input` to :func:`input`.
340
341.. 2to3fixer:: reduce
342
343 Handles the move of :func:`reduce` to :func:`functools.reduce`.
344
Benjamin Peterson448e81b2012-12-07 22:44:10 -0500345.. 2to3fixer:: reload
346
347 Converts :func:`reload` to :func:`imp.reload`.
348
Benjamin Petersonf91df042009-02-13 02:50:59 +0000349.. 2to3fixer:: renames
350
351 Changes :data:`sys.maxint` to :data:`sys.maxsize`.
352
353.. 2to3fixer:: repr
354
355 Replaces backtick repr with the :func:`repr` function.
356
357.. 2to3fixer:: set_literal
358
359 Replaces use of the :class:`set` constructor with set literals. This fixer
360 is optional.
361
362.. 2to3fixer:: standard_error
363
364 Renames :exc:`StandardError` to :exc:`Exception`.
365
366.. 2to3fixer:: sys_exc
367
368 Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`,
369 :data:`sys.exc_traceback` to use :func:`sys.exc_info`.
370
371.. 2to3fixer:: throw
372
373 Fixes the API change in generator's :meth:`throw` method.
374
375.. 2to3fixer:: tuple_params
376
377 Removes implicit tuple parameter unpacking. This fixer inserts temporary
378 variables.
379
380.. 2to3fixer:: types
381
382 Fixes code broken from the removal of some members in the :mod:`types`
383 module.
384
385.. 2to3fixer:: unicode
386
387 Renames :class:`unicode` to :class:`str`.
388
389.. 2to3fixer:: urllib
390
391 Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib`
392 package.
393
394.. 2to3fixer:: ws_comma
395
396 Removes excess whitespace from comma separated items. This fixer is
397 optional.
398
399.. 2to3fixer:: xrange
400
401 Renames :func:`xrange` to :func:`range` and wraps existing :func:`range`
402 calls with :class:`list`.
403
404.. 2to3fixer:: xreadlines
405
406 Changes ``for x in file.xreadlines()`` to ``for x in file``.
407
408.. 2to3fixer:: zip
409
410 Wraps :func:`zip` usage in a :class:`list` call. This is disabled when
411 ``from future_builtins import zip`` appears.
Benjamin Petersond6313712008-07-31 16:23:04 +0000412
413
414:mod:`lib2to3` - 2to3's library
415-------------------------------
416
417.. module:: lib2to3
418 :synopsis: the 2to3 library
419.. moduleauthor:: Guido van Rossum
420.. moduleauthor:: Collin Winter
Benjamin Peterson25c95f12009-05-08 20:42:26 +0000421.. moduleauthor:: Benjamin Peterson <benjamin@python.org>
Benjamin Petersond6313712008-07-31 16:23:04 +0000422
Benjamin Petersone9bbc8b2008-09-28 02:06:32 +0000423
Georg Brandle720c0a2009-04-27 16:20:50 +0000424.. note::
Benjamin Petersone9bbc8b2008-09-28 02:06:32 +0000425
426 The :mod:`lib2to3` API should be considered unstable and may change
427 drastically in the future.
428
Benjamin Petersond6313712008-07-31 16:23:04 +0000429.. XXX What is the public interface anyway?