Doc/library/2to3.rst

.. _2to3-reference:

2to3 - Automated Python 2 to 3 code translation
===============================================

.. sectionauthor:: Benjamin Peterson <benjamin@python.org>

2to3 is a Python program that reads Python 2.x source code and applies a series
of *fixers* to transform it into valid Python 3.x code.  The standard library
contains a rich set of fixers that will handle almost all code.  2to3 supporting
library :mod:`lib2to3` is, however, a flexible and generic library, so it is
possible to write your own fixers for 2to3.  :mod:`lib2to3` could also be
adapted to custom applications in which Python code needs to be edited
automatically.


.. _2to3-using:

Using 2to3
----------

2to3 will usually be installed with the Python interpreter as a script.  It is
also located in the :file:`Tools/scripts` directory of the Python root.

2to3's basic arguments are a list of files or directories to transform.  The
directories are to recursively traversed for Python sources.

Here is a sample Python 2.x source file, :file:`example.py`::

   def greet(name):
       print "Hello, {0}!".format(name)
   print "What's your name?"
   name = raw_input()
   greet(name)

It can be converted to Python 3.x code via 2to3 on the command line::

   $ 2to3 example.py

A diff against the original source file is printed.  2to3 can also write the
needed modifications right back to the source file.  (A backup of the original
file is made unless :option:`-n` is also given.)  Writing the changes back is
enabled with the :option:`-w` flag::

   $ 2to3 -w example.py

After transformation, :file:`example.py` looks like this::

   def greet(name):
       print("Hello, {0}!".format(name))
   print("What's your name?")
   name = input()
   greet(name)

Comments and exact indentation are preserved throughout the translation process.

By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`.  The
:option:`-l` flag lists all available fixers.  An explicit set of fixers to run
can be given with :option:`-f`.  Likewise the :option:`-x` explicitly disables a
fixer.  The following example runs only the ``imports`` and ``has_key`` fixers::

   $ 2to3 -f imports -f has_key example.py

This command runs every fixer except the ``apply`` fixer::

   $ 2to3 -x apply example.py

Some fixers are *explicit*, meaning they aren't run by default and must be
listed on the command line to be run.  Here, in addition to the default fixers,
the ``idioms`` fixer is run::

   $ 2to3 -f all -f idioms example.py

Notice how passing ``all`` enables all default fixers.

Sometimes 2to3 will find a place in your source code that needs to be changed,
but 2to3 cannot fix automatically.  In this case, 2to3 will print a warning
beneath the diff for a file.  You should address the warning in order to have
compliant 3.x code.

2to3 can also refactor doctests.  To enable this mode, use the :option:`-d`
flag.  Note that *only* doctests will be refactored.  This also doesn't require
the module to be valid Python.  For example, doctest like examples in a reST
document could also be refactored with this option.

The :option:`-v` option enables output of more information on the translation
process.


.. _2to3-fixers:

Fixers
------

Each step of tranforming code is encapsulated in a fixer.  The command ``2to3
-l`` lists them.  As :ref:`documented above <2to3-using>`, each can be turned on
and off individually.  They are described here in more detail.


.. 2to3fixer:: apply

   Removes usage of :func:`apply`.  For example ``apply(function, *args,
   **kwargs)`` is converted to ``function(*args, **kwargs)``.

.. 2to3fixer:: basestring

   Converts :class:`basestring` to :class:`str`.

.. 2to3fixer:: buffer

   Converts :class:`buffer` to :class:`memoryview`.  This fixer is optional
   because the :class:`memoryview` API is similar but not exactly the same as
   that of :class:`buffer`.

.. 2to3fixer:: callable

   Converts ``callable(x)`` to ``hasattr(x, "__call_")``.

.. 2to3fixer:: dict

   Fixes dictionary iteration methods.  :meth:`dict.iteritems` is converted to
   :meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and
   :meth:`dict.itervalues` to :meth:`dict.values`.  It also wraps existing
   usages of :meth:`dict.items`, :meth:`dict.keys`, and :meth:`dict.values` in a
   call to :class:`list`.

.. 2to3fixer:: except

   Converts ``except X, T`` to ``except X as T``.

.. 2to3fixer:: exec

   Converts the :keyword:`exec` statement to the :func:`exec` function.

.. 2to3fixer:: execfile

   Removes usage of :func:`execfile`.  The argument to :func:`execfile` is
   wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`.

.. 2to3fixer:: filter

   Wraps :func:`filter` usage in a :class:`list` call.

.. 2to3fixer:: funcattrs

   Fixes function attributes that have been renamed.  For example,
   ``my_function.func_closure`` is converted to ``my_function.__closure__``.

.. 2to3fixer:: future

   Removes ``from __future__ import new_feature`` statements.

.. 2to3fixer:: getcwdu

   Renames :func:`os.getcwdu` to :func:`os.getcwd`.

.. 2to3fixer:: has_key

   Changes ``dict.has_key(key)`` to ``key in dict``.

.. 2to3fixer:: idioms

   This optional fixer preforms several transformations that make Python code
   more idiomatic.  Type comparisions like ``type(x) is SomeClass`` and
   ``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``.
   ``while 1`` becomes ``while True``.  This fixer also tries to make use of
   :func:`sorted` in appropiate places.  For example, this block ::

       L = list(some_iterable)
       L.sort()

   is changed to ::

      L = sorted(some_iterable)

.. 2to3fixer:: import

   Detects sibling imports and converts them to relative imports.

.. 2to3fixer:: imports

   Handles module renames in the standard library.

.. 2to3fixer:: imports2

   Handles other modules renames in the standard library.  It is separate from
   the :2to3fixer:`imports` fixer only because of technical limitations.

.. 2to3fixer:: input

   Converts ``input(prompt)`` to ``eval(input(prompt))``

.. 2to3fixer:: intern

   Converts :func:`intern` to :func:`sys.intern`.

.. 2to3fixer:: isinstance

   Fixes duplicate types in the second argument of :func:`isinstance`.  For
   example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x,
   (int))``.

.. 2to3fixer:: itertools_imports

   Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and
   :func:`itertools.imap`.  Imports of :func:`itertools.ifilterfalse` are also
   changed to :func:`itertools.filterfalse`.

.. 2to3fixer:: itertools

   Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and
   :func:`itertools.imap` to their built-in equivalents.
   :func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`.

.. 2to3fixer:: long

   Strips the ``L`` prefix on long literals and renames :class:`long` to
   :class:`int`.

.. 2to3fixer:: map

   Wraps :func:`map` in a :class:`list` call.  It also changes ``map(None, x)``
   to ``list(x)``.  Using ``from future_builtins import map`` disables this
   fixer.

.. 2to3fixer:: metaclass

   Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class
   body) to the new (``class X(metaclass=Meta)``).

.. 2to3fixer:: methodattrs

   Fixes old method attribute names.  For example, ``meth.im_func`` is converted
   to ``meth.__func__``.

.. 2to3fixer:: ne

   Converts the old not-equal syntax, ``<>``, to ``!=``.

.. 2to3fixer:: next

   Converts the use of iterator's :meth:`~iterator.next` methods to the
   :func:`next` function.  It also renames :meth:`next` methods to
   :meth:`~object.__next__`.

.. 2to3fixer:: nonzero

   Renames :meth:`~object.__nonzero__` to :meth:`~object.__bool__`.

.. 2to3fixer:: numliterals

   Converts octal literals into the new syntax.

.. 2to3fixer:: paren

   Add extra parenthesis where they are required in list comprehensions.  For
   example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``.

.. 2to3fixer:: print

   Converts the :keyword:`print` statement to the :func:`print` function.

.. 2to3fixer:: raises

   Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise
   E(V).with_traceback(T)``.  If ``E`` is a tuple, the translation will be
   incorrect because substituting tuples for exceptions has been removed in 3.0.

.. 2to3fixer:: raw_input

   Converts :func:`raw_input` to :func:`input`.

.. 2to3fixer:: reduce

   Handles the move of :func:`reduce` to :func:`functools.reduce`.

.. 2to3fixer:: renames

   Changes :data:`sys.maxint` to :data:`sys.maxsize`.

.. 2to3fixer:: repr

   Replaces backtick repr with the :func:`repr` function.

.. 2to3fixer:: set_literal

   Replaces use of the :class:`set` constructor with set literals.  This fixer
   is optional.

.. 2to3fixer:: standard_error

   Renames :exc:`StandardError` to :exc:`Exception`.

.. 2to3fixer:: sys_exc

   Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`,
   :data:`sys.exc_traceback` to use :func:`sys.exc_info`.

.. 2to3fixer:: throw

   Fixes the API change in generator's :meth:`throw` method.

.. 2to3fixer:: tuple_params

   Removes implicit tuple parameter unpacking.  This fixer inserts temporary
   variables.

.. 2to3fixer:: types

   Fixes code broken from the removal of some members in the :mod:`types`
   module.

.. 2to3fixer:: unicode

   Renames :class:`unicode` to :class:`str`.

.. 2to3fixer:: urllib

   Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib`
   package.

.. 2to3fixer:: ws_comma

   Removes excess whitespace from comma separated items.  This fixer is
   optional.

.. 2to3fixer:: xrange

   Renames :func:`xrange` to :func:`range` and wraps existing :func:`range`
   calls with :class:`list`.

.. 2to3fixer:: xreadlines

   Changes ``for x in file.xreadlines()`` to ``for x in file``.

.. 2to3fixer:: zip

   Wraps :func:`zip` usage in a :class:`list` call.  This is disabled when
   ``from future_builtins import zip`` appears.


:mod:`lib2to3` - 2to3's library
-------------------------------

.. module:: lib2to3
   :synopsis: the 2to3 library
.. moduleauthor:: Guido van Rossum
.. moduleauthor:: Collin Winter
.. moduleauthor:: Benjamin Peterson <benjamin@python.org>


.. note::

   The :mod:`lib2to3` API should be considered unstable and may change
   drastically in the future.

.. XXX What is the public interface anyway?