Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 1 | .. _2to3-reference: |
| 2 | |
| 3 | 2to3 - Automated Python 2 to 3 code translation |
| 4 | =============================================== |
| 5 | |
Benjamin Peterson | 058e31e | 2009-01-16 03:54:08 +0000 | [diff] [blame] | 6 | .. sectionauthor:: Benjamin Peterson <benjamin@python.org> |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 7 | |
Benjamin Peterson | 8951b61 | 2008-09-03 02:27:16 +0000 | [diff] [blame] | 8 | 2to3 is a Python program that reads Python 2.x source code and applies a series |
| 9 | of *fixers* to transform it into valid Python 3.x code. The standard library |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 10 | contains a rich set of fixers that will handle almost all code. 2to3 supporting |
| 11 | library :mod:`lib2to3` is, however, a flexible and generic library, so it is |
| 12 | possible to write your own fixers for 2to3. :mod:`lib2to3` could also be |
| 13 | adapted to custom applications in which Python code needs to be edited |
| 14 | automatically. |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 15 | |
| 16 | |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 17 | .. _2to3-using: |
| 18 | |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 19 | Using 2to3 |
| 20 | ---------- |
| 21 | |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 22 | 2to3 will usually be installed with the Python interpreter as a script. It is |
| 23 | also located in the :file:`Tools/scripts` directory of the Python root. |
| 24 | |
| 25 | 2to3's basic arguments are a list of files or directories to transform. The |
| 26 | directories are to recursively traversed for Python sources. |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 27 | |
| 28 | Here is a sample Python 2.x source file, :file:`example.py`:: |
| 29 | |
| 30 | def greet(name): |
| 31 | print "Hello, {0}!".format(name) |
| 32 | print "What's your name?" |
| 33 | name = raw_input() |
| 34 | greet(name) |
| 35 | |
| 36 | It can be converted to Python 3.x code via 2to3 on the command line:: |
| 37 | |
| 38 | $ 2to3 example.py |
| 39 | |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 40 | A diff against the original source file is printed. 2to3 can also write the |
Benjamin Peterson | ad3d5c2 | 2009-02-26 03:38:59 +0000 | [diff] [blame] | 41 | needed modifications right back to the source file. (A backup of the original |
| 42 | file is made unless :option:`-n` is also given.) Writing the changes back is |
| 43 | enabled with the :option:`-w` flag:: |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 44 | |
| 45 | $ 2to3 -w example.py |
| 46 | |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 47 | After transformation, :file:`example.py` looks like this:: |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 48 | |
| 49 | def greet(name): |
| 50 | print("Hello, {0}!".format(name)) |
| 51 | print("What's your name?") |
| 52 | name = input() |
| 53 | greet(name) |
| 54 | |
Benjamin Peterson | 1a6e0d0 | 2008-10-25 15:49:17 +0000 | [diff] [blame] | 55 | Comments and exact indentation are preserved throughout the translation process. |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 56 | |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 57 | By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`. The |
| 58 | :option:`-l` flag lists all available fixers. An explicit set of fixers to run |
| 59 | can be given with :option:`-f`. Likewise the :option:`-x` explicitly disables a |
| 60 | fixer. The following example runs only the ``imports`` and ``has_key`` fixers:: |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 61 | |
| 62 | $ 2to3 -f imports -f has_key example.py |
| 63 | |
Benjamin Peterson | 206e307 | 2008-10-19 14:07:49 +0000 | [diff] [blame] | 64 | This command runs every fixer except the ``apply`` fixer:: |
| 65 | |
| 66 | $ 2to3 -x apply example.py |
| 67 | |
Benjamin Peterson | 1a6e0d0 | 2008-10-25 15:49:17 +0000 | [diff] [blame] | 68 | Some fixers are *explicit*, meaning they aren't run by default and must be |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 69 | listed on the command line to be run. Here, in addition to the default fixers, |
| 70 | the ``idioms`` fixer is run:: |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 71 | |
| 72 | $ 2to3 -f all -f idioms example.py |
| 73 | |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 74 | Notice how passing ``all`` enables all default fixers. |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 75 | |
Benjamin Peterson | 1a6e0d0 | 2008-10-25 15:49:17 +0000 | [diff] [blame] | 76 | Sometimes 2to3 will find a place in your source code that needs to be changed, |
| 77 | but 2to3 cannot fix automatically. In this case, 2to3 will print a warning |
| 78 | beneath the diff for a file. You should address the warning in order to have |
| 79 | compliant 3.x code. |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 80 | |
| 81 | 2to3 can also refactor doctests. To enable this mode, use the :option:`-d` |
Benjamin Peterson | e9bbc8b | 2008-09-28 02:06:32 +0000 | [diff] [blame] | 82 | flag. Note that *only* doctests will be refactored. This also doesn't require |
| 83 | the module to be valid Python. For example, doctest like examples in a reST |
| 84 | document could also be refactored with this option. |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 85 | |
Benjamin Peterson | 206e307 | 2008-10-19 14:07:49 +0000 | [diff] [blame] | 86 | The :option:`-v` option enables output of more information on the translation |
| 87 | process. |
Benjamin Peterson | ae5360b | 2008-09-08 23:05:23 +0000 | [diff] [blame] | 88 | |
Benjamin Peterson | 8f6713f | 2009-11-13 02:29:35 +0000 | [diff] [blame] | 89 | Since some print statements can be parsed as function calls or statements, 2to3 |
| 90 | cannot always read files containing the print function. When 2to3 detects the |
| 91 | presence of the ``from __future__ import print_function`` compiler directive, it |
| 92 | modifies its internal grammar to interpert :func:`print` as a function. This |
| 93 | change can also be enabled manually with the :option:`-p` flag. Use |
| 94 | :option:`-p` to run fixers on code that already has had its print statements |
| 95 | converted. |
| 96 | |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 97 | |
| 98 | .. _2to3-fixers: |
| 99 | |
| 100 | Fixers |
| 101 | ------ |
| 102 | |
Georg Brandl | ae2dbe2 | 2009-03-13 19:04:40 +0000 | [diff] [blame] | 103 | Each step of transforming code is encapsulated in a fixer. The command ``2to3 |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 104 | -l`` lists them. As :ref:`documented above <2to3-using>`, each can be turned on |
| 105 | and off individually. They are described here in more detail. |
| 106 | |
| 107 | |
| 108 | .. 2to3fixer:: apply |
| 109 | |
| 110 | Removes usage of :func:`apply`. For example ``apply(function, *args, |
| 111 | **kwargs)`` is converted to ``function(*args, **kwargs)``. |
| 112 | |
| 113 | .. 2to3fixer:: basestring |
| 114 | |
| 115 | Converts :class:`basestring` to :class:`str`. |
| 116 | |
| 117 | .. 2to3fixer:: buffer |
| 118 | |
| 119 | Converts :class:`buffer` to :class:`memoryview`. This fixer is optional |
| 120 | because the :class:`memoryview` API is similar but not exactly the same as |
| 121 | that of :class:`buffer`. |
| 122 | |
| 123 | .. 2to3fixer:: callable |
| 124 | |
Benjamin Peterson | 68dbebc | 2009-12-31 03:30:26 +0000 | [diff] [blame] | 125 | Converts ``callable(x)`` to ``isinstance(x, collections.Callable)``, adding |
| 126 | an import to :mod:`collections` if needed. |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 127 | |
| 128 | .. 2to3fixer:: dict |
| 129 | |
| 130 | Fixes dictionary iteration methods. :meth:`dict.iteritems` is converted to |
| 131 | :meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and |
Benjamin Peterson | 4e74fc8 | 2010-03-21 23:22:58 +0000 | [diff] [blame] | 132 | :meth:`dict.itervalues` to :meth:`dict.values`. Similarly, |
| 133 | :meth:`dict.viewitems`, :meth:`dict.viewkeys` and :meth:`dict.viewvalues` are |
| 134 | converted respectively to :meth:`dict.items`, :meth:`dict.keys` and |
| 135 | :meth:`dict.values`. It also wraps existing usages of :meth:`dict.items`, |
| 136 | :meth:`dict.keys`, and :meth:`dict.values` in a call to :class:`list`. |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 137 | |
| 138 | .. 2to3fixer:: except |
| 139 | |
| 140 | Converts ``except X, T`` to ``except X as T``. |
| 141 | |
| 142 | .. 2to3fixer:: exec |
| 143 | |
| 144 | Converts the :keyword:`exec` statement to the :func:`exec` function. |
| 145 | |
| 146 | .. 2to3fixer:: execfile |
| 147 | |
| 148 | Removes usage of :func:`execfile`. The argument to :func:`execfile` is |
| 149 | wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`. |
| 150 | |
Benjamin Peterson | 4e74fc8 | 2010-03-21 23:22:58 +0000 | [diff] [blame] | 151 | .. 2to3fixer:: exitfunc |
| 152 | |
| 153 | Changes assignment of :attr:`sys.exitfunc` to use of the :mod:`atexit` |
| 154 | module. |
| 155 | |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 156 | .. 2to3fixer:: filter |
| 157 | |
| 158 | Wraps :func:`filter` usage in a :class:`list` call. |
| 159 | |
| 160 | .. 2to3fixer:: funcattrs |
| 161 | |
| 162 | Fixes function attributes that have been renamed. For example, |
| 163 | ``my_function.func_closure`` is converted to ``my_function.__closure__``. |
| 164 | |
| 165 | .. 2to3fixer:: future |
| 166 | |
| 167 | Removes ``from __future__ import new_feature`` statements. |
| 168 | |
| 169 | .. 2to3fixer:: getcwdu |
| 170 | |
| 171 | Renames :func:`os.getcwdu` to :func:`os.getcwd`. |
| 172 | |
| 173 | .. 2to3fixer:: has_key |
| 174 | |
| 175 | Changes ``dict.has_key(key)`` to ``key in dict``. |
| 176 | |
| 177 | .. 2to3fixer:: idioms |
| 178 | |
Georg Brandl | ae2dbe2 | 2009-03-13 19:04:40 +0000 | [diff] [blame] | 179 | This optional fixer performs several transformations that make Python code |
| 180 | more idiomatic. Type comparisons like ``type(x) is SomeClass`` and |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 181 | ``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``. |
| 182 | ``while 1`` becomes ``while True``. This fixer also tries to make use of |
Georg Brandl | ae2dbe2 | 2009-03-13 19:04:40 +0000 | [diff] [blame] | 183 | :func:`sorted` in appropriate places. For example, this block :: |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 184 | |
| 185 | L = list(some_iterable) |
| 186 | L.sort() |
| 187 | |
| 188 | is changed to :: |
| 189 | |
| 190 | L = sorted(some_iterable) |
| 191 | |
| 192 | .. 2to3fixer:: import |
| 193 | |
| 194 | Detects sibling imports and converts them to relative imports. |
| 195 | |
| 196 | .. 2to3fixer:: imports |
| 197 | |
| 198 | Handles module renames in the standard library. |
| 199 | |
| 200 | .. 2to3fixer:: imports2 |
| 201 | |
| 202 | Handles other modules renames in the standard library. It is separate from |
| 203 | the :2to3fixer:`imports` fixer only because of technical limitations. |
| 204 | |
| 205 | .. 2to3fixer:: input |
| 206 | |
| 207 | Converts ``input(prompt)`` to ``eval(input(prompt))`` |
| 208 | |
| 209 | .. 2to3fixer:: intern |
| 210 | |
| 211 | Converts :func:`intern` to :func:`sys.intern`. |
| 212 | |
| 213 | .. 2to3fixer:: isinstance |
| 214 | |
| 215 | Fixes duplicate types in the second argument of :func:`isinstance`. For |
| 216 | example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x, |
| 217 | (int))``. |
| 218 | |
| 219 | .. 2to3fixer:: itertools_imports |
| 220 | |
| 221 | Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and |
| 222 | :func:`itertools.imap`. Imports of :func:`itertools.ifilterfalse` are also |
| 223 | changed to :func:`itertools.filterfalse`. |
| 224 | |
| 225 | .. 2to3fixer:: itertools |
| 226 | |
| 227 | Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and |
Georg Brandl | c5605df | 2009-08-13 08:26:44 +0000 | [diff] [blame] | 228 | :func:`itertools.imap` to their built-in equivalents. |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 229 | :func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`. |
| 230 | |
| 231 | .. 2to3fixer:: long |
| 232 | |
| 233 | Strips the ``L`` prefix on long literals and renames :class:`long` to |
| 234 | :class:`int`. |
| 235 | |
| 236 | .. 2to3fixer:: map |
| 237 | |
| 238 | Wraps :func:`map` in a :class:`list` call. It also changes ``map(None, x)`` |
| 239 | to ``list(x)``. Using ``from future_builtins import map`` disables this |
| 240 | fixer. |
| 241 | |
| 242 | .. 2to3fixer:: metaclass |
| 243 | |
| 244 | Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class |
| 245 | body) to the new (``class X(metaclass=Meta)``). |
| 246 | |
| 247 | .. 2to3fixer:: methodattrs |
| 248 | |
| 249 | Fixes old method attribute names. For example, ``meth.im_func`` is converted |
| 250 | to ``meth.__func__``. |
| 251 | |
| 252 | .. 2to3fixer:: ne |
| 253 | |
| 254 | Converts the old not-equal syntax, ``<>``, to ``!=``. |
| 255 | |
| 256 | .. 2to3fixer:: next |
| 257 | |
Georg Brandl | c5605df | 2009-08-13 08:26:44 +0000 | [diff] [blame] | 258 | Converts the use of iterator's :meth:`~iterator.next` methods to the |
| 259 | :func:`next` function. It also renames :meth:`next` methods to |
| 260 | :meth:`~object.__next__`. |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 261 | |
| 262 | .. 2to3fixer:: nonzero |
| 263 | |
| 264 | Renames :meth:`~object.__nonzero__` to :meth:`~object.__bool__`. |
| 265 | |
| 266 | .. 2to3fixer:: numliterals |
| 267 | |
| 268 | Converts octal literals into the new syntax. |
| 269 | |
| 270 | .. 2to3fixer:: paren |
| 271 | |
| 272 | Add extra parenthesis where they are required in list comprehensions. For |
| 273 | example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``. |
| 274 | |
| 275 | .. 2to3fixer:: print |
| 276 | |
| 277 | Converts the :keyword:`print` statement to the :func:`print` function. |
| 278 | |
Benjamin Peterson | d793847 | 2010-07-01 17:51:42 +0000 | [diff] [blame] | 279 | .. 2to3fixer:: raise |
Benjamin Peterson | f91df04 | 2009-02-13 02:50:59 +0000 | [diff] [blame] | 280 | |
| 281 | Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise |
| 282 | E(V).with_traceback(T)``. If ``E`` is a tuple, the translation will be |
| 283 | incorrect because substituting tuples for exceptions has been removed in 3.0. |
| 284 | |
| 285 | .. 2to3fixer:: raw_input |
| 286 | |
| 287 | Converts :func:`raw_input` to :func:`input`. |
| 288 | |
| 289 | .. 2to3fixer:: reduce |
| 290 | |
| 291 | Handles the move of :func:`reduce` to :func:`functools.reduce`. |
| 292 | |
| 293 | .. 2to3fixer:: renames |
| 294 | |
| 295 | Changes :data:`sys.maxint` to :data:`sys.maxsize`. |
| 296 | |
| 297 | .. 2to3fixer:: repr |
| 298 | |
| 299 | Replaces backtick repr with the :func:`repr` function. |
| 300 | |
| 301 | .. 2to3fixer:: set_literal |
| 302 | |
| 303 | Replaces use of the :class:`set` constructor with set literals. This fixer |
| 304 | is optional. |
| 305 | |
| 306 | .. 2to3fixer:: standard_error |
| 307 | |
| 308 | Renames :exc:`StandardError` to :exc:`Exception`. |
| 309 | |
| 310 | .. 2to3fixer:: sys_exc |
| 311 | |
| 312 | Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`, |
| 313 | :data:`sys.exc_traceback` to use :func:`sys.exc_info`. |
| 314 | |
| 315 | .. 2to3fixer:: throw |
| 316 | |
| 317 | Fixes the API change in generator's :meth:`throw` method. |
| 318 | |
| 319 | .. 2to3fixer:: tuple_params |
| 320 | |
| 321 | Removes implicit tuple parameter unpacking. This fixer inserts temporary |
| 322 | variables. |
| 323 | |
| 324 | .. 2to3fixer:: types |
| 325 | |
| 326 | Fixes code broken from the removal of some members in the :mod:`types` |
| 327 | module. |
| 328 | |
| 329 | .. 2to3fixer:: unicode |
| 330 | |
| 331 | Renames :class:`unicode` to :class:`str`. |
| 332 | |
| 333 | .. 2to3fixer:: urllib |
| 334 | |
| 335 | Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib` |
| 336 | package. |
| 337 | |
| 338 | .. 2to3fixer:: ws_comma |
| 339 | |
| 340 | Removes excess whitespace from comma separated items. This fixer is |
| 341 | optional. |
| 342 | |
| 343 | .. 2to3fixer:: xrange |
| 344 | |
| 345 | Renames :func:`xrange` to :func:`range` and wraps existing :func:`range` |
| 346 | calls with :class:`list`. |
| 347 | |
| 348 | .. 2to3fixer:: xreadlines |
| 349 | |
| 350 | Changes ``for x in file.xreadlines()`` to ``for x in file``. |
| 351 | |
| 352 | .. 2to3fixer:: zip |
| 353 | |
| 354 | Wraps :func:`zip` usage in a :class:`list` call. This is disabled when |
| 355 | ``from future_builtins import zip`` appears. |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 356 | |
| 357 | |
| 358 | :mod:`lib2to3` - 2to3's library |
| 359 | ------------------------------- |
| 360 | |
| 361 | .. module:: lib2to3 |
| 362 | :synopsis: the 2to3 library |
| 363 | .. moduleauthor:: Guido van Rossum |
| 364 | .. moduleauthor:: Collin Winter |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 365 | .. moduleauthor:: Benjamin Peterson <benjamin@python.org> |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 366 | |
Benjamin Peterson | e9bbc8b | 2008-09-28 02:06:32 +0000 | [diff] [blame] | 367 | |
Georg Brandl | e720c0a | 2009-04-27 16:20:50 +0000 | [diff] [blame] | 368 | .. note:: |
Benjamin Peterson | e9bbc8b | 2008-09-28 02:06:32 +0000 | [diff] [blame] | 369 | |
| 370 | The :mod:`lib2to3` API should be considered unstable and may change |
| 371 | drastically in the future. |
| 372 | |
Benjamin Peterson | d631371 | 2008-07-31 16:23:04 +0000 | [diff] [blame] | 373 | .. XXX What is the public interface anyway? |