| .. _pyporting-howto: |
| |
| ********************************* |
| Porting Python 2 Code to Python 3 |
| ********************************* |
| |
| :author: Brett Cannon |
| |
| .. topic:: Abstract |
| |
| With Python 3 being the future of Python while Python 2 is still in active |
| use, it is good to have your project available for both major releases of |
| Python. This guide is meant to help you choose which strategy works best |
| for your project to support both Python 2 & 3 along with how to execute |
| that strategy. |
| |
| If you are looking to port an extension module instead of pure Python code, |
| please see :ref:`cporting-howto`. |
| |
| |
| Choosing a Strategy |
| =================== |
| |
| When a project chooses to support both Python 2 & 3, |
| a decision needs to be made as to how to go about accomplishing that goal. |
| The chosen strategy will depend on how large the project's existing |
| codebase is and how much divergence you want from your current Python 2 codebase |
| (e.g., changing your code to work simultaneously with Python 2 and 3). |
| |
| If you would prefer to maintain a codebase which is semantically **and** |
| syntactically compatible with Python 2 & 3 simultaneously, you can write |
| :ref:`use_same_source`. While this tends to lead to somewhat non-idiomatic |
| code, it does mean you keep a rapid development process for you, the developer. |
| |
| If your project is brand-new or does not have a large codebase, then you may |
| want to consider writing/porting :ref:`all of your code for Python 3 |
| and use 3to2 <use_3to2>` to port your code for Python 2. |
| |
| Finally, you do have the option of :ref:`using 2to3 <use_2to3>` to translate |
| Python 2 code into Python 3 code (with some manual help). This can take the |
| form of branching your code and using 2to3 to start a Python 3 branch. You can |
| also have users perform the translation at installation time automatically so |
| that you only have to maintain a Python 2 codebase. |
| |
| Regardless of which approach you choose, porting is not as hard or |
| time-consuming as you might initially think. You can also tackle the problem |
| piece-meal as a good portion of porting is simply updating your code to follow |
| current best practices in a Python 2/3 compatible way. |
| |
| |
| Universal Bits of Advice |
| ------------------------ |
| |
| Regardless of what strategy you pick, there are a few things you should |
| consider. |
| |
| One is make sure you have a robust test suite. You need to make sure everything |
| continues to work, just like when you support a new minor/feature release of |
| Python. This means making sure your test suite is thorough and is ported |
| properly between Python 2 & 3. You will also most likely want to use something |
| like tox_ to automate testing between both a Python 2 and Python 3 interpreter. |
| |
| Two, once your project has Python 3 support, make sure to add the proper |
| classifier on the Cheeseshop_ (PyPI_). To have your project listed as Python 3 |
| compatible it must have the |
| `Python 3 classifier <http://pypi.python.org/pypi?:action=browse&c=533>`_ |
| (from |
| http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/):: |
| |
| setup( |
| name='Your Library', |
| version='1.0', |
| classifiers=[ |
| # make sure to use :: Python *and* :: Python :: 3 so |
| # that pypi can list the package on the python 3 page |
| 'Programming Language :: Python', |
| 'Programming Language :: Python :: 3' |
| ], |
| packages=['yourlibrary'], |
| # make sure to add custom_fixers to the MANIFEST.in |
| include_package_data=True, |
| # ... |
| ) |
| |
| |
| Doing so will cause your project to show up in the |
| `Python 3 packages list |
| <http://pypi.python.org/pypi?:action=browse&c=533&show=all>`_. You will know |
| you set the classifier properly as visiting your project page on the Cheeseshop |
| will show a Python 3 logo in the upper-left corner of the page. |
| |
| Three, the six_ project provides a library which helps iron out differences |
| between Python 2 & 3. If you find there is a sticky point that is a continual |
| point of contention in your translation or maintenance of code, consider using |
| a source-compatible solution relying on six. If you have to create your own |
| Python 2/3 compatible solution, you can use ``sys.version_info[0] >= 3`` as a |
| guard. |
| |
| Four, read all the approaches. Just because some bit of advice applies to one |
| approach more than another doesn't mean that some advice doesn't apply to other |
| strategies. This is especially true of whether you decide to use 2to3 or be |
| source-compatible; tips for one approach almost always apply to the other. |
| |
| Five, drop support for older Python versions if possible. `Python 2.5`_ |
| introduced a lot of useful syntax and libraries which have become idiomatic |
| in Python 3. `Python 2.6`_ introduced future statements which makes |
| compatibility much easier if you are going from Python 2 to 3. |
| `Python 2.7`_ continues the trend in the stdlib. So choose the newest version |
| of Python which you believe can be your minimum support version |
| and work from there. |
| |
| Six, target the newest version of Python 3 that you can. Beyond just the usual |
| bugfixes, compatibility has continued to improve between Python 2 and 3 as time |
| has passed. This is especially true for Python 3.3 where the ``u`` prefix for |
| strings is allowed, making source-compatible Python code easier. |
| |
| Seven, make sure to look at the `Other Resources`_ for tips from other people |
| which may help you out. |
| |
| |
| .. _tox: http://codespeak.net/tox/ |
| .. _Cheeseshop: |
| .. _PyPI: http://pypi.python.org/ |
| .. _six: http://packages.python.org/six |
| .. _Python 2.7: http://www.python.org/2.7.x |
| .. _Python 2.6: http://www.python.org/2.6.x |
| .. _Python 2.5: http://www.python.org/2.5.x |
| .. _Python 2.4: http://www.python.org/2.4.x |
| .. _Python 2.3: http://www.python.org/2.3.x |
| .. _Python 2.2: http://www.python.org/2.2.x |
| |
| |
| .. _use_3to2: |
| |
| Python 3 and 3to2 |
| ================= |
| |
| If you are starting a new project or your codebase is small enough, you may |
| want to consider writing your code for Python 3 and backporting to Python 2 |
| using 3to2_. Thanks to Python 3 being more strict about things than Python 2 |
| (e.g., bytes vs. strings), the source translation can be easier and more |
| straightforward than from Python 2 to 3. Plus it gives you more direct |
| experience developing in Python 3 which, since it is the future of Python, is a |
| good thing long-term. |
| |
| A drawback of this approach is that 3to2 is a third-party project. This means |
| that the Python core developers (and thus this guide) can make no promises |
| about how well 3to2 works at any time. There is nothing to suggest, though, |
| that 3to2 is not a high-quality project. |
| |
| |
| .. _3to2: https://bitbucket.org/amentajo/lib3to2/overview |
| |
| |
| .. _use_2to3: |
| |
| Python 2 and 2to3 |
| ================= |
| |
| Included with Python since 2.6, the 2to3_ tool (and :mod:`lib2to3` module) |
| helps with porting Python 2 to Python 3 by performing various source |
| translations. This is a perfect solution for projects which wish to branch |
| their Python 3 code from their Python 2 codebase and maintain them as |
| independent codebases. You can even begin preparing to use this approach |
| today by writing future-compatible Python code which works cleanly in |
| Python 2 in conjunction with 2to3; all steps outlined below will work |
| with Python 2 code up to the point when the actual use of 2to3 occurs. |
| |
| Use of 2to3 as an on-demand translation step at install time is also possible, |
| preventing the need to maintain a separate Python 3 codebase, but this approach |
| does come with some drawbacks. While users will only have to pay the |
| translation cost once at installation, you as a developer will need to pay the |
| cost regularly during development. If your codebase is sufficiently large |
| enough then the translation step ends up acting like a compilation step, |
| robbing you of the rapid development process you are used to with Python. |
| Obviously the time required to translate a project will vary, so do an |
| experimental translation just to see how long it takes to evaluate whether you |
| prefer this approach compared to using :ref:`use_same_source` or simply keeping |
| a separate Python 3 codebase. |
| |
| Below are the typical steps taken by a project which tries to support |
| Python 2 & 3 while keeping the code directly executable by Python 2. |
| |
| |
| Support Python 2.7 |
| ------------------ |
| |
| As a first step, make sure that your project is compatible with `Python 2.7`_. |
| This is just good to do as Python 2.7 is the last release of Python 2 and thus |
| will be used for a rather long time. It also allows for use of the ``-3`` flag |
| to Python to help discover places in your code which 2to3 cannot handle but are |
| known to cause issues. |
| |
| Try to Support `Python 2.6`_ and Newer Only |
| ------------------------------------------- |
| |
| While not possible for all projects, if you can support `Python 2.6`_ and newer |
| **only**, your life will be much easier. Various future statements, stdlib |
| additions, etc. exist only in Python 2.6 and later which greatly assist in |
| porting to Python 3. But if you project must keep support for `Python 2.5`_ (or |
| even `Python 2.4`_) then it is still possible to port to Python 3. |
| |
| Below are the benefits you gain if you only have to support Python 2.6 and |
| newer. Some of these options are personal choice while others are |
| **strongly** recommended (the ones that are more for personal choice are |
| labeled as such). If you continue to support older versions of Python then you |
| at least need to watch out for situations that these solutions fix. |
| |
| |
| ``from __future__ import print_function`` |
| ''''''''''''''''''''''''''''''''''''''''' |
| |
| This is a personal choice. 2to3 handles the translation from the print |
| statement to the print function rather well so this is an optional step. This |
| future statement does help, though, with getting used to typing |
| ``print('Hello, World')`` instead of ``print 'Hello, World'``. |
| |
| |
| ``from __future__ import unicode_literals`` |
| ''''''''''''''''''''''''''''''''''''''''''' |
| |
| Another personal choice. You can always mark what you want to be a (unicode) |
| string with a ``u`` prefix to get the same effect. But regardless of whether |
| you use this future statement or not, you **must** make sure you know exactly |
| which Python 2 strings you want to be bytes, and which are to be strings. This |
| means you should, **at minimum** mark all strings that are meant to be text |
| strings with a ``u`` prefix if you do not use this future statement. Python 3.3 |
| allows strings to continue to have the ``u`` prefix (it's a no-op in that case) |
| to make it easier for code to be source-compatible between Python 2 & 3. |
| |
| |
| Bytes literals |
| '''''''''''''' |
| |
| This is a **very** important one. The ability to prefix Python 2 strings that |
| are meant to contain bytes with a ``b`` prefix help to very clearly delineate |
| what is and is not a Python 3 string. When you run 2to3 on code, all Python 2 |
| strings become Python 3 strings **unless** they are prefixed with ``b``. |
| |
| This point cannot be stressed enough: make sure you know what all of your string |
| literals in Python 2 are meant to become in Python 3. Any string literal that |
| should be treated as bytes should have the ``b`` prefix. Any string literal |
| that should be Unicode/text in Python 2 should either have the ``u`` literal |
| (supported, but ignored, in Python 3.3 and later) or you should have |
| ``from __future__ import unicode_literals`` at the top of the file. But the key |
| point is you should know how Python 3 will treat everyone one of your string |
| literals and you should mark them as appropriate. |
| |
| There are some differences between byte literals in Python 2 and those in |
| Python 3 thanks to the bytes type just being an alias to ``str`` in Python 2. |
| Probably the biggest "gotcha" is that indexing results in different values. In |
| Python 2, the value of ``b'py'[1]`` is ``'y'``, while in Python 3 it's ``121``. |
| You can avoid this disparity by always slicing at the size of a single element: |
| ``b'py'[1:2]`` is ``'y'`` in Python 2 and ``b'y'`` in Python 3 (i.e., close |
| enough). |
| |
| You cannot concatenate bytes and strings in Python 3. But since Python |
| 2 has bytes aliased to ``str``, it will succeed: ``b'a' + u'b'`` works in |
| Python 2, but ``b'a' + 'b'`` in Python 3 is a :exc:`TypeError`. A similar issue |
| also comes about when doing comparisons between bytes and strings. |
| |
| |
| Supporting `Python 2.5`_ and Newer Only |
| --------------------------------------- |
| |
| If you are supporting `Python 2.5`_ and newer there are still some features of |
| Python that you can utilize. |
| |
| |
| ``from __future__ import absolute_import`` |
| '''''''''''''''''''''''''''''''''''''''''' |
| |
| Implicit relative imports (e.g., importing ``spam.bacon`` from within |
| ``spam.eggs`` with the statement ``import bacon``) does not work in Python 3. |
| This future statement moves away from that and allows the use of explicit |
| relative imports (e.g., ``from . import bacon``). |
| |
| In `Python 2.5`_ you must use |
| the __future__ statement to get to use explicit relative imports and prevent |
| implicit ones. In `Python 2.6`_ explicit relative imports are available without |
| the statement, but you still want the __future__ statement to prevent implicit |
| relative imports. In `Python 2.7`_ the __future__ statement is not needed. In |
| other words, unless you are only supporting Python 2.7 or a version earlier |
| than Python 2.5, use the __future__ statement. |
| |
| |
| Mark all Unicode strings with a ``u`` prefix |
| ''''''''''''''''''''''''''''''''''''''''''''' |
| |
| While Python 2.6 has a ``__future__`` statement to automatically cause Python 2 |
| to treat all string literals as Unicode, Python 2.5 does not have that shortcut. |
| This means you should go through and mark all string literals with a ``u`` |
| prefix to turn them explicitly into Unicode strings where appropriate. That |
| leaves all unmarked string literals to be considered byte literals in Python 3. |
| |
| |
| |
| Handle Common "Gotchas" |
| ----------------------- |
| |
| There are a few things that just consistently come up as sticking points for |
| people which 2to3 cannot handle automatically or can easily be done in Python 2 |
| to help modernize your code. |
| |
| |
| ``from __future__ import division`` |
| ''''''''''''''''''''''''''''''''''' |
| |
| While the exact same outcome can be had by using the ``-Qnew`` argument to |
| Python, using this future statement lifts the requirement that your users use |
| the flag to get the expected behavior of division in Python 3 |
| (e.g., ``1/2 == 0.5; 1//2 == 0``). |
| |
| |
| |
| Specify when opening a file as binary |
| ''''''''''''''''''''''''''''''''''''' |
| |
| Unless you have been working on Windows, there is a chance you have not always |
| bothered to add the ``b`` mode when opening a binary file (e.g., ``rb`` for |
| binary reading). Under Python 3, binary files and text files are clearly |
| distinct and mutually incompatible; see the :mod:`io` module for details. |
| Therefore, you **must** make a decision of whether a file will be used for |
| binary access (allowing to read and/or write bytes data) or text access |
| (allowing to read and/or write unicode data). |
| |
| Text files |
| '''''''''' |
| |
| Text files created using ``open()`` under Python 2 return byte strings, |
| while under Python 3 they return unicode strings. Depending on your porting |
| strategy, this can be an issue. |
| |
| If you want text files to return unicode strings in Python 2, you have two |
| possibilities: |
| |
| * Under Python 2.6 and higher, use :func:`io.open`. Since :func:`io.open` |
| is essentially the same function in both Python 2 and Python 3, it will |
| help iron out any issues that might arise. |
| |
| * If pre-2.6 compatibility is needed, then you should use :func:`codecs.open` |
| instead. This will make sure that you get back unicode strings in Python 2. |
| |
| Subclass ``object`` |
| ''''''''''''''''''' |
| |
| New-style classes have been around since `Python 2.2`_. You need to make sure |
| you are subclassing from ``object`` to avoid odd edge cases involving method |
| resolution order, etc. This continues to be totally valid in Python 3 (although |
| unneeded as all classes implicitly inherit from ``object``). |
| |
| |
| Deal With the Bytes/String Dichotomy |
| '''''''''''''''''''''''''''''''''''' |
| |
| One of the biggest issues people have when porting code to Python 3 is handling |
| the bytes/string dichotomy. Because Python 2 allowed the ``str`` type to hold |
| textual data, people have over the years been rather loose in their delineation |
| of what ``str`` instances held text compared to bytes. In Python 3 you cannot |
| be so care-free anymore and need to properly handle the difference. The key |
| handling this issue is to make sure that **every** string literal in your |
| Python 2 code is either syntactically of functionally marked as either bytes or |
| text data. After this is done you then need to make sure your APIs are designed |
| to either handle a specific type or made to be properly polymorphic. |
| |
| |
| Mark Up Python 2 String Literals |
| ******************************** |
| |
| First thing you must do is designate every single string literal in Python 2 |
| as either textual or bytes data. If you are only supporting Python 2.6 or |
| newer, this can be accomplished by marking bytes literals with a ``b`` prefix |
| and then designating textual data with a ``u`` prefix or using the |
| ``unicode_literals`` future statement. |
| |
| If your project supports versions of Python predating 2.6, then you should use |
| the six_ project and its ``b()`` function to denote bytes literals. For text |
| literals you can either use six's ``u()`` function or use a ``u`` prefix. |
| |
| |
| Decide what APIs Will Accept |
| **************************** |
| |
| In Python 2 it was very easy to accidentally create an API that accepted both |
| bytes and textual data. But in Python 3, thanks to the more strict handling of |
| disparate types, this loose usage of bytes and text together tends to fail. |
| |
| Take the dict ``{b'a': 'bytes', u'a': 'text'}`` in Python 2.6. It creates the |
| dict ``{u'a': 'text'}`` since ``b'a' == u'a'``. But in Python 3 the equivalent |
| dict creates ``{b'a': 'bytes', 'a': 'text'}``, i.e., no lost data. Similar |
| issues can crop up when transitioning Python 2 code to Python 3. |
| |
| This means you need to choose what an API is going to accept and create and |
| consistently stick to that API in both Python 2 and 3. |
| |
| |
| Bytes / Unicode Comparison |
| ************************** |
| |
| In Python 3, mixing bytes and unicode is forbidden in most situations; it |
| will raise a :class:`TypeError` where Python 2 would have attempted an implicit |
| coercion between types. However, there is one case where it doesn't and |
| it can be very misleading:: |
| |
| >>> b"" == "" |
| False |
| |
| This is because an equality comparison is required by the language to always |
| succeed (and return ``False`` for incompatible types). However, this also |
| means that code incorrectly ported to Python 3 can display buggy behaviour |
| if such comparisons are silently executed. To detect such situations, |
| Python 3 has a ``-b`` flag that will display a warning:: |
| |
| $ python3 -b |
| >>> b"" == "" |
| __main__:1: BytesWarning: Comparison between bytes and string |
| False |
| |
| To turn the warning into an exception, use the ``-bb`` flag instead:: |
| |
| $ python3 -bb |
| >>> b"" == "" |
| Traceback (most recent call last): |
| File "<stdin>", line 1, in <module> |
| BytesWarning: Comparison between bytes and string |
| |
| |
| Indexing bytes objects |
| '''''''''''''''''''''' |
| |
| Another potentially surprising change is the indexing behaviour of bytes |
| objects in Python 3:: |
| |
| >>> b"xyz"[0] |
| 120 |
| |
| Indeed, Python 3 bytes objects (as well as :class:`bytearray` objects) |
| are sequences of integers. But code converted from Python 2 will often |
| assume that indexing a bytestring produces another bytestring, not an |
| integer. To reconcile both behaviours, use slicing:: |
| |
| >>> b"xyz"[0:1] |
| b'x' |
| >>> n = 1 |
| >>> b"xyz"[n:n+1] |
| b'y' |
| |
| The only remaining gotcha is that an out-of-bounds slice returns an empty |
| bytes object instead of raising ``IndexError``: |
| |
| >>> b"xyz"[3] |
| Traceback (most recent call last): |
| File "<stdin>", line 1, in <module> |
| IndexError: index out of range |
| >>> b"xyz"[3:4] |
| b'' |
| |
| |
| ``__str__()``/``__unicode__()`` |
| ''''''''''''''''''''''''''''''' |
| |
| In Python 2, objects can specify both a string and unicode representation of |
| themselves. In Python 3, though, there is only a string representation. This |
| becomes an issue as people can inadvertently do things in their ``__str__()`` |
| methods which have unpredictable results (e.g., infinite recursion if you |
| happen to use the ``unicode(self).encode('utf8')`` idiom as the body of your |
| ``__str__()`` method). |
| |
| There are two ways to solve this issue. One is to use a custom 2to3 fixer. The |
| blog post at http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ |
| specifies how to do this. That will allow 2to3 to change all instances of ``def |
| __unicode(self): ...`` to ``def __str__(self): ...``. This does require that you |
| define your ``__str__()`` method in Python 2 before your ``__unicode__()`` |
| method. |
| |
| The other option is to use a mixin class. This allows you to only define a |
| ``__unicode__()`` method for your class and let the mixin derive |
| ``__str__()`` for you (code from |
| http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/):: |
| |
| import sys |
| |
| class UnicodeMixin(object): |
| |
| """Mixin class to handle defining the proper __str__/__unicode__ |
| methods in Python 2 or 3.""" |
| |
| if sys.version_info[0] >= 3: # Python 3 |
| def __str__(self): |
| return self.__unicode__() |
| else: # Python 2 |
| def __str__(self): |
| return self.__unicode__().encode('utf8') |
| |
| |
| class Spam(UnicodeMixin): |
| |
| def __unicode__(self): |
| return u'spam-spam-bacon-spam' # 2to3 will remove the 'u' prefix |
| |
| |
| Don't Index on Exceptions |
| ''''''''''''''''''''''''' |
| |
| In Python 2, the following worked:: |
| |
| >>> exc = Exception(1, 2, 3) |
| >>> exc.args[1] |
| 2 |
| >>> exc[1] # Python 2 only! |
| 2 |
| |
| But in Python 3, indexing directly on an exception is an error. You need to |
| make sure to only index on the :attr:`BaseException.args` attribute which is a |
| sequence containing all arguments passed to the :meth:`__init__` method. |
| |
| Even better is to use the documented attributes the exception provides. |
| |
| Don't use ``__getslice__`` & Friends |
| '''''''''''''''''''''''''''''''''''' |
| |
| Been deprecated for a while, but Python 3 finally drops support for |
| ``__getslice__()``, etc. Move completely over to :meth:`__getitem__` and |
| friends. |
| |
| |
| Updating doctests |
| ''''''''''''''''' |
| |
| 2to3_ will attempt to generate fixes for doctests that it comes across. It's |
| not perfect, though. If you wrote a monolithic set of doctests (e.g., a single |
| docstring containing all of your doctests), you should at least consider |
| breaking the doctests up into smaller pieces to make it more manageable to fix. |
| Otherwise it might very well be worth your time and effort to port your tests |
| to :mod:`unittest`. |
| |
| |
| Update `map` for imbalanced input sequences |
| ''''''''''''''''''''''''''''''''''''''''''' |
| |
| With Python 2, `map` would pad input sequences of unequal length with |
| `None` values, returning a sequence as long as the longest input sequence. |
| |
| With Python 3, if the input sequences to `map` are of unequal length, `map` |
| will stop at the termination of the shortest of the sequences. For full |
| compatibility with `map` from Python 2.x, also wrap the sequences in |
| :func:`itertools.zip_longest`, e.g. ``map(func, *sequences)`` becomes |
| ``list(map(func, itertools.zip_longest(*sequences)))``. |
| |
| Eliminate ``-3`` Warnings |
| ------------------------- |
| |
| When you run your application's test suite, run it using the ``-3`` flag passed |
| to Python. This will cause various warnings to be raised during execution about |
| things that 2to3 cannot handle automatically (e.g., modules that have been |
| removed). Try to eliminate those warnings to make your code even more portable |
| to Python 3. |
| |
| |
| Run 2to3 |
| -------- |
| |
| Once you have made your Python 2 code future-compatible with Python 3, it's |
| time to use 2to3_ to actually port your code. |
| |
| |
| Manually |
| '''''''' |
| |
| To manually convert source code using 2to3_, you use the ``2to3`` script that |
| is installed with Python 2.6 and later.:: |
| |
| 2to3 <directory or file to convert> |
| |
| This will cause 2to3 to write out a diff with all of the fixers applied for the |
| converted source code. If you would like 2to3 to go ahead and apply the changes |
| you can pass it the ``-w`` flag:: |
| |
| 2to3 -w <stuff to convert> |
| |
| There are other flags available to control exactly which fixers are applied, |
| etc. |
| |
| |
| During Installation |
| ''''''''''''''''''' |
| |
| When a user installs your project for Python 3, you can have either |
| :mod:`distutils` or Distribute_ run 2to3_ on your behalf. |
| For distutils, use the following idiom:: |
| |
| try: # Python 3 |
| from distutils.command.build_py import build_py_2to3 as build_py |
| except ImportError: # Python 2 |
| from distutils.command.build_py import build_py |
| |
| setup(cmdclass = {'build_py': build_py}, |
| # ... |
| ) |
| |
| For Distribute:: |
| |
| setup(use_2to3=True, |
| # ... |
| ) |
| |
| This will allow you to not have to distribute a separate Python 3 version of |
| your project. It does require, though, that when you perform development that |
| you at least build your project and use the built Python 3 source for testing. |
| |
| |
| Verify & Test |
| ------------- |
| |
| At this point you should (hopefully) have your project converted in such a way |
| that it works in Python 3. Verify it by running your unit tests and making sure |
| nothing has gone awry. If you miss something then figure out how to fix it in |
| Python 3, backport to your Python 2 code, and run your code through 2to3 again |
| to verify the fix transforms properly. |
| |
| |
| .. _2to3: http://docs.python.org/py3k/library/2to3.html |
| .. _Distribute: http://packages.python.org/distribute/ |
| |
| |
| .. _use_same_source: |
| |
| Python 2/3 Compatible Source |
| ============================ |
| |
| While it may seem counter-intuitive, you can write Python code which is |
| source-compatible between Python 2 & 3. It does lead to code that is not |
| entirely idiomatic Python (e.g., having to extract the currently raised |
| exception from ``sys.exc_info()[1]``), but it can be run under Python 2 |
| **and** Python 3 without using 2to3_ as a translation step (although the tool |
| should be used to help find potential portability problems). This allows you to |
| continue to have a rapid development process regardless of whether you are |
| developing under Python 2 or Python 3. Whether this approach or using |
| :ref:`use_2to3` works best for you will be a per-project decision. |
| |
| To get a complete idea of what issues you will need to deal with, see the |
| `What's New in Python 3.0`_. Others have reorganized the data in other formats |
| such as http://docs.pythonsprints.com/python3_porting/py-porting.html . |
| |
| The following are some steps to take to try to support both Python 2 & 3 from |
| the same source code. |
| |
| |
| .. _What's New in Python 3.0: http://docs.python.org/release/3.0/whatsnew/3.0.html |
| |
| |
| Follow The Steps for Using 2to3_ |
| -------------------------------- |
| |
| All of the steps outlined in how to |
| :ref:`port Python 2 code with 2to3 <use_2to3>` apply |
| to creating a Python 2/3 codebase. This includes trying only support Python 2.6 |
| or newer (the :mod:`__future__` statements work in Python 3 without issue), |
| eliminating warnings that are triggered by ``-3``, etc. |
| |
| You should even consider running 2to3_ over your code (without committing the |
| changes). This will let you know where potential pain points are within your |
| code so that you can fix them properly before they become an issue. |
| |
| |
| Use six_ |
| -------- |
| |
| The six_ project contains many things to help you write portable Python code. |
| You should make sure to read its documentation from beginning to end and use |
| any and all features it provides. That way you will minimize any mistakes you |
| might make in writing cross-version code. |
| |
| |
| Capturing the Currently Raised Exception |
| ---------------------------------------- |
| |
| One change between Python 2 and 3 that will require changing how you code (if |
| you support `Python 2.5`_ and earlier) is |
| accessing the currently raised exception. In Python 2.5 and earlier the syntax |
| to access the current exception is:: |
| |
| try: |
| raise Exception() |
| except Exception, exc: |
| # Current exception is 'exc' |
| pass |
| |
| This syntax changed in Python 3 (and backported to `Python 2.6`_ and later) |
| to:: |
| |
| try: |
| raise Exception() |
| except Exception as exc: |
| # Current exception is 'exc' |
| # In Python 3, 'exc' is restricted to the block; Python 2.6 will "leak" |
| pass |
| |
| Because of this syntax change you must change to capturing the current |
| exception to:: |
| |
| try: |
| raise Exception() |
| except Exception: |
| import sys |
| exc = sys.exc_info()[1] |
| # Current exception is 'exc' |
| pass |
| |
| You can get more information about the raised exception from |
| :func:`sys.exc_info` than simply the current exception instance, but you most |
| likely don't need it. |
| |
| .. note:: |
| In Python 3, the traceback is attached to the exception instance |
| through the ``__traceback__`` attribute. If the instance is saved in |
| a local variable that persists outside of the ``except`` block, the |
| traceback will create a reference cycle with the current frame and its |
| dictionary of local variables. This will delay reclaiming dead |
| resources until the next cyclic :term:`garbage collection` pass. |
| |
| In Python 2, this problem only occurs if you save the traceback itself |
| (e.g. the third element of the tuple returned by :func:`sys.exc_info`) |
| in a variable. |
| |
| |
| Other Resources |
| =============== |
| |
| The authors of the following blog posts, wiki pages, and books deserve special |
| thanks for making public their tips for porting Python 2 code to Python 3 (and |
| thus helping provide information for this document): |
| |
| * http://python3porting.com/ |
| * http://docs.pythonsprints.com/python3_porting/py-porting.html |
| * http://techspot.zzzeek.org/2011/01/24/zzzeek-s-guide-to-python-3-porting/ |
| * http://dabeaz.blogspot.com/2011/01/porting-py65-and-my-superboard-to.html |
| * http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/ |
| * http://lucumr.pocoo.org/2010/2/11/porting-to-python-3-a-guide/ |
| * http://wiki.python.org/moin/PortingPythonToPy3k |
| * https://wiki.ubuntu.com/Python/3 |
| |
| If you feel there is something missing from this document that should be added, |
| please email the python-porting_ mailing list. |
| |
| .. _python-porting: http://mail.python.org/mailman/listinfo/python-porting |