Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1 | :tocdepth: 2 |
| 2 | |
| 3 | =============== |
| 4 | Programming FAQ |
| 5 | =============== |
| 6 | |
| 7 | .. contents:: |
| 8 | |
| 9 | General Questions |
| 10 | ================= |
| 11 | |
| 12 | Is there a source code level debugger with breakpoints, single-stepping, etc.? |
| 13 | ------------------------------------------------------------------------------ |
| 14 | |
| 15 | Yes. |
| 16 | |
| 17 | The pdb module is a simple but adequate console-mode debugger for Python. It is |
| 18 | part of the standard Python library, and is :mod:`documented in the Library |
| 19 | Reference Manual <pdb>`. You can also write your own debugger by using the code |
| 20 | for pdb as an example. |
| 21 | |
| 22 | The IDLE interactive development environment, which is part of the standard |
| 23 | Python distribution (normally available as Tools/scripts/idle), includes a |
| 24 | graphical debugger. There is documentation for the IDLE debugger at |
| 25 | http://www.python.org/idle/doc/idle2.html#Debugger. |
| 26 | |
| 27 | PythonWin is a Python IDE that includes a GUI debugger based on pdb. The |
| 28 | Pythonwin debugger colors breakpoints and has quite a few cool features such as |
| 29 | debugging non-Pythonwin programs. Pythonwin is available as part of the `Python |
| 30 | for Windows Extensions <http://sourceforge.net/projects/pywin32/>`__ project and |
| 31 | as a part of the ActivePython distribution (see |
| 32 | http://www.activestate.com/Products/ActivePython/index.html). |
| 33 | |
| 34 | `Boa Constructor <http://boa-constructor.sourceforge.net/>`_ is an IDE and GUI |
| 35 | builder that uses wxWidgets. It offers visual frame creation and manipulation, |
| 36 | an object inspector, many views on the source like object browsers, inheritance |
| 37 | hierarchies, doc string generated html documentation, an advanced debugger, |
| 38 | integrated help, and Zope support. |
| 39 | |
| 40 | `Eric <http://www.die-offenbachs.de/eric/index.html>`_ is an IDE built on PyQt |
| 41 | and the Scintilla editing component. |
| 42 | |
| 43 | Pydb is a version of the standard Python debugger pdb, modified for use with DDD |
| 44 | (Data Display Debugger), a popular graphical debugger front end. Pydb can be |
| 45 | found at http://bashdb.sourceforge.net/pydb/ and DDD can be found at |
| 46 | http://www.gnu.org/software/ddd. |
| 47 | |
| 48 | There are a number of commercial Python IDEs that include graphical debuggers. |
| 49 | They include: |
| 50 | |
| 51 | * Wing IDE (http://wingware.com/) |
| 52 | * Komodo IDE (http://www.activestate.com/Products/Komodo) |
| 53 | |
| 54 | |
| 55 | Is there a tool to help find bugs or perform static analysis? |
| 56 | ------------------------------------------------------------- |
| 57 | |
| 58 | Yes. |
| 59 | |
| 60 | PyChecker is a static analysis tool that finds bugs in Python source code and |
| 61 | warns about code complexity and style. You can get PyChecker from |
| 62 | http://pychecker.sf.net. |
| 63 | |
| 64 | `Pylint <http://www.logilab.org/projects/pylint>`_ is another tool that checks |
| 65 | if a module satisfies a coding standard, and also makes it possible to write |
| 66 | plug-ins to add a custom feature. In addition to the bug checking that |
| 67 | PyChecker performs, Pylint offers some additional features such as checking line |
| 68 | length, whether variable names are well-formed according to your coding |
| 69 | standard, whether declared interfaces are fully implemented, and more. |
Georg Brandl | 495f7b5 | 2009-10-27 15:28:25 +0000 | [diff] [blame] | 70 | http://www.logilab.org/card/pylint_manual provides a full list of Pylint's |
| 71 | features. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 72 | |
| 73 | |
| 74 | How can I create a stand-alone binary from a Python script? |
| 75 | ----------------------------------------------------------- |
| 76 | |
| 77 | You don't need the ability to compile Python to C code if all you want is a |
| 78 | stand-alone program that users can download and run without having to install |
| 79 | the Python distribution first. There are a number of tools that determine the |
| 80 | set of modules required by a program and bind these modules together with a |
| 81 | Python binary to produce a single executable. |
| 82 | |
| 83 | One is to use the freeze tool, which is included in the Python source tree as |
| 84 | ``Tools/freeze``. It converts Python byte code to C arrays; a C compiler you can |
| 85 | embed all your modules into a new program, which is then linked with the |
| 86 | standard Python modules. |
| 87 | |
| 88 | It works by scanning your source recursively for import statements (in both |
| 89 | forms) and looking for the modules in the standard Python path as well as in the |
| 90 | source directory (for built-in modules). It then turns the bytecode for modules |
| 91 | written in Python into C code (array initializers that can be turned into code |
| 92 | objects using the marshal module) and creates a custom-made config file that |
| 93 | only contains those built-in modules which are actually used in the program. It |
| 94 | then compiles the generated C code and links it with the rest of the Python |
| 95 | interpreter to form a self-contained binary which acts exactly like your script. |
| 96 | |
| 97 | Obviously, freeze requires a C compiler. There are several other utilities |
| 98 | which don't. One is Thomas Heller's py2exe (Windows only) at |
| 99 | |
| 100 | http://www.py2exe.org/ |
| 101 | |
| 102 | Another is Christian Tismer's `SQFREEZE <http://starship.python.net/crew/pirx>`_ |
| 103 | which appends the byte code to a specially-prepared Python interpreter that can |
| 104 | find the byte code in the executable. |
| 105 | |
| 106 | Other tools include Fredrik Lundh's `Squeeze |
| 107 | <http://www.pythonware.com/products/python/squeeze>`_ and Anthony Tuininga's |
| 108 | `cx_Freeze <http://starship.python.net/crew/atuining/cx_Freeze/index.html>`_. |
| 109 | |
| 110 | |
| 111 | Are there coding standards or a style guide for Python programs? |
| 112 | ---------------------------------------------------------------- |
| 113 | |
| 114 | Yes. The coding style required for standard library modules is documented as |
| 115 | :pep:`8`. |
| 116 | |
| 117 | |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 118 | Core Language |
| 119 | ============= |
| 120 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 121 | Why am I getting an UnboundLocalError when the variable has a value? |
| 122 | -------------------------------------------------------------------- |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 123 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 124 | It can be a surprise to get the UnboundLocalError in previously working |
| 125 | code when it is modified by adding an assignment statement somewhere in |
| 126 | the body of a function. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 127 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 128 | This code: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 129 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 130 | >>> x = 10 |
| 131 | >>> def bar(): |
| 132 | ... print(x) |
| 133 | >>> bar() |
| 134 | 10 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 135 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 136 | works, but this code: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 137 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 138 | >>> x = 10 |
| 139 | >>> def foo(): |
| 140 | ... print(x) |
| 141 | ... x += 1 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 142 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 143 | results in an UnboundLocalError: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 144 | |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 145 | >>> foo() |
| 146 | Traceback (most recent call last): |
| 147 | ... |
| 148 | UnboundLocalError: local variable 'x' referenced before assignment |
| 149 | |
| 150 | This is because when you make an assignment to a variable in a scope, that |
| 151 | variable becomes local to that scope and shadows any similarly named variable |
| 152 | in the outer scope. Since the last statement in foo assigns a new value to |
| 153 | ``x``, the compiler recognizes it as a local variable. Consequently when the |
R. David Murray | 18163c3 | 2009-11-14 22:27:22 +0000 | [diff] [blame] | 154 | earlier ``print(x)`` attempts to print the uninitialized local variable and |
R. David Murray | c04a694 | 2009-11-14 22:21:32 +0000 | [diff] [blame] | 155 | an error results. |
| 156 | |
| 157 | In the example above you can access the outer scope variable by declaring it |
| 158 | global: |
| 159 | |
| 160 | >>> x = 10 |
| 161 | >>> def foobar(): |
| 162 | ... global x |
| 163 | ... print(x) |
| 164 | ... x += 1 |
| 165 | >>> foobar() |
| 166 | 10 |
| 167 | |
| 168 | This explicit declaration is required in order to remind you that (unlike the |
| 169 | superficially analogous situation with class and instance variables) you are |
| 170 | actually modifying the value of the variable in the outer scope: |
| 171 | |
| 172 | >>> print(x) |
| 173 | 11 |
| 174 | |
| 175 | You can do a similar thing in a nested scope using the :keyword:`nonlocal` |
| 176 | keyword: |
| 177 | |
| 178 | >>> def foo(): |
| 179 | ... x = 10 |
| 180 | ... def bar(): |
| 181 | ... nonlocal x |
| 182 | ... print(x) |
| 183 | ... x += 1 |
| 184 | ... bar() |
| 185 | ... print(x) |
| 186 | >>> foo() |
| 187 | 10 |
| 188 | 11 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 189 | |
| 190 | |
| 191 | What are the rules for local and global variables in Python? |
| 192 | ------------------------------------------------------------ |
| 193 | |
| 194 | In Python, variables that are only referenced inside a function are implicitly |
| 195 | global. If a variable is assigned a new value anywhere within the function's |
| 196 | body, it's assumed to be a local. If a variable is ever assigned a new value |
| 197 | inside the function, the variable is implicitly local, and you need to |
| 198 | explicitly declare it as 'global'. |
| 199 | |
| 200 | Though a bit surprising at first, a moment's consideration explains this. On |
| 201 | one hand, requiring :keyword:`global` for assigned variables provides a bar |
| 202 | against unintended side-effects. On the other hand, if ``global`` was required |
| 203 | for all global references, you'd be using ``global`` all the time. You'd have |
Georg Brandl | c4a55fc | 2010-02-06 18:46:57 +0000 | [diff] [blame] | 204 | to declare as global every reference to a built-in function or to a component of |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 205 | an imported module. This clutter would defeat the usefulness of the ``global`` |
| 206 | declaration for identifying side-effects. |
| 207 | |
| 208 | |
| 209 | How do I share global variables across modules? |
| 210 | ------------------------------------------------ |
| 211 | |
| 212 | The canonical way to share information across modules within a single program is |
| 213 | to create a special module (often called config or cfg). Just import the config |
| 214 | module in all modules of your application; the module then becomes available as |
| 215 | a global name. Because there is only one instance of each module, any changes |
| 216 | made to the module object get reflected everywhere. For example: |
| 217 | |
| 218 | config.py:: |
| 219 | |
| 220 | x = 0 # Default value of the 'x' configuration setting |
| 221 | |
| 222 | mod.py:: |
| 223 | |
| 224 | import config |
| 225 | config.x = 1 |
| 226 | |
| 227 | main.py:: |
| 228 | |
| 229 | import config |
| 230 | import mod |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 231 | print(config.x) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 232 | |
| 233 | Note that using a module is also the basis for implementing the Singleton design |
| 234 | pattern, for the same reason. |
| 235 | |
| 236 | |
| 237 | What are the "best practices" for using import in a module? |
| 238 | ----------------------------------------------------------- |
| 239 | |
| 240 | In general, don't use ``from modulename import *``. Doing so clutters the |
| 241 | importer's namespace. Some people avoid this idiom even with the few modules |
| 242 | that were designed to be imported in this manner. Modules designed in this |
Georg Brandl | d404fa6 | 2009-10-13 16:55:12 +0000 | [diff] [blame] | 243 | manner include :mod:`tkinter`, and :mod:`threading`. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 244 | |
| 245 | Import modules at the top of a file. Doing so makes it clear what other modules |
| 246 | your code requires and avoids questions of whether the module name is in scope. |
| 247 | Using one import per line makes it easy to add and delete module imports, but |
| 248 | using multiple imports per line uses less screen space. |
| 249 | |
| 250 | It's good practice if you import modules in the following order: |
| 251 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 252 | 1. standard library modules -- e.g. ``sys``, ``os``, ``getopt``, ``re`` |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 253 | 2. third-party library modules (anything installed in Python's site-packages |
| 254 | directory) -- e.g. mx.DateTime, ZODB, PIL.Image, etc. |
| 255 | 3. locally-developed modules |
| 256 | |
| 257 | Never use relative package imports. If you're writing code that's in the |
| 258 | ``package.sub.m1`` module and want to import ``package.sub.m2``, do not just |
Georg Brandl | 11b6362 | 2009-12-20 14:21:27 +0000 | [diff] [blame] | 259 | write ``from . import m2``, even though it's legal. Write ``from package.sub |
| 260 | import m2`` instead. See :pep:`328` for details. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 261 | |
| 262 | It is sometimes necessary to move imports to a function or class to avoid |
| 263 | problems with circular imports. Gordon McMillan says: |
| 264 | |
| 265 | Circular imports are fine where both modules use the "import <module>" form |
| 266 | of import. They fail when the 2nd module wants to grab a name out of the |
| 267 | first ("from module import name") and the import is at the top level. That's |
| 268 | because names in the 1st are not yet available, because the first module is |
| 269 | busy importing the 2nd. |
| 270 | |
| 271 | In this case, if the second module is only used in one function, then the import |
| 272 | can easily be moved into that function. By the time the import is called, the |
| 273 | first module will have finished initializing, and the second module can do its |
| 274 | import. |
| 275 | |
| 276 | It may also be necessary to move imports out of the top level of code if some of |
| 277 | the modules are platform-specific. In that case, it may not even be possible to |
| 278 | import all of the modules at the top of the file. In this case, importing the |
| 279 | correct modules in the corresponding platform-specific code is a good option. |
| 280 | |
| 281 | Only move imports into a local scope, such as inside a function definition, if |
| 282 | it's necessary to solve a problem such as avoiding a circular import or are |
| 283 | trying to reduce the initialization time of a module. This technique is |
| 284 | especially helpful if many of the imports are unnecessary depending on how the |
| 285 | program executes. You may also want to move imports into a function if the |
| 286 | modules are only ever used in that function. Note that loading a module the |
| 287 | first time may be expensive because of the one time initialization of the |
| 288 | module, but loading a module multiple times is virtually free, costing only a |
| 289 | couple of dictionary lookups. Even if the module name has gone out of scope, |
| 290 | the module is probably available in :data:`sys.modules`. |
| 291 | |
| 292 | If only instances of a specific class use a module, then it is reasonable to |
| 293 | import the module in the class's ``__init__`` method and then assign the module |
| 294 | to an instance variable so that the module is always available (via that |
| 295 | instance variable) during the life of the object. Note that to delay an import |
| 296 | until the class is instantiated, the import must be inside a method. Putting |
| 297 | the import inside the class but outside of any method still causes the import to |
| 298 | occur when the module is initialized. |
| 299 | |
| 300 | |
| 301 | How can I pass optional or keyword parameters from one function to another? |
| 302 | --------------------------------------------------------------------------- |
| 303 | |
| 304 | Collect the arguments using the ``*`` and ``**`` specifiers in the function's |
| 305 | parameter list; this gives you the positional arguments as a tuple and the |
| 306 | keyword arguments as a dictionary. You can then pass these arguments when |
| 307 | calling another function by using ``*`` and ``**``:: |
| 308 | |
| 309 | def f(x, *args, **kwargs): |
| 310 | ... |
| 311 | kwargs['width'] = '14.3c' |
| 312 | ... |
| 313 | g(x, *args, **kwargs) |
| 314 | |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 315 | |
| 316 | How do I write a function with output parameters (call by reference)? |
| 317 | --------------------------------------------------------------------- |
| 318 | |
| 319 | Remember that arguments are passed by assignment in Python. Since assignment |
| 320 | just creates references to objects, there's no alias between an argument name in |
| 321 | the caller and callee, and so no call-by-reference per se. You can achieve the |
| 322 | desired effect in a number of ways. |
| 323 | |
| 324 | 1) By returning a tuple of the results:: |
| 325 | |
| 326 | def func2(a, b): |
| 327 | a = 'new-value' # a and b are local names |
| 328 | b = b + 1 # assigned to new objects |
| 329 | return a, b # return new values |
| 330 | |
| 331 | x, y = 'old-value', 99 |
| 332 | x, y = func2(x, y) |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 333 | print(x, y) # output: new-value 100 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 334 | |
| 335 | This is almost always the clearest solution. |
| 336 | |
| 337 | 2) By using global variables. This isn't thread-safe, and is not recommended. |
| 338 | |
| 339 | 3) By passing a mutable (changeable in-place) object:: |
| 340 | |
| 341 | def func1(a): |
| 342 | a[0] = 'new-value' # 'a' references a mutable list |
| 343 | a[1] = a[1] + 1 # changes a shared object |
| 344 | |
| 345 | args = ['old-value', 99] |
| 346 | func1(args) |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 347 | print(args[0], args[1]) # output: new-value 100 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 348 | |
| 349 | 4) By passing in a dictionary that gets mutated:: |
| 350 | |
| 351 | def func3(args): |
| 352 | args['a'] = 'new-value' # args is a mutable dictionary |
| 353 | args['b'] = args['b'] + 1 # change it in-place |
| 354 | |
| 355 | args = {'a':' old-value', 'b': 99} |
| 356 | func3(args) |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 357 | print(args['a'], args['b']) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 358 | |
| 359 | 5) Or bundle up values in a class instance:: |
| 360 | |
| 361 | class callByRef: |
| 362 | def __init__(self, **args): |
| 363 | for (key, value) in args.items(): |
| 364 | setattr(self, key, value) |
| 365 | |
| 366 | def func4(args): |
| 367 | args.a = 'new-value' # args is a mutable callByRef |
| 368 | args.b = args.b + 1 # change object in-place |
| 369 | |
| 370 | args = callByRef(a='old-value', b=99) |
| 371 | func4(args) |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 372 | print(args.a, args.b) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 373 | |
| 374 | |
| 375 | There's almost never a good reason to get this complicated. |
| 376 | |
| 377 | Your best choice is to return a tuple containing the multiple results. |
| 378 | |
| 379 | |
| 380 | How do you make a higher order function in Python? |
| 381 | -------------------------------------------------- |
| 382 | |
| 383 | You have two choices: you can use nested scopes or you can use callable objects. |
| 384 | For example, suppose you wanted to define ``linear(a,b)`` which returns a |
| 385 | function ``f(x)`` that computes the value ``a*x+b``. Using nested scopes:: |
| 386 | |
| 387 | def linear(a, b): |
| 388 | def result(x): |
| 389 | return a * x + b |
| 390 | return result |
| 391 | |
| 392 | Or using a callable object:: |
| 393 | |
| 394 | class linear: |
| 395 | |
| 396 | def __init__(self, a, b): |
| 397 | self.a, self.b = a, b |
| 398 | |
| 399 | def __call__(self, x): |
| 400 | return self.a * x + self.b |
| 401 | |
| 402 | In both cases, :: |
| 403 | |
| 404 | taxes = linear(0.3, 2) |
| 405 | |
| 406 | gives a callable object where ``taxes(10e6) == 0.3 * 10e6 + 2``. |
| 407 | |
| 408 | The callable object approach has the disadvantage that it is a bit slower and |
| 409 | results in slightly longer code. However, note that a collection of callables |
| 410 | can share their signature via inheritance:: |
| 411 | |
| 412 | class exponential(linear): |
| 413 | # __init__ inherited |
| 414 | def __call__(self, x): |
| 415 | return self.a * (x ** self.b) |
| 416 | |
| 417 | Object can encapsulate state for several methods:: |
| 418 | |
| 419 | class counter: |
| 420 | |
| 421 | value = 0 |
| 422 | |
| 423 | def set(self, x): |
| 424 | self.value = x |
| 425 | |
| 426 | def up(self): |
| 427 | self.value = self.value + 1 |
| 428 | |
| 429 | def down(self): |
| 430 | self.value = self.value - 1 |
| 431 | |
| 432 | count = counter() |
| 433 | inc, dec, reset = count.up, count.down, count.set |
| 434 | |
| 435 | Here ``inc()``, ``dec()`` and ``reset()`` act like functions which share the |
| 436 | same counting variable. |
| 437 | |
| 438 | |
| 439 | How do I copy an object in Python? |
| 440 | ---------------------------------- |
| 441 | |
| 442 | In general, try :func:`copy.copy` or :func:`copy.deepcopy` for the general case. |
| 443 | Not all objects can be copied, but most can. |
| 444 | |
| 445 | Some objects can be copied more easily. Dictionaries have a :meth:`~dict.copy` |
| 446 | method:: |
| 447 | |
| 448 | newdict = olddict.copy() |
| 449 | |
| 450 | Sequences can be copied by slicing:: |
| 451 | |
| 452 | new_l = l[:] |
| 453 | |
| 454 | |
| 455 | How can I find the methods or attributes of an object? |
| 456 | ------------------------------------------------------ |
| 457 | |
| 458 | For an instance x of a user-defined class, ``dir(x)`` returns an alphabetized |
| 459 | list of the names containing the instance attributes and methods and attributes |
| 460 | defined by its class. |
| 461 | |
| 462 | |
| 463 | How can my code discover the name of an object? |
| 464 | ----------------------------------------------- |
| 465 | |
| 466 | Generally speaking, it can't, because objects don't really have names. |
| 467 | Essentially, assignment always binds a name to a value; The same is true of |
| 468 | ``def`` and ``class`` statements, but in that case the value is a |
| 469 | callable. Consider the following code:: |
| 470 | |
| 471 | class A: |
| 472 | pass |
| 473 | |
| 474 | B = A |
| 475 | |
| 476 | a = B() |
| 477 | b = a |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 478 | print(b) |
| 479 | <__main__.A object at 0x16D07CC> |
| 480 | print(a) |
| 481 | <__main__.A object at 0x16D07CC> |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 482 | |
| 483 | Arguably the class has a name: even though it is bound to two names and invoked |
| 484 | through the name B the created instance is still reported as an instance of |
| 485 | class A. However, it is impossible to say whether the instance's name is a or |
| 486 | b, since both names are bound to the same value. |
| 487 | |
| 488 | Generally speaking it should not be necessary for your code to "know the names" |
| 489 | of particular values. Unless you are deliberately writing introspective |
| 490 | programs, this is usually an indication that a change of approach might be |
| 491 | beneficial. |
| 492 | |
| 493 | In comp.lang.python, Fredrik Lundh once gave an excellent analogy in answer to |
| 494 | this question: |
| 495 | |
| 496 | The same way as you get the name of that cat you found on your porch: the cat |
| 497 | (object) itself cannot tell you its name, and it doesn't really care -- so |
| 498 | the only way to find out what it's called is to ask all your neighbours |
| 499 | (namespaces) if it's their cat (object)... |
| 500 | |
| 501 | ....and don't be surprised if you'll find that it's known by many names, or |
| 502 | no name at all! |
| 503 | |
| 504 | |
| 505 | What's up with the comma operator's precedence? |
| 506 | ----------------------------------------------- |
| 507 | |
| 508 | Comma is not an operator in Python. Consider this session:: |
| 509 | |
| 510 | >>> "a" in "b", "a" |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 511 | (False, 'a') |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 512 | |
| 513 | Since the comma is not an operator, but a separator between expressions the |
| 514 | above is evaluated as if you had entered:: |
| 515 | |
| 516 | >>> ("a" in "b"), "a" |
| 517 | |
| 518 | not:: |
| 519 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 520 | >>> "a" in ("b", "a") |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 521 | |
| 522 | The same is true of the various assignment operators (``=``, ``+=`` etc). They |
| 523 | are not truly operators but syntactic delimiters in assignment statements. |
| 524 | |
| 525 | |
| 526 | Is there an equivalent of C's "?:" ternary operator? |
| 527 | ---------------------------------------------------- |
| 528 | |
Antoine Pitrou | c5b266e | 2011-12-03 22:11:11 +0100 | [diff] [blame] | 529 | Yes, there is. The syntax is as follows:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 530 | |
| 531 | [on_true] if [expression] else [on_false] |
| 532 | |
| 533 | x, y = 50, 25 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 534 | small = x if x < y else y |
| 535 | |
Antoine Pitrou | c5b266e | 2011-12-03 22:11:11 +0100 | [diff] [blame] | 536 | Before this syntax was introduced in Python 2.5, a common idiom was to use |
| 537 | logical operators:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 538 | |
Antoine Pitrou | c5b266e | 2011-12-03 22:11:11 +0100 | [diff] [blame] | 539 | [expression] and [on_true] or [on_false] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 540 | |
Antoine Pitrou | c5b266e | 2011-12-03 22:11:11 +0100 | [diff] [blame] | 541 | However, this idiom is unsafe, as it can give wrong results when *on_true* |
| 542 | has a false boolean value. Therefore, it is always better to use |
| 543 | the ``... if ... else ...`` form. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 544 | |
| 545 | |
| 546 | Is it possible to write obfuscated one-liners in Python? |
| 547 | -------------------------------------------------------- |
| 548 | |
| 549 | Yes. Usually this is done by nesting :keyword:`lambda` within |
| 550 | :keyword:`lambda`. See the following three examples, due to Ulf Bartelt:: |
| 551 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 552 | from functools import reduce |
| 553 | |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 554 | # Primes < 1000 |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 555 | print(list(filter(None,map(lambda y:y*reduce(lambda x,y:x*y!=0, |
| 556 | map(lambda x,y=y:y%x,range(2,int(pow(y,0.5)+1))),1),range(2,1000))))) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 557 | |
| 558 | # First 10 Fibonacci numbers |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 559 | print(list(map(lambda x,f=lambda x,f:(f(x-1,f)+f(x-2,f)) if x>1 else 1: |
| 560 | f(x,f), range(10)))) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 561 | |
| 562 | # Mandelbrot set |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 563 | print((lambda Ru,Ro,Iu,Io,IM,Sx,Sy:reduce(lambda x,y:x+y,map(lambda y, |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 564 | Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,Sy=Sy,L=lambda yc,Iu=Iu,Io=Io,Ru=Ru,Ro=Ro,i=IM, |
| 565 | Sx=Sx,Sy=Sy:reduce(lambda x,y:x+y,map(lambda x,xc=Ru,yc=yc,Ru=Ru,Ro=Ro, |
| 566 | i=i,Sx=Sx,F=lambda xc,yc,x,y,k,f=lambda xc,yc,x,y,k,f:(k<=0)or (x*x+y*y |
| 567 | >=4.0) or 1+f(xc,yc,x*x-y*y+xc,2.0*x*y+yc,k-1,f):f(xc,yc,x,y,k,f):chr( |
| 568 | 64+F(Ru+x*(Ro-Ru)/Sx,yc,0,0,i)),range(Sx))):L(Iu+y*(Io-Iu)/Sy),range(Sy |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 569 | ))))(-2.1, 0.7, -1.2, 1.2, 30, 80, 24)) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 570 | # \___ ___/ \___ ___/ | | |__ lines on screen |
| 571 | # V V | |______ columns on screen |
| 572 | # | | |__________ maximum of "iterations" |
| 573 | # | |_________________ range on y axis |
| 574 | # |____________________________ range on x axis |
| 575 | |
| 576 | Don't try this at home, kids! |
| 577 | |
| 578 | |
| 579 | Numbers and strings |
| 580 | =================== |
| 581 | |
| 582 | How do I specify hexadecimal and octal integers? |
| 583 | ------------------------------------------------ |
| 584 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 585 | To specify an octal digit, precede the octal value with a zero, and then a lower |
| 586 | or uppercase "o". For example, to set the variable "a" to the octal value "10" |
| 587 | (8 in decimal), type:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 588 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 589 | >>> a = 0o10 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 590 | >>> a |
| 591 | 8 |
| 592 | |
| 593 | Hexadecimal is just as easy. Simply precede the hexadecimal number with a zero, |
| 594 | and then a lower or uppercase "x". Hexadecimal digits can be specified in lower |
| 595 | or uppercase. For example, in the Python interpreter:: |
| 596 | |
| 597 | >>> a = 0xa5 |
| 598 | >>> a |
| 599 | 165 |
| 600 | >>> b = 0XB2 |
| 601 | >>> b |
| 602 | 178 |
| 603 | |
| 604 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 605 | Why does -22 // 10 return -3? |
| 606 | ----------------------------- |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 607 | |
| 608 | It's primarily driven by the desire that ``i % j`` have the same sign as ``j``. |
| 609 | If you want that, and also want:: |
| 610 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 611 | i == (i // j) * j + (i % j) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 612 | |
| 613 | then integer division has to return the floor. C also requires that identity to |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 614 | hold, and then compilers that truncate ``i // j`` need to make ``i % j`` have |
| 615 | the same sign as ``i``. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 616 | |
| 617 | There are few real use cases for ``i % j`` when ``j`` is negative. When ``j`` |
| 618 | is positive, there are many, and in virtually all of them it's more useful for |
| 619 | ``i % j`` to be ``>= 0``. If the clock says 10 now, what did it say 200 hours |
| 620 | ago? ``-190 % 12 == 2`` is useful; ``-190 % 12 == -10`` is a bug waiting to |
| 621 | bite. |
| 622 | |
| 623 | |
| 624 | How do I convert a string to a number? |
| 625 | -------------------------------------- |
| 626 | |
| 627 | For integers, use the built-in :func:`int` type constructor, e.g. ``int('144') |
| 628 | == 144``. Similarly, :func:`float` converts to floating-point, |
| 629 | e.g. ``float('144') == 144.0``. |
| 630 | |
| 631 | By default, these interpret the number as decimal, so that ``int('0144') == |
| 632 | 144`` and ``int('0x144')`` raises :exc:`ValueError`. ``int(string, base)`` takes |
| 633 | the base to convert from as a second optional argument, so ``int('0x144', 16) == |
| 634 | 324``. If the base is specified as 0, the number is interpreted using Python's |
| 635 | rules: a leading '0' indicates octal, and '0x' indicates a hex number. |
| 636 | |
| 637 | Do not use the built-in function :func:`eval` if all you need is to convert |
| 638 | strings to numbers. :func:`eval` will be significantly slower and it presents a |
| 639 | security risk: someone could pass you a Python expression that might have |
| 640 | unwanted side effects. For example, someone could pass |
| 641 | ``__import__('os').system("rm -rf $HOME")`` which would erase your home |
| 642 | directory. |
| 643 | |
| 644 | :func:`eval` also has the effect of interpreting numbers as Python expressions, |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 645 | so that e.g. ``eval('09')`` gives a syntax error because Python does not allow |
| 646 | leading '0' in a decimal number (except '0'). |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 647 | |
| 648 | |
| 649 | How do I convert a number to a string? |
| 650 | -------------------------------------- |
| 651 | |
| 652 | To convert, e.g., the number 144 to the string '144', use the built-in type |
| 653 | constructor :func:`str`. If you want a hexadecimal or octal representation, use |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 654 | the built-in functions :func:`hex` or :func:`oct`. For fancy formatting, see |
| 655 | the :ref:`string-formatting` section, e.g. ``"{:04d}".format(144)`` yields |
Georg Brandl | 11b6362 | 2009-12-20 14:21:27 +0000 | [diff] [blame] | 656 | ``'0144'`` and ``"{:.3f}".format(1/3)`` yields ``'0.333'``. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 657 | |
| 658 | |
| 659 | How do I modify a string in place? |
| 660 | ---------------------------------- |
| 661 | |
Antoine Pitrou | c5b266e | 2011-12-03 22:11:11 +0100 | [diff] [blame] | 662 | You can't, because strings are immutable. In most situations, you should |
| 663 | simply construct a new string from the various parts you want to assemble |
| 664 | it from. However, if you need an object with the ability to modify in-place |
| 665 | unicode data, try using a :class:`io.StringIO` object or the :mod:`array` |
| 666 | module:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 667 | |
| 668 | >>> s = "Hello, world" |
Antoine Pitrou | c5b266e | 2011-12-03 22:11:11 +0100 | [diff] [blame] | 669 | >>> sio = io.StringIO(s) |
| 670 | >>> sio.getvalue() |
| 671 | 'Hello, world' |
| 672 | >>> sio.seek(7) |
| 673 | 7 |
| 674 | >>> sio.write("there!") |
| 675 | 6 |
| 676 | >>> sio.getvalue() |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 677 | 'Hello, there!' |
| 678 | |
| 679 | >>> import array |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 680 | >>> a = array.array('u', s) |
| 681 | >>> print(a) |
| 682 | array('u', 'Hello, world') |
| 683 | >>> a[0] = 'y' |
| 684 | >>> print(a) |
| 685 | array('u', 'yello world') |
| 686 | >>> a.tounicode() |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 687 | 'yello, world' |
| 688 | |
| 689 | |
| 690 | How do I use strings to call functions/methods? |
| 691 | ----------------------------------------------- |
| 692 | |
| 693 | There are various techniques. |
| 694 | |
| 695 | * The best is to use a dictionary that maps strings to functions. The primary |
| 696 | advantage of this technique is that the strings do not need to match the names |
| 697 | of the functions. This is also the primary technique used to emulate a case |
| 698 | construct:: |
| 699 | |
| 700 | def a(): |
| 701 | pass |
| 702 | |
| 703 | def b(): |
| 704 | pass |
| 705 | |
| 706 | dispatch = {'go': a, 'stop': b} # Note lack of parens for funcs |
| 707 | |
| 708 | dispatch[get_input()]() # Note trailing parens to call function |
| 709 | |
| 710 | * Use the built-in function :func:`getattr`:: |
| 711 | |
| 712 | import foo |
| 713 | getattr(foo, 'bar')() |
| 714 | |
| 715 | Note that :func:`getattr` works on any object, including classes, class |
| 716 | instances, modules, and so on. |
| 717 | |
| 718 | This is used in several places in the standard library, like this:: |
| 719 | |
| 720 | class Foo: |
| 721 | def do_foo(self): |
| 722 | ... |
| 723 | |
| 724 | def do_bar(self): |
| 725 | ... |
| 726 | |
| 727 | f = getattr(foo_instance, 'do_' + opname) |
| 728 | f() |
| 729 | |
| 730 | |
| 731 | * Use :func:`locals` or :func:`eval` to resolve the function name:: |
| 732 | |
| 733 | def myFunc(): |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 734 | print("hello") |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 735 | |
| 736 | fname = "myFunc" |
| 737 | |
| 738 | f = locals()[fname] |
| 739 | f() |
| 740 | |
| 741 | f = eval(fname) |
| 742 | f() |
| 743 | |
| 744 | Note: Using :func:`eval` is slow and dangerous. If you don't have absolute |
| 745 | control over the contents of the string, someone could pass a string that |
| 746 | resulted in an arbitrary function being executed. |
| 747 | |
| 748 | Is there an equivalent to Perl's chomp() for removing trailing newlines from strings? |
| 749 | ------------------------------------------------------------------------------------- |
| 750 | |
Antoine Pitrou | f352040 | 2011-12-03 22:19:55 +0100 | [diff] [blame] | 751 | You can use ``S.rstrip("\r\n")`` to remove all occurrences of any line |
| 752 | terminator from the end of the string ``S`` without removing other trailing |
| 753 | whitespace. If the string ``S`` represents more than one line, with several |
| 754 | empty lines at the end, the line terminators for all the blank lines will |
| 755 | be removed:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 756 | |
| 757 | >>> lines = ("line 1 \r\n" |
| 758 | ... "\r\n" |
| 759 | ... "\r\n") |
| 760 | >>> lines.rstrip("\n\r") |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 761 | 'line 1 ' |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 762 | |
| 763 | Since this is typically only desired when reading text one line at a time, using |
| 764 | ``S.rstrip()`` this way works well. |
| 765 | |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 766 | |
| 767 | Is there a scanf() or sscanf() equivalent? |
| 768 | ------------------------------------------ |
| 769 | |
| 770 | Not as such. |
| 771 | |
| 772 | For simple input parsing, the easiest approach is usually to split the line into |
| 773 | whitespace-delimited words using the :meth:`~str.split` method of string objects |
| 774 | and then convert decimal strings to numeric values using :func:`int` or |
| 775 | :func:`float`. ``split()`` supports an optional "sep" parameter which is useful |
| 776 | if the line uses something other than whitespace as a separator. |
| 777 | |
Brian Curtin | 5a7a52f | 2010-09-23 13:45:21 +0000 | [diff] [blame] | 778 | For more complicated input parsing, regular expressions are more powerful |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 779 | than C's :c:func:`sscanf` and better suited for the task. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 780 | |
| 781 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 782 | What does 'UnicodeDecodeError' or 'UnicodeEncodeError' error mean? |
| 783 | ------------------------------------------------------------------- |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 784 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 785 | See the :ref:`unicode-howto`. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 786 | |
| 787 | |
Antoine Pitrou | 432259f | 2011-12-09 23:10:31 +0100 | [diff] [blame] | 788 | Performance |
| 789 | =========== |
| 790 | |
| 791 | My program is too slow. How do I speed it up? |
| 792 | --------------------------------------------- |
| 793 | |
| 794 | That's a tough one, in general. First, here are a list of things to |
| 795 | remember before diving further: |
| 796 | |
Georg Brandl | 300a691 | 2012-03-14 22:40:08 +0100 | [diff] [blame^] | 797 | * Performance characteristics vary across Python implementations. This FAQ |
Antoine Pitrou | 432259f | 2011-12-09 23:10:31 +0100 | [diff] [blame] | 798 | focusses on :term:`CPython`. |
Georg Brandl | 300a691 | 2012-03-14 22:40:08 +0100 | [diff] [blame^] | 799 | * Behaviour can vary across operating systems, especially when talking about |
Antoine Pitrou | 432259f | 2011-12-09 23:10:31 +0100 | [diff] [blame] | 800 | I/O or multi-threading. |
| 801 | * You should always find the hot spots in your program *before* attempting to |
| 802 | optimize any code (see the :mod:`profile` module). |
| 803 | * Writing benchmark scripts will allow you to iterate quickly when searching |
| 804 | for improvements (see the :mod:`timeit` module). |
| 805 | * It is highly recommended to have good code coverage (through unit testing |
| 806 | or any other technique) before potentially introducing regressions hidden |
| 807 | in sophisticated optimizations. |
| 808 | |
| 809 | That being said, there are many tricks to speed up Python code. Here are |
| 810 | some general principles which go a long way towards reaching acceptable |
| 811 | performance levels: |
| 812 | |
| 813 | * Making your algorithms faster (or changing to faster ones) can yield |
| 814 | much larger benefits than trying to sprinkle micro-optimization tricks |
| 815 | all over your code. |
| 816 | |
| 817 | * Use the right data structures. Study documentation for the :ref:`bltin-types` |
| 818 | and the :mod:`collections` module. |
| 819 | |
| 820 | * When the standard library provides a primitive for doing something, it is |
| 821 | likely (although not guaranteed) to be faster than any alternative you |
| 822 | may come up with. This is doubly true for primitives written in C, such |
| 823 | as builtins and some extension types. For example, be sure to use |
| 824 | either the :meth:`list.sort` built-in method or the related :func:`sorted` |
| 825 | function to do sorting (and see the |
| 826 | `sorting mini-HOWTO <http://wiki.python.org/moin/HowTo/Sorting>`_ for examples |
| 827 | of moderately advanced usage). |
| 828 | |
| 829 | * Abstractions tend to create indirections and force the interpreter to work |
| 830 | more. If the levels of indirection outweigh the amount of useful work |
| 831 | done, your program will be slower. You should avoid excessive abstraction, |
| 832 | especially under the form of tiny functions or methods (which are also often |
| 833 | detrimental to readability). |
| 834 | |
| 835 | If you have reached the limit of what pure Python can allow, there are tools |
| 836 | to take you further away. For example, `Cython <http://cython.org>`_ can |
| 837 | compile a slightly modified version of Python code into a C extension, and |
| 838 | can be used on many different platforms. Cython can take advantage of |
| 839 | compilation (and optional type annotations) to make your code significantly |
| 840 | faster than when interpreted. If you are confident in your C programming |
| 841 | skills, you can also :ref:`write a C extension module <extending-index>` |
| 842 | yourself. |
| 843 | |
| 844 | .. seealso:: |
| 845 | The wiki page devoted to `performance tips |
| 846 | <http://wiki.python.org/moin/PythonSpeed/PerformanceTips>`_. |
| 847 | |
| 848 | .. _efficient_string_concatenation: |
| 849 | |
Antoine Pitrou | fd9ebd4 | 2011-11-25 16:33:53 +0100 | [diff] [blame] | 850 | What is the most efficient way to concatenate many strings together? |
| 851 | -------------------------------------------------------------------- |
| 852 | |
| 853 | :class:`str` and :class:`bytes` objects are immutable, therefore concatenating |
| 854 | many strings together is inefficient as each concatenation creates a new |
| 855 | object. In the general case, the total runtime cost is quadratic in the |
| 856 | total string length. |
| 857 | |
| 858 | To accumulate many :class:`str` objects, the recommended idiom is to place |
| 859 | them into a list and call :meth:`str.join` at the end:: |
| 860 | |
| 861 | chunks = [] |
| 862 | for s in my_strings: |
| 863 | chunks.append(s) |
| 864 | result = ''.join(chunks) |
| 865 | |
| 866 | (another reasonably efficient idiom is to use :class:`io.StringIO`) |
| 867 | |
| 868 | To accumulate many :class:`bytes` objects, the recommended idiom is to extend |
| 869 | a :class:`bytearray` object using in-place concatenation (the ``+=`` operator):: |
| 870 | |
| 871 | result = bytearray() |
| 872 | for b in my_bytes_objects: |
| 873 | result += b |
| 874 | |
| 875 | |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 876 | Sequences (Tuples/Lists) |
| 877 | ======================== |
| 878 | |
| 879 | How do I convert between tuples and lists? |
| 880 | ------------------------------------------ |
| 881 | |
| 882 | The type constructor ``tuple(seq)`` converts any sequence (actually, any |
| 883 | iterable) into a tuple with the same items in the same order. |
| 884 | |
| 885 | For example, ``tuple([1, 2, 3])`` yields ``(1, 2, 3)`` and ``tuple('abc')`` |
| 886 | yields ``('a', 'b', 'c')``. If the argument is a tuple, it does not make a copy |
| 887 | but returns the same object, so it is cheap to call :func:`tuple` when you |
| 888 | aren't sure that an object is already a tuple. |
| 889 | |
| 890 | The type constructor ``list(seq)`` converts any sequence or iterable into a list |
| 891 | with the same items in the same order. For example, ``list((1, 2, 3))`` yields |
| 892 | ``[1, 2, 3]`` and ``list('abc')`` yields ``['a', 'b', 'c']``. If the argument |
| 893 | is a list, it makes a copy just like ``seq[:]`` would. |
| 894 | |
| 895 | |
| 896 | What's a negative index? |
| 897 | ------------------------ |
| 898 | |
| 899 | Python sequences are indexed with positive numbers and negative numbers. For |
| 900 | positive numbers 0 is the first index 1 is the second index and so forth. For |
| 901 | negative indices -1 is the last index and -2 is the penultimate (next to last) |
| 902 | index and so forth. Think of ``seq[-n]`` as the same as ``seq[len(seq)-n]``. |
| 903 | |
| 904 | Using negative indices can be very convenient. For example ``S[:-1]`` is all of |
| 905 | the string except for its last character, which is useful for removing the |
| 906 | trailing newline from a string. |
| 907 | |
| 908 | |
| 909 | How do I iterate over a sequence in reverse order? |
| 910 | -------------------------------------------------- |
| 911 | |
Georg Brandl | c4a55fc | 2010-02-06 18:46:57 +0000 | [diff] [blame] | 912 | Use the :func:`reversed` built-in function, which is new in Python 2.4:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 913 | |
| 914 | for x in reversed(sequence): |
| 915 | ... # do something with x... |
| 916 | |
| 917 | This won't touch your original sequence, but build a new copy with reversed |
| 918 | order to iterate over. |
| 919 | |
| 920 | With Python 2.3, you can use an extended slice syntax:: |
| 921 | |
| 922 | for x in sequence[::-1]: |
| 923 | ... # do something with x... |
| 924 | |
| 925 | |
| 926 | How do you remove duplicates from a list? |
| 927 | ----------------------------------------- |
| 928 | |
| 929 | See the Python Cookbook for a long discussion of many ways to do this: |
| 930 | |
| 931 | http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560 |
| 932 | |
| 933 | If you don't mind reordering the list, sort it and then scan from the end of the |
| 934 | list, deleting duplicates as you go:: |
| 935 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 936 | if mylist: |
| 937 | mylist.sort() |
| 938 | last = mylist[-1] |
| 939 | for i in range(len(mylist)-2, -1, -1): |
| 940 | if last == mylist[i]: |
| 941 | del mylist[i] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 942 | else: |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 943 | last = mylist[i] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 944 | |
Antoine Pitrou | f352040 | 2011-12-03 22:19:55 +0100 | [diff] [blame] | 945 | If all elements of the list may be used as set keys (i.e. they are all |
| 946 | :term:`hashable`) this is often faster :: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 947 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 948 | mylist = list(set(mylist)) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 949 | |
| 950 | This converts the list into a set, thereby removing duplicates, and then back |
| 951 | into a list. |
| 952 | |
| 953 | |
| 954 | How do you make an array in Python? |
| 955 | ----------------------------------- |
| 956 | |
| 957 | Use a list:: |
| 958 | |
| 959 | ["this", 1, "is", "an", "array"] |
| 960 | |
| 961 | Lists are equivalent to C or Pascal arrays in their time complexity; the primary |
| 962 | difference is that a Python list can contain objects of many different types. |
| 963 | |
| 964 | The ``array`` module also provides methods for creating arrays of fixed types |
| 965 | with compact representations, but they are slower to index than lists. Also |
| 966 | note that the Numeric extensions and others define array-like structures with |
| 967 | various characteristics as well. |
| 968 | |
| 969 | To get Lisp-style linked lists, you can emulate cons cells using tuples:: |
| 970 | |
| 971 | lisp_list = ("like", ("this", ("example", None) ) ) |
| 972 | |
| 973 | If mutability is desired, you could use lists instead of tuples. Here the |
| 974 | analogue of lisp car is ``lisp_list[0]`` and the analogue of cdr is |
| 975 | ``lisp_list[1]``. Only do this if you're sure you really need to, because it's |
| 976 | usually a lot slower than using Python lists. |
| 977 | |
| 978 | |
| 979 | How do I create a multidimensional list? |
| 980 | ---------------------------------------- |
| 981 | |
| 982 | You probably tried to make a multidimensional array like this:: |
| 983 | |
| 984 | A = [[None] * 2] * 3 |
| 985 | |
| 986 | This looks correct if you print it:: |
| 987 | |
| 988 | >>> A |
| 989 | [[None, None], [None, None], [None, None]] |
| 990 | |
| 991 | But when you assign a value, it shows up in multiple places: |
| 992 | |
| 993 | >>> A[0][0] = 5 |
| 994 | >>> A |
| 995 | [[5, None], [5, None], [5, None]] |
| 996 | |
| 997 | The reason is that replicating a list with ``*`` doesn't create copies, it only |
| 998 | creates references to the existing objects. The ``*3`` creates a list |
| 999 | containing 3 references to the same list of length two. Changes to one row will |
| 1000 | show in all rows, which is almost certainly not what you want. |
| 1001 | |
| 1002 | The suggested approach is to create a list of the desired length first and then |
| 1003 | fill in each element with a newly created list:: |
| 1004 | |
| 1005 | A = [None] * 3 |
| 1006 | for i in range(3): |
| 1007 | A[i] = [None] * 2 |
| 1008 | |
| 1009 | This generates a list containing 3 different lists of length two. You can also |
| 1010 | use a list comprehension:: |
| 1011 | |
| 1012 | w, h = 2, 3 |
| 1013 | A = [[None] * w for i in range(h)] |
| 1014 | |
| 1015 | Or, you can use an extension that provides a matrix datatype; `Numeric Python |
Georg Brandl | 495f7b5 | 2009-10-27 15:28:25 +0000 | [diff] [blame] | 1016 | <http://numpy.scipy.org/>`_ is the best known. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1017 | |
| 1018 | |
| 1019 | How do I apply a method to a sequence of objects? |
| 1020 | ------------------------------------------------- |
| 1021 | |
| 1022 | Use a list comprehension:: |
| 1023 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1024 | result = [obj.method() for obj in mylist] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1025 | |
| 1026 | |
| 1027 | Dictionaries |
| 1028 | ============ |
| 1029 | |
| 1030 | How can I get a dictionary to display its keys in a consistent order? |
| 1031 | --------------------------------------------------------------------- |
| 1032 | |
| 1033 | You can't. Dictionaries store their keys in an unpredictable order, so the |
| 1034 | display order of a dictionary's elements will be similarly unpredictable. |
| 1035 | |
| 1036 | This can be frustrating if you want to save a printable version to a file, make |
| 1037 | some changes and then compare it with some other printed dictionary. In this |
| 1038 | case, use the ``pprint`` module to pretty-print the dictionary; the items will |
| 1039 | be presented in order sorted by the key. |
| 1040 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1041 | A more complicated solution is to subclass ``dict`` to create a |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1042 | ``SortedDict`` class that prints itself in a predictable order. Here's one |
| 1043 | simpleminded implementation of such a class:: |
| 1044 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1045 | class SortedDict(dict): |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1046 | def __repr__(self): |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1047 | keys = sorted(self.keys()) |
| 1048 | result = ("{!r}: {!r}".format(k, self[k]) for k in keys) |
| 1049 | return "{{{}}}".format(", ".join(result)) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1050 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1051 | __str__ = __repr__ |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1052 | |
| 1053 | This will work for many common situations you might encounter, though it's far |
| 1054 | from a perfect solution. The largest flaw is that if some values in the |
| 1055 | dictionary are also dictionaries, their values won't be presented in any |
| 1056 | particular order. |
| 1057 | |
| 1058 | |
| 1059 | I want to do a complicated sort: can you do a Schwartzian Transform in Python? |
| 1060 | ------------------------------------------------------------------------------ |
| 1061 | |
| 1062 | The technique, attributed to Randal Schwartz of the Perl community, sorts the |
| 1063 | elements of a list by a metric which maps each element to its "sort value". In |
| 1064 | Python, just use the ``key`` argument for the ``sort()`` method:: |
| 1065 | |
| 1066 | Isorted = L[:] |
| 1067 | Isorted.sort(key=lambda s: int(s[10:15])) |
| 1068 | |
| 1069 | The ``key`` argument is new in Python 2.4, for older versions this kind of |
| 1070 | sorting is quite simple to do with list comprehensions. To sort a list of |
| 1071 | strings by their uppercase values:: |
| 1072 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1073 | tmp1 = [(x.upper(), x) for x in L] # Schwartzian transform |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1074 | tmp1.sort() |
| 1075 | Usorted = [x[1] for x in tmp1] |
| 1076 | |
| 1077 | To sort by the integer value of a subfield extending from positions 10-15 in |
| 1078 | each string:: |
| 1079 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1080 | tmp2 = [(int(s[10:15]), s) for s in L] # Schwartzian transform |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1081 | tmp2.sort() |
| 1082 | Isorted = [x[1] for x in tmp2] |
| 1083 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1084 | For versions prior to 3.0, Isorted may also be computed by :: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1085 | |
| 1086 | def intfield(s): |
| 1087 | return int(s[10:15]) |
| 1088 | |
| 1089 | def Icmp(s1, s2): |
| 1090 | return cmp(intfield(s1), intfield(s2)) |
| 1091 | |
| 1092 | Isorted = L[:] |
| 1093 | Isorted.sort(Icmp) |
| 1094 | |
| 1095 | but since this method calls ``intfield()`` many times for each element of L, it |
| 1096 | is slower than the Schwartzian Transform. |
| 1097 | |
| 1098 | |
| 1099 | How can I sort one list by values from another list? |
| 1100 | ---------------------------------------------------- |
| 1101 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1102 | Merge them into an iterator of tuples, sort the resulting list, and then pick |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1103 | out the element you want. :: |
| 1104 | |
| 1105 | >>> list1 = ["what", "I'm", "sorting", "by"] |
| 1106 | >>> list2 = ["something", "else", "to", "sort"] |
| 1107 | >>> pairs = zip(list1, list2) |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1108 | >>> pairs = sorted(pairs) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1109 | >>> pairs |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1110 | [("I'm", 'else'), ('by', 'sort'), ('sorting', 'to'), ('what', 'something')] |
| 1111 | >>> result = [x[1] for x in pairs] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1112 | >>> result |
| 1113 | ['else', 'sort', 'to', 'something'] |
| 1114 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1115 | |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1116 | An alternative for the last step is:: |
| 1117 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1118 | >>> result = [] |
| 1119 | >>> for p in pairs: result.append(p[1]) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1120 | |
| 1121 | If you find this more legible, you might prefer to use this instead of the final |
| 1122 | list comprehension. However, it is almost twice as slow for long lists. Why? |
| 1123 | First, the ``append()`` operation has to reallocate memory, and while it uses |
| 1124 | some tricks to avoid doing that each time, it still has to do it occasionally, |
| 1125 | and that costs quite a bit. Second, the expression "result.append" requires an |
| 1126 | extra attribute lookup, and third, there's a speed reduction from having to make |
| 1127 | all those function calls. |
| 1128 | |
| 1129 | |
| 1130 | Objects |
| 1131 | ======= |
| 1132 | |
| 1133 | What is a class? |
| 1134 | ---------------- |
| 1135 | |
| 1136 | A class is the particular object type created by executing a class statement. |
| 1137 | Class objects are used as templates to create instance objects, which embody |
| 1138 | both the data (attributes) and code (methods) specific to a datatype. |
| 1139 | |
| 1140 | A class can be based on one or more other classes, called its base class(es). It |
| 1141 | then inherits the attributes and methods of its base classes. This allows an |
| 1142 | object model to be successively refined by inheritance. You might have a |
| 1143 | generic ``Mailbox`` class that provides basic accessor methods for a mailbox, |
| 1144 | and subclasses such as ``MboxMailbox``, ``MaildirMailbox``, ``OutlookMailbox`` |
| 1145 | that handle various specific mailbox formats. |
| 1146 | |
| 1147 | |
| 1148 | What is a method? |
| 1149 | ----------------- |
| 1150 | |
| 1151 | A method is a function on some object ``x`` that you normally call as |
| 1152 | ``x.name(arguments...)``. Methods are defined as functions inside the class |
| 1153 | definition:: |
| 1154 | |
| 1155 | class C: |
| 1156 | def meth (self, arg): |
| 1157 | return arg * 2 + self.attribute |
| 1158 | |
| 1159 | |
| 1160 | What is self? |
| 1161 | ------------- |
| 1162 | |
| 1163 | Self is merely a conventional name for the first argument of a method. A method |
| 1164 | defined as ``meth(self, a, b, c)`` should be called as ``x.meth(a, b, c)`` for |
| 1165 | some instance ``x`` of the class in which the definition occurs; the called |
| 1166 | method will think it is called as ``meth(x, a, b, c)``. |
| 1167 | |
| 1168 | See also :ref:`why-self`. |
| 1169 | |
| 1170 | |
| 1171 | How do I check if an object is an instance of a given class or of a subclass of it? |
| 1172 | ----------------------------------------------------------------------------------- |
| 1173 | |
| 1174 | Use the built-in function ``isinstance(obj, cls)``. You can check if an object |
| 1175 | is an instance of any of a number of classes by providing a tuple instead of a |
| 1176 | single class, e.g. ``isinstance(obj, (class1, class2, ...))``, and can also |
| 1177 | check whether an object is one of Python's built-in types, e.g. |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1178 | ``isinstance(obj, str)`` or ``isinstance(obj, (int, float, complex))``. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1179 | |
| 1180 | Note that most programs do not use :func:`isinstance` on user-defined classes |
| 1181 | very often. If you are developing the classes yourself, a more proper |
| 1182 | object-oriented style is to define methods on the classes that encapsulate a |
| 1183 | particular behaviour, instead of checking the object's class and doing a |
| 1184 | different thing based on what class it is. For example, if you have a function |
| 1185 | that does something:: |
| 1186 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1187 | def search(obj): |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1188 | if isinstance(obj, Mailbox): |
| 1189 | # ... code to search a mailbox |
| 1190 | elif isinstance(obj, Document): |
| 1191 | # ... code to search a document |
| 1192 | elif ... |
| 1193 | |
| 1194 | A better approach is to define a ``search()`` method on all the classes and just |
| 1195 | call it:: |
| 1196 | |
| 1197 | class Mailbox: |
| 1198 | def search(self): |
| 1199 | # ... code to search a mailbox |
| 1200 | |
| 1201 | class Document: |
| 1202 | def search(self): |
| 1203 | # ... code to search a document |
| 1204 | |
| 1205 | obj.search() |
| 1206 | |
| 1207 | |
| 1208 | What is delegation? |
| 1209 | ------------------- |
| 1210 | |
| 1211 | Delegation is an object oriented technique (also called a design pattern). |
| 1212 | Let's say you have an object ``x`` and want to change the behaviour of just one |
| 1213 | of its methods. You can create a new class that provides a new implementation |
| 1214 | of the method you're interested in changing and delegates all other methods to |
| 1215 | the corresponding method of ``x``. |
| 1216 | |
| 1217 | Python programmers can easily implement delegation. For example, the following |
| 1218 | class implements a class that behaves like a file but converts all written data |
| 1219 | to uppercase:: |
| 1220 | |
| 1221 | class UpperOut: |
| 1222 | |
| 1223 | def __init__(self, outfile): |
| 1224 | self._outfile = outfile |
| 1225 | |
| 1226 | def write(self, s): |
| 1227 | self._outfile.write(s.upper()) |
| 1228 | |
| 1229 | def __getattr__(self, name): |
| 1230 | return getattr(self._outfile, name) |
| 1231 | |
| 1232 | Here the ``UpperOut`` class redefines the ``write()`` method to convert the |
| 1233 | argument string to uppercase before calling the underlying |
| 1234 | ``self.__outfile.write()`` method. All other methods are delegated to the |
| 1235 | underlying ``self.__outfile`` object. The delegation is accomplished via the |
| 1236 | ``__getattr__`` method; consult :ref:`the language reference <attribute-access>` |
| 1237 | for more information about controlling attribute access. |
| 1238 | |
| 1239 | Note that for more general cases delegation can get trickier. When attributes |
| 1240 | must be set as well as retrieved, the class must define a :meth:`__setattr__` |
| 1241 | method too, and it must do so carefully. The basic implementation of |
| 1242 | :meth:`__setattr__` is roughly equivalent to the following:: |
| 1243 | |
| 1244 | class X: |
| 1245 | ... |
| 1246 | def __setattr__(self, name, value): |
| 1247 | self.__dict__[name] = value |
| 1248 | ... |
| 1249 | |
| 1250 | Most :meth:`__setattr__` implementations must modify ``self.__dict__`` to store |
| 1251 | local state for self without causing an infinite recursion. |
| 1252 | |
| 1253 | |
| 1254 | How do I call a method defined in a base class from a derived class that overrides it? |
| 1255 | -------------------------------------------------------------------------------------- |
| 1256 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1257 | Use the built-in :func:`super` function:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1258 | |
| 1259 | class Derived(Base): |
| 1260 | def meth (self): |
| 1261 | super(Derived, self).meth() |
| 1262 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1263 | For version prior to 3.0, you may be using classic classes: For a class |
| 1264 | definition such as ``class Derived(Base): ...`` you can call method ``meth()`` |
| 1265 | defined in ``Base`` (or one of ``Base``'s base classes) as ``Base.meth(self, |
| 1266 | arguments...)``. Here, ``Base.meth`` is an unbound method, so you need to |
| 1267 | provide the ``self`` argument. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1268 | |
| 1269 | |
| 1270 | How can I organize my code to make it easier to change the base class? |
| 1271 | ---------------------------------------------------------------------- |
| 1272 | |
| 1273 | You could define an alias for the base class, assign the real base class to it |
| 1274 | before your class definition, and use the alias throughout your class. Then all |
| 1275 | you have to change is the value assigned to the alias. Incidentally, this trick |
| 1276 | is also handy if you want to decide dynamically (e.g. depending on availability |
| 1277 | of resources) which base class to use. Example:: |
| 1278 | |
| 1279 | BaseAlias = <real base class> |
| 1280 | |
| 1281 | class Derived(BaseAlias): |
| 1282 | def meth(self): |
| 1283 | BaseAlias.meth(self) |
| 1284 | ... |
| 1285 | |
| 1286 | |
| 1287 | How do I create static class data and static class methods? |
| 1288 | ----------------------------------------------------------- |
| 1289 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1290 | Both static data and static methods (in the sense of C++ or Java) are supported |
| 1291 | in Python. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1292 | |
| 1293 | For static data, simply define a class attribute. To assign a new value to the |
| 1294 | attribute, you have to explicitly use the class name in the assignment:: |
| 1295 | |
| 1296 | class C: |
| 1297 | count = 0 # number of times C.__init__ called |
| 1298 | |
| 1299 | def __init__(self): |
| 1300 | C.count = C.count + 1 |
| 1301 | |
| 1302 | def getcount(self): |
| 1303 | return C.count # or return self.count |
| 1304 | |
| 1305 | ``c.count`` also refers to ``C.count`` for any ``c`` such that ``isinstance(c, |
| 1306 | C)`` holds, unless overridden by ``c`` itself or by some class on the base-class |
| 1307 | search path from ``c.__class__`` back to ``C``. |
| 1308 | |
| 1309 | Caution: within a method of C, an assignment like ``self.count = 42`` creates a |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1310 | new and unrelated instance named "count" in ``self``'s own dict. Rebinding of a |
| 1311 | class-static data name must always specify the class whether inside a method or |
| 1312 | not:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1313 | |
| 1314 | C.count = 314 |
| 1315 | |
Antoine Pitrou | f352040 | 2011-12-03 22:19:55 +0100 | [diff] [blame] | 1316 | Static methods are possible:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1317 | |
| 1318 | class C: |
| 1319 | @staticmethod |
| 1320 | def static(arg1, arg2, arg3): |
| 1321 | # No 'self' parameter! |
| 1322 | ... |
| 1323 | |
| 1324 | However, a far more straightforward way to get the effect of a static method is |
| 1325 | via a simple module-level function:: |
| 1326 | |
| 1327 | def getcount(): |
| 1328 | return C.count |
| 1329 | |
| 1330 | If your code is structured so as to define one class (or tightly related class |
| 1331 | hierarchy) per module, this supplies the desired encapsulation. |
| 1332 | |
| 1333 | |
| 1334 | How can I overload constructors (or methods) in Python? |
| 1335 | ------------------------------------------------------- |
| 1336 | |
| 1337 | This answer actually applies to all methods, but the question usually comes up |
| 1338 | first in the context of constructors. |
| 1339 | |
| 1340 | In C++ you'd write |
| 1341 | |
| 1342 | .. code-block:: c |
| 1343 | |
| 1344 | class C { |
| 1345 | C() { cout << "No arguments\n"; } |
| 1346 | C(int i) { cout << "Argument is " << i << "\n"; } |
| 1347 | } |
| 1348 | |
| 1349 | In Python you have to write a single constructor that catches all cases using |
| 1350 | default arguments. For example:: |
| 1351 | |
| 1352 | class C: |
| 1353 | def __init__(self, i=None): |
| 1354 | if i is None: |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1355 | print("No arguments") |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1356 | else: |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1357 | print("Argument is", i) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1358 | |
| 1359 | This is not entirely equivalent, but close enough in practice. |
| 1360 | |
| 1361 | You could also try a variable-length argument list, e.g. :: |
| 1362 | |
| 1363 | def __init__(self, *args): |
| 1364 | ... |
| 1365 | |
| 1366 | The same approach works for all method definitions. |
| 1367 | |
| 1368 | |
| 1369 | I try to use __spam and I get an error about _SomeClassName__spam. |
| 1370 | ------------------------------------------------------------------ |
| 1371 | |
| 1372 | Variable names with double leading underscores are "mangled" to provide a simple |
| 1373 | but effective way to define class private variables. Any identifier of the form |
| 1374 | ``__spam`` (at least two leading underscores, at most one trailing underscore) |
| 1375 | is textually replaced with ``_classname__spam``, where ``classname`` is the |
| 1376 | current class name with any leading underscores stripped. |
| 1377 | |
| 1378 | This doesn't guarantee privacy: an outside user can still deliberately access |
| 1379 | the "_classname__spam" attribute, and private values are visible in the object's |
| 1380 | ``__dict__``. Many Python programmers never bother to use private variable |
| 1381 | names at all. |
| 1382 | |
| 1383 | |
| 1384 | My class defines __del__ but it is not called when I delete the object. |
| 1385 | ----------------------------------------------------------------------- |
| 1386 | |
| 1387 | There are several possible reasons for this. |
| 1388 | |
| 1389 | The del statement does not necessarily call :meth:`__del__` -- it simply |
| 1390 | decrements the object's reference count, and if this reaches zero |
| 1391 | :meth:`__del__` is called. |
| 1392 | |
| 1393 | If your data structures contain circular links (e.g. a tree where each child has |
| 1394 | a parent reference and each parent has a list of children) the reference counts |
| 1395 | will never go back to zero. Once in a while Python runs an algorithm to detect |
| 1396 | such cycles, but the garbage collector might run some time after the last |
| 1397 | reference to your data structure vanishes, so your :meth:`__del__` method may be |
| 1398 | called at an inconvenient and random time. This is inconvenient if you're trying |
| 1399 | to reproduce a problem. Worse, the order in which object's :meth:`__del__` |
| 1400 | methods are executed is arbitrary. You can run :func:`gc.collect` to force a |
| 1401 | collection, but there *are* pathological cases where objects will never be |
| 1402 | collected. |
| 1403 | |
| 1404 | Despite the cycle collector, it's still a good idea to define an explicit |
| 1405 | ``close()`` method on objects to be called whenever you're done with them. The |
| 1406 | ``close()`` method can then remove attributes that refer to subobjecs. Don't |
| 1407 | call :meth:`__del__` directly -- :meth:`__del__` should call ``close()`` and |
| 1408 | ``close()`` should make sure that it can be called more than once for the same |
| 1409 | object. |
| 1410 | |
| 1411 | Another way to avoid cyclical references is to use the :mod:`weakref` module, |
| 1412 | which allows you to point to objects without incrementing their reference count. |
| 1413 | Tree data structures, for instance, should use weak references for their parent |
| 1414 | and sibling references (if they need them!). |
| 1415 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1416 | .. XXX relevant for Python 3? |
| 1417 | |
| 1418 | If the object has ever been a local variable in a function that caught an |
| 1419 | expression in an except clause, chances are that a reference to the object |
| 1420 | still exists in that function's stack frame as contained in the stack trace. |
| 1421 | Normally, calling :func:`sys.exc_clear` will take care of this by clearing |
| 1422 | the last recorded exception. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1423 | |
| 1424 | Finally, if your :meth:`__del__` method raises an exception, a warning message |
| 1425 | is printed to :data:`sys.stderr`. |
| 1426 | |
| 1427 | |
| 1428 | How do I get a list of all instances of a given class? |
| 1429 | ------------------------------------------------------ |
| 1430 | |
| 1431 | Python does not keep track of all instances of a class (or of a built-in type). |
| 1432 | You can program the class's constructor to keep track of all instances by |
| 1433 | keeping a list of weak references to each instance. |
| 1434 | |
| 1435 | |
| 1436 | Modules |
| 1437 | ======= |
| 1438 | |
| 1439 | How do I create a .pyc file? |
| 1440 | ---------------------------- |
| 1441 | |
| 1442 | When a module is imported for the first time (or when the source is more recent |
| 1443 | than the current compiled file) a ``.pyc`` file containing the compiled code |
| 1444 | should be created in the same directory as the ``.py`` file. |
| 1445 | |
| 1446 | One reason that a ``.pyc`` file may not be created is permissions problems with |
| 1447 | the directory. This can happen, for example, if you develop as one user but run |
| 1448 | as another, such as if you are testing with a web server. Creation of a .pyc |
| 1449 | file is automatic if you're importing a module and Python has the ability |
| 1450 | (permissions, free space, etc...) to write the compiled module back to the |
| 1451 | directory. |
| 1452 | |
| 1453 | Running Python on a top level script is not considered an import and no ``.pyc`` |
| 1454 | will be created. For example, if you have a top-level module ``abc.py`` that |
| 1455 | imports another module ``xyz.py``, when you run abc, ``xyz.pyc`` will be created |
| 1456 | since xyz is imported, but no ``abc.pyc`` file will be created since ``abc.py`` |
| 1457 | isn't being imported. |
| 1458 | |
| 1459 | If you need to create abc.pyc -- that is, to create a .pyc file for a module |
| 1460 | that is not imported -- you can, using the :mod:`py_compile` and |
| 1461 | :mod:`compileall` modules. |
| 1462 | |
| 1463 | The :mod:`py_compile` module can manually compile any module. One way is to use |
| 1464 | the ``compile()`` function in that module interactively:: |
| 1465 | |
| 1466 | >>> import py_compile |
| 1467 | >>> py_compile.compile('abc.py') |
| 1468 | |
| 1469 | This will write the ``.pyc`` to the same location as ``abc.py`` (or you can |
| 1470 | override that with the optional parameter ``cfile``). |
| 1471 | |
| 1472 | You can also automatically compile all files in a directory or directories using |
| 1473 | the :mod:`compileall` module. You can do it from the shell prompt by running |
| 1474 | ``compileall.py`` and providing the path of a directory containing Python files |
| 1475 | to compile:: |
| 1476 | |
| 1477 | python -m compileall . |
| 1478 | |
| 1479 | |
| 1480 | How do I find the current module name? |
| 1481 | -------------------------------------- |
| 1482 | |
| 1483 | A module can find out its own module name by looking at the predefined global |
| 1484 | variable ``__name__``. If this has the value ``'__main__'``, the program is |
| 1485 | running as a script. Many modules that are usually used by importing them also |
| 1486 | provide a command-line interface or a self-test, and only execute this code |
| 1487 | after checking ``__name__``:: |
| 1488 | |
| 1489 | def main(): |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1490 | print('Running test...') |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1491 | ... |
| 1492 | |
| 1493 | if __name__ == '__main__': |
| 1494 | main() |
| 1495 | |
| 1496 | |
| 1497 | How can I have modules that mutually import each other? |
| 1498 | ------------------------------------------------------- |
| 1499 | |
| 1500 | Suppose you have the following modules: |
| 1501 | |
| 1502 | foo.py:: |
| 1503 | |
| 1504 | from bar import bar_var |
| 1505 | foo_var = 1 |
| 1506 | |
| 1507 | bar.py:: |
| 1508 | |
| 1509 | from foo import foo_var |
| 1510 | bar_var = 2 |
| 1511 | |
| 1512 | The problem is that the interpreter will perform the following steps: |
| 1513 | |
| 1514 | * main imports foo |
| 1515 | * Empty globals for foo are created |
| 1516 | * foo is compiled and starts executing |
| 1517 | * foo imports bar |
| 1518 | * Empty globals for bar are created |
| 1519 | * bar is compiled and starts executing |
| 1520 | * bar imports foo (which is a no-op since there already is a module named foo) |
| 1521 | * bar.foo_var = foo.foo_var |
| 1522 | |
| 1523 | The last step fails, because Python isn't done with interpreting ``foo`` yet and |
| 1524 | the global symbol dictionary for ``foo`` is still empty. |
| 1525 | |
| 1526 | The same thing happens when you use ``import foo``, and then try to access |
| 1527 | ``foo.foo_var`` in global code. |
| 1528 | |
| 1529 | There are (at least) three possible workarounds for this problem. |
| 1530 | |
| 1531 | Guido van Rossum recommends avoiding all uses of ``from <module> import ...``, |
| 1532 | and placing all code inside functions. Initializations of global variables and |
| 1533 | class variables should use constants or built-in functions only. This means |
| 1534 | everything from an imported module is referenced as ``<module>.<name>``. |
| 1535 | |
| 1536 | Jim Roskind suggests performing steps in the following order in each module: |
| 1537 | |
| 1538 | * exports (globals, functions, and classes that don't need imported base |
| 1539 | classes) |
| 1540 | * ``import`` statements |
| 1541 | * active code (including globals that are initialized from imported values). |
| 1542 | |
| 1543 | van Rossum doesn't like this approach much because the imports appear in a |
| 1544 | strange place, but it does work. |
| 1545 | |
| 1546 | Matthias Urlichs recommends restructuring your code so that the recursive import |
| 1547 | is not necessary in the first place. |
| 1548 | |
| 1549 | These solutions are not mutually exclusive. |
| 1550 | |
| 1551 | |
| 1552 | __import__('x.y.z') returns <module 'x'>; how do I get z? |
| 1553 | --------------------------------------------------------- |
| 1554 | |
| 1555 | Try:: |
| 1556 | |
| 1557 | __import__('x.y.z').y.z |
| 1558 | |
| 1559 | For more realistic situations, you may have to do something like :: |
| 1560 | |
| 1561 | m = __import__(s) |
| 1562 | for i in s.split(".")[1:]: |
| 1563 | m = getattr(m, i) |
| 1564 | |
| 1565 | See :mod:`importlib` for a convenience function called |
| 1566 | :func:`~importlib.import_module`. |
| 1567 | |
| 1568 | |
| 1569 | |
| 1570 | When I edit an imported module and reimport it, the changes don't show up. Why does this happen? |
| 1571 | ------------------------------------------------------------------------------------------------- |
| 1572 | |
| 1573 | For reasons of efficiency as well as consistency, Python only reads the module |
| 1574 | file on the first time a module is imported. If it didn't, in a program |
| 1575 | consisting of many modules where each one imports the same basic module, the |
| 1576 | basic module would be parsed and re-parsed many times. To force rereading of a |
| 1577 | changed module, do this:: |
| 1578 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1579 | import imp |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1580 | import modname |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1581 | imp.reload(modname) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1582 | |
| 1583 | Warning: this technique is not 100% fool-proof. In particular, modules |
| 1584 | containing statements like :: |
| 1585 | |
| 1586 | from modname import some_objects |
| 1587 | |
| 1588 | will continue to work with the old version of the imported objects. If the |
| 1589 | module contains class definitions, existing class instances will *not* be |
| 1590 | updated to use the new class definition. This can result in the following |
| 1591 | paradoxical behaviour: |
| 1592 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1593 | >>> import imp |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1594 | >>> import cls |
| 1595 | >>> c = cls.C() # Create an instance of C |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1596 | >>> imp.reload(cls) |
| 1597 | <module 'cls' from 'cls.py'> |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1598 | >>> isinstance(c, cls.C) # isinstance is false?!? |
| 1599 | False |
| 1600 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1601 | The nature of the problem is made clear if you print out the "identity" of the |
| 1602 | class objects: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1603 | |
Georg Brandl | 62eaaf6 | 2009-12-19 17:51:41 +0000 | [diff] [blame] | 1604 | >>> hex(id(c.__class__)) |
| 1605 | '0x7352a0' |
| 1606 | >>> hex(id(cls.C)) |
| 1607 | '0x4198d0' |