Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 1 | ====================== |
| 2 | Design and History FAQ |
| 3 | ====================== |
| 4 | |
| 5 | Why does Python use indentation for grouping of statements? |
| 6 | ----------------------------------------------------------- |
| 7 | |
| 8 | Guido van Rossum believes that using indentation for grouping is extremely |
| 9 | elegant and contributes a lot to the clarity of the average Python program. |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 10 | Most people learn to love this feature after a while. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 11 | |
| 12 | Since there are no begin/end brackets there cannot be a disagreement between |
| 13 | grouping perceived by the parser and the human reader. Occasionally C |
| 14 | programmers will encounter a fragment of code like this:: |
| 15 | |
| 16 | if (x <= y) |
| 17 | x++; |
| 18 | y--; |
| 19 | z++; |
| 20 | |
| 21 | Only the ``x++`` statement is executed if the condition is true, but the |
| 22 | indentation leads you to believe otherwise. Even experienced C programmers will |
| 23 | sometimes stare at it a long time wondering why ``y`` is being decremented even |
| 24 | for ``x > y``. |
| 25 | |
| 26 | Because there are no begin/end brackets, Python is much less prone to |
| 27 | coding-style conflicts. In C there are many different ways to place the braces. |
| 28 | If you're used to reading and writing code that uses one style, you will feel at |
| 29 | least slightly uneasy when reading (or being required to write) another style. |
| 30 | |
Georg Brandl | 6faee4e | 2010-09-21 14:48:28 +0000 | [diff] [blame] | 31 | Many coding styles place begin/end brackets on a line by themselves. This makes |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 32 | programs considerably longer and wastes valuable screen space, making it harder |
| 33 | to get a good overview of a program. Ideally, a function should fit on one |
| 34 | screen (say, 20-30 lines). 20 lines of Python can do a lot more work than 20 |
| 35 | lines of C. This is not solely due to the lack of begin/end brackets -- the |
| 36 | lack of declarations and the high-level data types are also responsible -- but |
| 37 | the indentation-based syntax certainly helps. |
| 38 | |
| 39 | |
| 40 | Why am I getting strange results with simple arithmetic operations? |
| 41 | ------------------------------------------------------------------- |
| 42 | |
| 43 | See the next question. |
| 44 | |
| 45 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 46 | Why are floating-point calculations so inaccurate? |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 47 | -------------------------------------------------- |
| 48 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 49 | Users are often surprised by results like this:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 50 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 51 | >>> 1.2 - 1.0 |
| 52 | 0.199999999999999996 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 53 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 54 | and think it is a bug in Python. It's not. This has little to do with Python, |
| 55 | and much more to do with how the underlying platform handles floating-point |
| 56 | numbers. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 57 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 58 | The :class:`float` type in CPython uses a C ``double`` for storage. A |
| 59 | :class:`float` object's value is stored in binary floating-point with a fixed |
| 60 | precision (typically 53 bits) and Python uses C operations, which in turn rely |
| 61 | on the hardware implementation in the processor, to perform floating-point |
| 62 | operations. This means that as far as floating-point operations are concerned, |
| 63 | Python behaves like many popular languages including C and Java. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 64 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 65 | Many numbers that can be written easily in decimal notation cannot be expressed |
| 66 | exactly in binary floating-point. For example, after:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 67 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 68 | >>> x = 1.2 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 69 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 70 | the value stored for ``x`` is a (very good) approximation to the decimal value |
| 71 | ``1.2``, but is not exactly equal to it. On a typical machine, the actual |
| 72 | stored value is:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 73 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 74 | 1.0011001100110011001100110011001100110011001100110011 (binary) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 75 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 76 | which is exactly:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 77 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 78 | 1.1999999999999999555910790149937383830547332763671875 (decimal) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 79 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 80 | The typical precision of 53 bits provides Python floats with 15-16 |
| 81 | decimal digits of accuracy. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 82 | |
Mark Dickinson | ba3b0d8 | 2012-05-13 21:00:35 +0100 | [diff] [blame] | 83 | For a fuller explanation, please see the :ref:`floating point arithmetic |
| 84 | <tut-fp-issues>` chapter in the Python tutorial. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 85 | |
| 86 | |
| 87 | Why are Python strings immutable? |
| 88 | --------------------------------- |
| 89 | |
| 90 | There are several advantages. |
| 91 | |
| 92 | One is performance: knowing that a string is immutable means we can allocate |
| 93 | space for it at creation time, and the storage requirements are fixed and |
| 94 | unchanging. This is also one of the reasons for the distinction between tuples |
| 95 | and lists. |
| 96 | |
| 97 | Another advantage is that strings in Python are considered as "elemental" as |
| 98 | numbers. No amount of activity will change the value 8 to anything else, and in |
| 99 | Python, no amount of activity will change the string "eight" to anything else. |
| 100 | |
| 101 | |
| 102 | .. _why-self: |
| 103 | |
| 104 | Why must 'self' be used explicitly in method definitions and calls? |
| 105 | ------------------------------------------------------------------- |
| 106 | |
| 107 | The idea was borrowed from Modula-3. It turns out to be very useful, for a |
| 108 | variety of reasons. |
| 109 | |
| 110 | First, it's more obvious that you are using a method or instance attribute |
| 111 | instead of a local variable. Reading ``self.x`` or ``self.meth()`` makes it |
| 112 | absolutely clear that an instance variable or method is used even if you don't |
| 113 | know the class definition by heart. In C++, you can sort of tell by the lack of |
| 114 | a local variable declaration (assuming globals are rare or easily recognizable) |
| 115 | -- but in Python, there are no local variable declarations, so you'd have to |
| 116 | look up the class definition to be sure. Some C++ and Java coding standards |
| 117 | call for instance attributes to have an ``m_`` prefix, so this explicitness is |
| 118 | still useful in those languages, too. |
| 119 | |
| 120 | Second, it means that no special syntax is necessary if you want to explicitly |
| 121 | reference or call the method from a particular class. In C++, if you want to |
| 122 | use a method from a base class which is overridden in a derived class, you have |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 123 | to use the ``::`` operator -- in Python you can write |
| 124 | ``baseclass.methodname(self, <argument list>)``. This is particularly useful |
| 125 | for :meth:`__init__` methods, and in general in cases where a derived class |
| 126 | method wants to extend the base class method of the same name and thus has to |
| 127 | call the base class method somehow. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 128 | |
| 129 | Finally, for instance variables it solves a syntactic problem with assignment: |
| 130 | since local variables in Python are (by definition!) those variables to which a |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 131 | value is assigned in a function body (and that aren't explicitly declared |
| 132 | global), there has to be some way to tell the interpreter that an assignment was |
| 133 | meant to assign to an instance variable instead of to a local variable, and it |
| 134 | should preferably be syntactic (for efficiency reasons). C++ does this through |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 135 | declarations, but Python doesn't have declarations and it would be a pity having |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 136 | to introduce them just for this purpose. Using the explicit ``self.var`` solves |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 137 | this nicely. Similarly, for using instance variables, having to write |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 138 | ``self.var`` means that references to unqualified names inside a method don't |
| 139 | have to search the instance's directories. To put it another way, local |
| 140 | variables and instance variables live in two different namespaces, and you need |
| 141 | to tell Python which namespace to use. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 142 | |
| 143 | |
| 144 | Why can't I use an assignment in an expression? |
| 145 | ----------------------------------------------- |
| 146 | |
| 147 | Many people used to C or Perl complain that they want to use this C idiom: |
| 148 | |
| 149 | .. code-block:: c |
| 150 | |
| 151 | while (line = readline(f)) { |
| 152 | // do something with line |
| 153 | } |
| 154 | |
| 155 | where in Python you're forced to write this:: |
| 156 | |
| 157 | while True: |
| 158 | line = f.readline() |
| 159 | if not line: |
| 160 | break |
| 161 | ... # do something with line |
| 162 | |
| 163 | The reason for not allowing assignment in Python expressions is a common, |
| 164 | hard-to-find bug in those other languages, caused by this construct: |
| 165 | |
| 166 | .. code-block:: c |
| 167 | |
| 168 | if (x = 0) { |
| 169 | // error handling |
| 170 | } |
| 171 | else { |
| 172 | // code that only works for nonzero x |
| 173 | } |
| 174 | |
| 175 | The error is a simple typo: ``x = 0``, which assigns 0 to the variable ``x``, |
| 176 | was written while the comparison ``x == 0`` is certainly what was intended. |
| 177 | |
| 178 | Many alternatives have been proposed. Most are hacks that save some typing but |
| 179 | use arbitrary or cryptic syntax or keywords, and fail the simple criterion for |
| 180 | language change proposals: it should intuitively suggest the proper meaning to a |
| 181 | human reader who has not yet been introduced to the construct. |
| 182 | |
| 183 | An interesting phenomenon is that most experienced Python programmers recognize |
| 184 | the ``while True`` idiom and don't seem to be missing the assignment in |
| 185 | expression construct much; it's only newcomers who express a strong desire to |
| 186 | add this to the language. |
| 187 | |
| 188 | There's an alternative way of spelling this that seems attractive but is |
| 189 | generally less robust than the "while True" solution:: |
| 190 | |
| 191 | line = f.readline() |
| 192 | while line: |
| 193 | ... # do something with line... |
| 194 | line = f.readline() |
| 195 | |
| 196 | The problem with this is that if you change your mind about exactly how you get |
| 197 | the next line (e.g. you want to change it into ``sys.stdin.readline()``) you |
| 198 | have to remember to change two places in your program -- the second occurrence |
| 199 | is hidden at the bottom of the loop. |
| 200 | |
| 201 | The best approach is to use iterators, making it possible to loop through |
Antoine Pitrou | 11cb961 | 2010-09-15 11:11:28 +0000 | [diff] [blame] | 202 | objects using the ``for`` statement. For example, :term:`file objects |
| 203 | <file object>` support the iterator protocol, so you can write simply:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 204 | |
| 205 | for line in f: |
| 206 | ... # do something with line... |
| 207 | |
| 208 | |
| 209 | |
| 210 | Why does Python use methods for some functionality (e.g. list.index()) but functions for other (e.g. len(list))? |
| 211 | ---------------------------------------------------------------------------------------------------------------- |
| 212 | |
| 213 | The major reason is history. Functions were used for those operations that were |
| 214 | generic for a group of types and which were intended to work even for objects |
| 215 | that didn't have methods at all (e.g. tuples). It is also convenient to have a |
| 216 | function that can readily be applied to an amorphous collection of objects when |
Ezio Melotti | 9beeefb | 2013-01-05 07:36:54 +0200 | [diff] [blame] | 217 | you use the functional features of Python (``map()``, ``zip()`` et al). |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 218 | |
| 219 | In fact, implementing ``len()``, ``max()``, ``min()`` as a built-in function is |
| 220 | actually less code than implementing them as methods for each type. One can |
| 221 | quibble about individual cases but it's a part of Python, and it's too late to |
| 222 | make such fundamental changes now. The functions have to remain to avoid massive |
| 223 | code breakage. |
| 224 | |
| 225 | .. XXX talk about protocols? |
| 226 | |
Georg Brandl | bfe95ac | 2009-12-19 17:46:40 +0000 | [diff] [blame] | 227 | .. note:: |
| 228 | |
| 229 | For string operations, Python has moved from external functions (the |
| 230 | ``string`` module) to methods. However, ``len()`` is still a function. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 231 | |
| 232 | |
| 233 | Why is join() a string method instead of a list or tuple method? |
| 234 | ---------------------------------------------------------------- |
| 235 | |
| 236 | Strings became much more like other standard types starting in Python 1.6, when |
| 237 | methods were added which give the same functionality that has always been |
| 238 | available using the functions of the string module. Most of these new methods |
| 239 | have been widely accepted, but the one which appears to make some programmers |
| 240 | feel uncomfortable is:: |
| 241 | |
| 242 | ", ".join(['1', '2', '4', '8', '16']) |
| 243 | |
| 244 | which gives the result:: |
| 245 | |
| 246 | "1, 2, 4, 8, 16" |
| 247 | |
| 248 | There are two common arguments against this usage. |
| 249 | |
| 250 | The first runs along the lines of: "It looks really ugly using a method of a |
| 251 | string literal (string constant)", to which the answer is that it might, but a |
| 252 | string literal is just a fixed value. If the methods are to be allowed on names |
| 253 | bound to strings there is no logical reason to make them unavailable on |
| 254 | literals. |
| 255 | |
| 256 | The second objection is typically cast as: "I am really telling a sequence to |
| 257 | join its members together with a string constant". Sadly, you aren't. For some |
| 258 | reason there seems to be much less difficulty with having :meth:`~str.split` as |
| 259 | a string method, since in that case it is easy to see that :: |
| 260 | |
| 261 | "1, 2, 4, 8, 16".split(", ") |
| 262 | |
| 263 | is an instruction to a string literal to return the substrings delimited by the |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 264 | given separator (or, by default, arbitrary runs of white space). |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 265 | |
| 266 | :meth:`~str.join` is a string method because in using it you are telling the |
| 267 | separator string to iterate over a sequence of strings and insert itself between |
| 268 | adjacent elements. This method can be used with any argument which obeys the |
| 269 | rules for sequence objects, including any new classes you might define yourself. |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 270 | Similar methods exist for bytes and bytearray objects. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 271 | |
| 272 | |
| 273 | How fast are exceptions? |
| 274 | ------------------------ |
| 275 | |
Georg Brandl | 12c3cd7 | 2012-03-17 16:58:05 +0100 | [diff] [blame] | 276 | A try/except block is extremely efficient if no exceptions are raised. Actually |
| 277 | catching an exception is expensive. In versions of Python prior to 2.0 it was |
| 278 | common to use this idiom:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 279 | |
| 280 | try: |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 281 | value = mydict[key] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 282 | except KeyError: |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 283 | mydict[key] = getvalue(key) |
| 284 | value = mydict[key] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 285 | |
| 286 | This only made sense when you expected the dict to have the key almost all the |
| 287 | time. If that wasn't the case, you coded it like this:: |
| 288 | |
Georg Brandl | 12c3cd7 | 2012-03-17 16:58:05 +0100 | [diff] [blame] | 289 | if key in mydict: |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 290 | value = mydict[key] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 291 | else: |
Georg Brandl | 12c3cd7 | 2012-03-17 16:58:05 +0100 | [diff] [blame] | 292 | value = mydict[key] = getvalue(key) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 293 | |
Georg Brandl | bfe95ac | 2009-12-19 17:46:40 +0000 | [diff] [blame] | 294 | For this specific case, you could also use ``value = dict.setdefault(key, |
| 295 | getvalue(key))``, but only if the ``getvalue()`` call is cheap enough because it |
| 296 | is evaluated in all cases. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 297 | |
| 298 | |
| 299 | Why isn't there a switch or case statement in Python? |
| 300 | ----------------------------------------------------- |
| 301 | |
| 302 | You can do this easily enough with a sequence of ``if... elif... elif... else``. |
| 303 | There have been some proposals for switch statement syntax, but there is no |
| 304 | consensus (yet) on whether and how to do range tests. See :pep:`275` for |
| 305 | complete details and the current status. |
| 306 | |
| 307 | For cases where you need to choose from a very large number of possibilities, |
| 308 | you can create a dictionary mapping case values to functions to call. For |
| 309 | example:: |
| 310 | |
| 311 | def function_1(...): |
| 312 | ... |
| 313 | |
| 314 | functions = {'a': function_1, |
| 315 | 'b': function_2, |
| 316 | 'c': self.method_1, ...} |
| 317 | |
| 318 | func = functions[value] |
| 319 | func() |
| 320 | |
| 321 | For calling methods on objects, you can simplify yet further by using the |
| 322 | :func:`getattr` built-in to retrieve methods with a particular name:: |
| 323 | |
| 324 | def visit_a(self, ...): |
| 325 | ... |
| 326 | ... |
| 327 | |
| 328 | def dispatch(self, value): |
| 329 | method_name = 'visit_' + str(value) |
| 330 | method = getattr(self, method_name) |
| 331 | method() |
| 332 | |
| 333 | It's suggested that you use a prefix for the method names, such as ``visit_`` in |
| 334 | this example. Without such a prefix, if values are coming from an untrusted |
| 335 | source, an attacker would be able to call any method on your object. |
| 336 | |
| 337 | |
| 338 | Can't you emulate threads in the interpreter instead of relying on an OS-specific thread implementation? |
| 339 | -------------------------------------------------------------------------------------------------------- |
| 340 | |
| 341 | Answer 1: Unfortunately, the interpreter pushes at least one C stack frame for |
| 342 | each Python stack frame. Also, extensions can call back into Python at almost |
| 343 | random moments. Therefore, a complete threads implementation requires thread |
| 344 | support for C. |
| 345 | |
| 346 | Answer 2: Fortunately, there is `Stackless Python <http://www.stackless.com>`_, |
| 347 | which has a completely redesigned interpreter loop that avoids the C stack. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 348 | |
| 349 | |
| 350 | Why can't lambda forms contain statements? |
| 351 | ------------------------------------------ |
| 352 | |
| 353 | Python lambda forms cannot contain statements because Python's syntactic |
| 354 | framework can't handle statements nested inside expressions. However, in |
| 355 | Python, this is not a serious problem. Unlike lambda forms in other languages, |
| 356 | where they add functionality, Python lambdas are only a shorthand notation if |
| 357 | you're too lazy to define a function. |
| 358 | |
| 359 | Functions are already first class objects in Python, and can be declared in a |
| 360 | local scope. Therefore the only advantage of using a lambda form instead of a |
| 361 | locally-defined function is that you don't need to invent a name for the |
| 362 | function -- but that's just a local variable to which the function object (which |
| 363 | is exactly the same type of object that a lambda form yields) is assigned! |
| 364 | |
| 365 | |
| 366 | Can Python be compiled to machine code, C or some other language? |
| 367 | ----------------------------------------------------------------- |
| 368 | |
Antoine Pitrou | 17bd792 | 2011-12-03 22:56:02 +0100 | [diff] [blame] | 369 | Practical answer: |
| 370 | |
| 371 | `Cython <http://cython.org/>`_ and `Pyrex <http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/>`_ |
| 372 | compile a modified version of Python with optional annotations into C |
| 373 | extensions. `Weave <http://www.scipy.org/Weave>`_ makes it easy to |
| 374 | intermingle Python and C code in various ways to increase performance. |
| 375 | `Nuitka <http://www.nuitka.net/>`_ is an up-and-coming compiler of Python |
| 376 | into C++ code, aiming to support the full Python language. |
| 377 | |
| 378 | Theoretical answer: |
| 379 | |
| 380 | .. XXX not sure what to make of this |
| 381 | |
| 382 | Not trivially. Python's high level data types, dynamic typing of objects and |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 383 | run-time invocation of the interpreter (using :func:`eval` or :func:`exec`) |
Antoine Pitrou | 17bd792 | 2011-12-03 22:56:02 +0100 | [diff] [blame] | 384 | together mean that a naïvely "compiled" Python program would probably consist |
| 385 | mostly of calls into the Python run-time system, even for seemingly simple |
| 386 | operations like ``x+1``. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 387 | |
| 388 | Several projects described in the Python newsgroup or at past `Python |
Georg Brandl | 495f7b5 | 2009-10-27 15:28:25 +0000 | [diff] [blame] | 389 | conferences <http://python.org/community/workshops/>`_ have shown that this |
| 390 | approach is feasible, although the speedups reached so far are only modest |
| 391 | (e.g. 2x). Jython uses the same strategy for compiling to Java bytecode. (Jim |
| 392 | Hugunin has demonstrated that in combination with whole-program analysis, |
| 393 | speedups of 1000x are feasible for small demo programs. See the proceedings |
| 394 | from the `1997 Python conference |
| 395 | <http://python.org/workshops/1997-10/proceedings/>`_ for more information.) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 396 | |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 397 | |
| 398 | How does Python manage memory? |
| 399 | ------------------------------ |
| 400 | |
| 401 | The details of Python memory management depend on the implementation. The |
Antoine Pitrou | c561a9a | 2011-12-03 23:06:50 +0100 | [diff] [blame] | 402 | standard implementation of Python, :term:`CPython`, uses reference counting to |
| 403 | detect inaccessible objects, and another mechanism to collect reference cycles, |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 404 | periodically executing a cycle detection algorithm which looks for inaccessible |
| 405 | cycles and deletes the objects involved. The :mod:`gc` module provides functions |
| 406 | to perform a garbage collection, obtain debugging statistics, and tune the |
| 407 | collector's parameters. |
| 408 | |
Antoine Pitrou | c561a9a | 2011-12-03 23:06:50 +0100 | [diff] [blame] | 409 | Other implementations (such as `Jython <http://www.jython.org>`_ or |
| 410 | `PyPy <http://www.pypy.org>`_), however, can rely on a different mechanism |
| 411 | such as a full-blown garbage collector. This difference can cause some |
| 412 | subtle porting problems if your Python code depends on the behavior of the |
| 413 | reference counting implementation. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 414 | |
Antoine Pitrou | c561a9a | 2011-12-03 23:06:50 +0100 | [diff] [blame] | 415 | In some Python implementations, the following code (which is fine in CPython) |
| 416 | will probably run out of file descriptors:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 417 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 418 | for file in very_long_list_of_files: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 419 | f = open(file) |
| 420 | c = f.read(1) |
| 421 | |
Antoine Pitrou | c561a9a | 2011-12-03 23:06:50 +0100 | [diff] [blame] | 422 | Indeed, using CPython's reference counting and destructor scheme, each new |
| 423 | assignment to *f* closes the previous file. With a traditional GC, however, |
| 424 | those file objects will only get collected (and closed) at varying and possibly |
| 425 | long intervals. |
| 426 | |
| 427 | If you want to write code that will work with any Python implementation, |
| 428 | you should explicitly close the file or use the :keyword:`with` statement; |
| 429 | this will work regardless of memory management scheme:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 430 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 431 | for file in very_long_list_of_files: |
| 432 | with open(file) as f: |
| 433 | c = f.read(1) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 434 | |
| 435 | |
Antoine Pitrou | c561a9a | 2011-12-03 23:06:50 +0100 | [diff] [blame] | 436 | Why doesn't CPython use a more traditional garbage collection scheme? |
| 437 | --------------------------------------------------------------------- |
| 438 | |
| 439 | For one thing, this is not a C standard feature and hence it's not portable. |
| 440 | (Yes, we know about the Boehm GC library. It has bits of assembler code for |
| 441 | *most* common platforms, not for all of them, and although it is mostly |
| 442 | transparent, it isn't completely transparent; patches are required to get |
| 443 | Python to work with it.) |
| 444 | |
| 445 | Traditional GC also becomes a problem when Python is embedded into other |
| 446 | applications. While in a standalone Python it's fine to replace the standard |
| 447 | malloc() and free() with versions provided by the GC library, an application |
| 448 | embedding Python may want to have its *own* substitute for malloc() and free(), |
| 449 | and may not want Python's. Right now, CPython works with anything that |
| 450 | implements malloc() and free() properly. |
| 451 | |
| 452 | |
| 453 | Why isn't all memory freed when CPython exits? |
| 454 | ---------------------------------------------- |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 455 | |
| 456 | Objects referenced from the global namespaces of Python modules are not always |
| 457 | deallocated when Python exits. This may happen if there are circular |
| 458 | references. There are also certain bits of memory that are allocated by the C |
| 459 | library that are impossible to free (e.g. a tool like Purify will complain about |
| 460 | these). Python is, however, aggressive about cleaning up memory on exit and |
| 461 | does try to destroy every single object. |
| 462 | |
| 463 | If you want to force Python to delete certain things on deallocation use the |
| 464 | :mod:`atexit` module to run a function that will force those deletions. |
| 465 | |
| 466 | |
| 467 | Why are there separate tuple and list data types? |
| 468 | ------------------------------------------------- |
| 469 | |
| 470 | Lists and tuples, while similar in many respects, are generally used in |
| 471 | fundamentally different ways. Tuples can be thought of as being similar to |
| 472 | Pascal records or C structs; they're small collections of related data which may |
| 473 | be of different types which are operated on as a group. For example, a |
| 474 | Cartesian coordinate is appropriately represented as a tuple of two or three |
| 475 | numbers. |
| 476 | |
| 477 | Lists, on the other hand, are more like arrays in other languages. They tend to |
| 478 | hold a varying number of objects all of which have the same type and which are |
| 479 | operated on one-by-one. For example, ``os.listdir('.')`` returns a list of |
| 480 | strings representing the files in the current directory. Functions which |
| 481 | operate on this output would generally not break if you added another file or |
| 482 | two to the directory. |
| 483 | |
| 484 | Tuples are immutable, meaning that once a tuple has been created, you can't |
| 485 | replace any of its elements with a new value. Lists are mutable, meaning that |
| 486 | you can always change a list's elements. Only immutable elements can be used as |
| 487 | dictionary keys, and hence only tuples and not lists can be used as keys. |
| 488 | |
| 489 | |
| 490 | How are lists implemented? |
| 491 | -------------------------- |
| 492 | |
| 493 | Python's lists are really variable-length arrays, not Lisp-style linked lists. |
| 494 | The implementation uses a contiguous array of references to other objects, and |
| 495 | keeps a pointer to this array and the array's length in a list head structure. |
| 496 | |
| 497 | This makes indexing a list ``a[i]`` an operation whose cost is independent of |
| 498 | the size of the list or the value of the index. |
| 499 | |
| 500 | When items are appended or inserted, the array of references is resized. Some |
| 501 | cleverness is applied to improve the performance of appending items repeatedly; |
| 502 | when the array must be grown, some extra space is allocated so the next few |
| 503 | times don't require an actual resize. |
| 504 | |
| 505 | |
| 506 | How are dictionaries implemented? |
| 507 | --------------------------------- |
| 508 | |
| 509 | Python's dictionaries are implemented as resizable hash tables. Compared to |
| 510 | B-trees, this gives better performance for lookup (the most common operation by |
| 511 | far) under most circumstances, and the implementation is simpler. |
| 512 | |
| 513 | Dictionaries work by computing a hash code for each key stored in the dictionary |
| 514 | using the :func:`hash` built-in function. The hash code varies widely depending |
Georg Brandl | b20a019 | 2012-03-14 07:50:17 +0100 | [diff] [blame] | 515 | on the key and a per-process seed; for example, "Python" could hash to |
| 516 | -539294296 while "python", a string that differs by a single bit, could hash |
| 517 | to 1142331976. The hash code is then used to calculate a location in an |
| 518 | internal array where the value will be stored. Assuming that you're storing |
| 519 | keys that all have different hash values, this means that dictionaries take |
| 520 | constant time -- O(1), in computer science notation -- to retrieve a key. It |
| 521 | also means that no sorted order of the keys is maintained, and traversing the |
| 522 | array as the ``.keys()`` and ``.items()`` do will output the dictionary's |
| 523 | content in some arbitrary jumbled order that can change with every invocation of |
| 524 | a program. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 525 | |
| 526 | |
| 527 | Why must dictionary keys be immutable? |
| 528 | -------------------------------------- |
| 529 | |
| 530 | The hash table implementation of dictionaries uses a hash value calculated from |
| 531 | the key value to find the key. If the key were a mutable object, its value |
| 532 | could change, and thus its hash could also change. But since whoever changes |
| 533 | the key object can't tell that it was being used as a dictionary key, it can't |
| 534 | move the entry around in the dictionary. Then, when you try to look up the same |
| 535 | object in the dictionary it won't be found because its hash value is different. |
| 536 | If you tried to look up the old value it wouldn't be found either, because the |
| 537 | value of the object found in that hash bin would be different. |
| 538 | |
| 539 | If you want a dictionary indexed with a list, simply convert the list to a tuple |
| 540 | first; the function ``tuple(L)`` creates a tuple with the same entries as the |
| 541 | list ``L``. Tuples are immutable and can therefore be used as dictionary keys. |
| 542 | |
| 543 | Some unacceptable solutions that have been proposed: |
| 544 | |
| 545 | - Hash lists by their address (object ID). This doesn't work because if you |
| 546 | construct a new list with the same value it won't be found; e.g.:: |
| 547 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 548 | mydict = {[1, 2]: '12'} |
| 549 | print(mydict[[1, 2]]) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 550 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 551 | would raise a KeyError exception because the id of the ``[1, 2]`` used in the |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 552 | second line differs from that in the first line. In other words, dictionary |
| 553 | keys should be compared using ``==``, not using :keyword:`is`. |
| 554 | |
| 555 | - Make a copy when using a list as a key. This doesn't work because the list, |
| 556 | being a mutable object, could contain a reference to itself, and then the |
| 557 | copying code would run into an infinite loop. |
| 558 | |
| 559 | - Allow lists as keys but tell the user not to modify them. This would allow a |
| 560 | class of hard-to-track bugs in programs when you forgot or modified a list by |
| 561 | accident. It also invalidates an important invariant of dictionaries: every |
| 562 | value in ``d.keys()`` is usable as a key of the dictionary. |
| 563 | |
| 564 | - Mark lists as read-only once they are used as a dictionary key. The problem |
| 565 | is that it's not just the top-level object that could change its value; you |
| 566 | could use a tuple containing a list as a key. Entering anything as a key into |
| 567 | a dictionary would require marking all objects reachable from there as |
| 568 | read-only -- and again, self-referential objects could cause an infinite loop. |
| 569 | |
| 570 | There is a trick to get around this if you need to, but use it at your own risk: |
| 571 | You can wrap a mutable structure inside a class instance which has both a |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 572 | :meth:`__eq__` and a :meth:`__hash__` method. You must then make sure that the |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 573 | hash value for all such wrapper objects that reside in a dictionary (or other |
| 574 | hash based structure), remain fixed while the object is in the dictionary (or |
| 575 | other structure). :: |
| 576 | |
| 577 | class ListWrapper: |
| 578 | def __init__(self, the_list): |
| 579 | self.the_list = the_list |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 580 | def __eq__(self, other): |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 581 | return self.the_list == other.the_list |
| 582 | def __hash__(self): |
| 583 | l = self.the_list |
| 584 | result = 98767 - len(l)*555 |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 585 | for i, el in enumerate(l): |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 586 | try: |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 587 | result = result + (hash(el) % 9999999) * 1001 + i |
| 588 | except Exception: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 589 | result = (result % 7777777) + i * 333 |
| 590 | return result |
| 591 | |
| 592 | Note that the hash computation is complicated by the possibility that some |
| 593 | members of the list may be unhashable and also by the possibility of arithmetic |
| 594 | overflow. |
| 595 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 596 | Furthermore it must always be the case that if ``o1 == o2`` (ie ``o1.__eq__(o2) |
| 597 | is True``) then ``hash(o1) == hash(o2)`` (ie, ``o1.__hash__() == o2.__hash__()``), |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 598 | regardless of whether the object is in a dictionary or not. If you fail to meet |
| 599 | these restrictions dictionaries and other hash based structures will misbehave. |
| 600 | |
| 601 | In the case of ListWrapper, whenever the wrapper object is in a dictionary the |
| 602 | wrapped list must not change to avoid anomalies. Don't do this unless you are |
| 603 | prepared to think hard about the requirements and the consequences of not |
| 604 | meeting them correctly. Consider yourself warned. |
| 605 | |
| 606 | |
| 607 | Why doesn't list.sort() return the sorted list? |
| 608 | ----------------------------------------------- |
| 609 | |
| 610 | In situations where performance matters, making a copy of the list just to sort |
| 611 | it would be wasteful. Therefore, :meth:`list.sort` sorts the list in place. In |
| 612 | order to remind you of that fact, it does not return the sorted list. This way, |
| 613 | you won't be fooled into accidentally overwriting a list when you need a sorted |
| 614 | copy but also need to keep the unsorted version around. |
| 615 | |
Antoine Pitrou | dec0f21 | 2011-12-03 23:08:57 +0100 | [diff] [blame] | 616 | If you want to return a new list, use the built-in :func:`sorted` function |
| 617 | instead. This function creates a new list from a provided iterable, sorts |
| 618 | it and returns it. For example, here's how to iterate over the keys of a |
| 619 | dictionary in sorted order:: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 620 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 621 | for key in sorted(mydict): |
| 622 | ... # do whatever with mydict[key]... |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 623 | |
| 624 | |
| 625 | How do you specify and enforce an interface spec in Python? |
| 626 | ----------------------------------------------------------- |
| 627 | |
| 628 | An interface specification for a module as provided by languages such as C++ and |
| 629 | Java describes the prototypes for the methods and functions of the module. Many |
| 630 | feel that compile-time enforcement of interface specifications helps in the |
| 631 | construction of large programs. |
| 632 | |
| 633 | Python 2.6 adds an :mod:`abc` module that lets you define Abstract Base Classes |
| 634 | (ABCs). You can then use :func:`isinstance` and :func:`issubclass` to check |
| 635 | whether an instance or a class implements a particular ABC. The |
Éric Araujo | b8edbdf | 2011-09-01 05:57:12 +0200 | [diff] [blame] | 636 | :mod:`collections.abc` module defines a set of useful ABCs such as |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 637 | :class:`Iterable`, :class:`Container`, and :class:`MutableMapping`. |
| 638 | |
| 639 | For Python, many of the advantages of interface specifications can be obtained |
| 640 | by an appropriate test discipline for components. There is also a tool, |
| 641 | PyChecker, which can be used to find problems due to subclassing. |
| 642 | |
| 643 | A good test suite for a module can both provide a regression test and serve as a |
| 644 | module interface specification and a set of examples. Many Python modules can |
| 645 | be run as a script to provide a simple "self test." Even modules which use |
| 646 | complex external interfaces can often be tested in isolation using trivial |
| 647 | "stub" emulations of the external interface. The :mod:`doctest` and |
| 648 | :mod:`unittest` modules or third-party test frameworks can be used to construct |
| 649 | exhaustive test suites that exercise every line of code in a module. |
| 650 | |
| 651 | An appropriate testing discipline can help build large complex applications in |
| 652 | Python as well as having interface specifications would. In fact, it can be |
| 653 | better because an interface specification cannot test certain properties of a |
| 654 | program. For example, the :meth:`append` method is expected to add new elements |
| 655 | to the end of some internal list; an interface specification cannot test that |
| 656 | your :meth:`append` implementation will actually do this correctly, but it's |
| 657 | trivial to check this property in a test suite. |
| 658 | |
| 659 | Writing test suites is very helpful, and you might want to design your code with |
| 660 | an eye to making it easily tested. One increasingly popular technique, |
| 661 | test-directed development, calls for writing parts of the test suite first, |
| 662 | before you write any of the actual code. Of course Python allows you to be |
| 663 | sloppy and not write test cases at all. |
| 664 | |
| 665 | |
| 666 | Why are default values shared between objects? |
| 667 | ---------------------------------------------- |
| 668 | |
| 669 | This type of bug commonly bites neophyte programmers. Consider this function:: |
| 670 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 671 | def foo(mydict={}): # Danger: shared reference to one dict for all calls |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 672 | ... compute something ... |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 673 | mydict[key] = value |
| 674 | return mydict |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 675 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 676 | The first time you call this function, ``mydict`` contains a single item. The |
| 677 | second time, ``mydict`` contains two items because when ``foo()`` begins |
| 678 | executing, ``mydict`` starts out with an item already in it. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 679 | |
| 680 | It is often expected that a function call creates new objects for default |
| 681 | values. This is not what happens. Default values are created exactly once, when |
| 682 | the function is defined. If that object is changed, like the dictionary in this |
| 683 | example, subsequent calls to the function will refer to this changed object. |
| 684 | |
| 685 | By definition, immutable objects such as numbers, strings, tuples, and ``None``, |
| 686 | are safe from change. Changes to mutable objects such as dictionaries, lists, |
| 687 | and class instances can lead to confusion. |
| 688 | |
| 689 | Because of this feature, it is good programming practice to not use mutable |
| 690 | objects as default values. Instead, use ``None`` as the default value and |
| 691 | inside the function, check if the parameter is ``None`` and create a new |
| 692 | list/dictionary/whatever if it is. For example, don't write:: |
| 693 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 694 | def foo(mydict={}): |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 695 | ... |
| 696 | |
| 697 | but:: |
| 698 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 699 | def foo(mydict=None): |
| 700 | if mydict is None: |
| 701 | mydict = {} # create a new dict for local namespace |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 702 | |
| 703 | This feature can be useful. When you have a function that's time-consuming to |
| 704 | compute, a common technique is to cache the parameters and the resulting value |
| 705 | of each call to the function, and return the cached value if the same value is |
| 706 | requested again. This is called "memoizing", and can be implemented like this:: |
| 707 | |
| 708 | # Callers will never provide a third parameter for this function. |
Ezio Melotti | 9beeefb | 2013-01-05 07:36:54 +0200 | [diff] [blame] | 709 | def expensive(arg1, arg2, _cache={}): |
Georg Brandl | bfe95ac | 2009-12-19 17:46:40 +0000 | [diff] [blame] | 710 | if (arg1, arg2) in _cache: |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 711 | return _cache[(arg1, arg2)] |
| 712 | |
| 713 | # Calculate the value |
| 714 | result = ... expensive computation ... |
| 715 | _cache[(arg1, arg2)] = result # Store result in the cache |
| 716 | return result |
| 717 | |
| 718 | You could use a global variable containing a dictionary instead of the default |
| 719 | value; it's a matter of taste. |
| 720 | |
| 721 | |
| 722 | Why is there no goto? |
| 723 | --------------------- |
| 724 | |
| 725 | You can use exceptions to provide a "structured goto" that even works across |
| 726 | function calls. Many feel that exceptions can conveniently emulate all |
| 727 | reasonable uses of the "go" or "goto" constructs of C, Fortran, and other |
| 728 | languages. For example:: |
| 729 | |
Ezio Melotti | 19cdee8 | 2013-01-05 06:53:27 +0200 | [diff] [blame] | 730 | class label(Exception): pass # declare a label |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 731 | |
| 732 | try: |
| 733 | ... |
Ezio Melotti | 9beeefb | 2013-01-05 07:36:54 +0200 | [diff] [blame] | 734 | if condition: raise label() # goto label |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 735 | ... |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 736 | except label: # where to goto |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 737 | pass |
| 738 | ... |
| 739 | |
| 740 | This doesn't allow you to jump into the middle of a loop, but that's usually |
| 741 | considered an abuse of goto anyway. Use sparingly. |
| 742 | |
| 743 | |
| 744 | Why can't raw strings (r-strings) end with a backslash? |
| 745 | ------------------------------------------------------- |
| 746 | |
| 747 | More precisely, they can't end with an odd number of backslashes: the unpaired |
| 748 | backslash at the end escapes the closing quote character, leaving an |
| 749 | unterminated string. |
| 750 | |
| 751 | Raw strings were designed to ease creating input for processors (chiefly regular |
| 752 | expression engines) that want to do their own backslash escape processing. Such |
| 753 | processors consider an unmatched trailing backslash to be an error anyway, so |
| 754 | raw strings disallow that. In return, they allow you to pass on the string |
| 755 | quote character by escaping it with a backslash. These rules work well when |
| 756 | r-strings are used for their intended purpose. |
| 757 | |
| 758 | If you're trying to build Windows pathnames, note that all Windows system calls |
| 759 | accept forward slashes too:: |
| 760 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 761 | f = open("/mydir/file.txt") # works fine! |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 762 | |
| 763 | If you're trying to build a pathname for a DOS command, try e.g. one of :: |
| 764 | |
| 765 | dir = r"\this\is\my\dos\dir" "\\" |
| 766 | dir = r"\this\is\my\dos\dir\ "[:-1] |
| 767 | dir = "\\this\\is\\my\\dos\\dir\\" |
| 768 | |
| 769 | |
| 770 | Why doesn't Python have a "with" statement for attribute assignments? |
| 771 | --------------------------------------------------------------------- |
| 772 | |
| 773 | Python has a 'with' statement that wraps the execution of a block, calling code |
| 774 | on the entrance and exit from the block. Some language have a construct that |
| 775 | looks like this:: |
| 776 | |
| 777 | with obj: |
Benjamin Peterson | 1baf465 | 2009-12-31 03:11:23 +0000 | [diff] [blame] | 778 | a = 1 # equivalent to obj.a = 1 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 779 | total = total + 1 # obj.total = obj.total + 1 |
| 780 | |
| 781 | In Python, such a construct would be ambiguous. |
| 782 | |
| 783 | Other languages, such as Object Pascal, Delphi, and C++, use static types, so |
| 784 | it's possible to know, in an unambiguous way, what member is being assigned |
| 785 | to. This is the main point of static typing -- the compiler *always* knows the |
| 786 | scope of every variable at compile time. |
| 787 | |
| 788 | Python uses dynamic types. It is impossible to know in advance which attribute |
| 789 | will be referenced at runtime. Member attributes may be added or removed from |
| 790 | objects on the fly. This makes it impossible to know, from a simple reading, |
| 791 | what attribute is being referenced: a local one, a global one, or a member |
| 792 | attribute? |
| 793 | |
| 794 | For instance, take the following incomplete snippet:: |
| 795 | |
| 796 | def foo(a): |
| 797 | with a: |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 798 | print(x) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 799 | |
| 800 | The snippet assumes that "a" must have a member attribute called "x". However, |
| 801 | there is nothing in Python that tells the interpreter this. What should happen |
| 802 | if "a" is, let us say, an integer? If there is a global variable named "x", |
| 803 | will it be used inside the with block? As you see, the dynamic nature of Python |
| 804 | makes such choices much harder. |
| 805 | |
| 806 | The primary benefit of "with" and similar language features (reduction of code |
| 807 | volume) can, however, easily be achieved in Python by assignment. Instead of:: |
| 808 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 809 | function(args).mydict[index][index].a = 21 |
| 810 | function(args).mydict[index][index].b = 42 |
| 811 | function(args).mydict[index][index].c = 63 |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 812 | |
| 813 | write this:: |
| 814 | |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 815 | ref = function(args).mydict[index][index] |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 816 | ref.a = 21 |
| 817 | ref.b = 42 |
| 818 | ref.c = 63 |
| 819 | |
| 820 | This also has the side-effect of increasing execution speed because name |
| 821 | bindings are resolved at run-time in Python, and the second version only needs |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 822 | to perform the resolution once. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 823 | |
| 824 | |
| 825 | Why are colons required for the if/while/def/class statements? |
| 826 | -------------------------------------------------------------- |
| 827 | |
| 828 | The colon is required primarily to enhance readability (one of the results of |
| 829 | the experimental ABC language). Consider this:: |
| 830 | |
| 831 | if a == b |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 832 | print(a) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 833 | |
| 834 | versus :: |
| 835 | |
| 836 | if a == b: |
Georg Brandl | 99de488 | 2009-12-20 14:24:06 +0000 | [diff] [blame] | 837 | print(a) |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 838 | |
| 839 | Notice how the second one is slightly easier to read. Notice further how a |
| 840 | colon sets off the example in this FAQ answer; it's a standard usage in English. |
| 841 | |
| 842 | Another minor reason is that the colon makes it easier for editors with syntax |
| 843 | highlighting; they can look for colons to decide when indentation needs to be |
| 844 | increased instead of having to do a more elaborate parsing of the program text. |
| 845 | |
| 846 | |
| 847 | Why does Python allow commas at the end of lists and tuples? |
| 848 | ------------------------------------------------------------ |
| 849 | |
| 850 | Python lets you add a trailing comma at the end of lists, tuples, and |
| 851 | dictionaries:: |
| 852 | |
| 853 | [1, 2, 3,] |
| 854 | ('a', 'b', 'c',) |
| 855 | d = { |
| 856 | "A": [1, 5], |
| 857 | "B": [6, 7], # last trailing comma is optional but good style |
| 858 | } |
| 859 | |
| 860 | |
| 861 | There are several reasons to allow this. |
| 862 | |
| 863 | When you have a literal value for a list, tuple, or dictionary spread across |
| 864 | multiple lines, it's easier to add more elements because you don't have to |
Georg Brandl | 7b8c132 | 2013-04-14 10:31:06 +0200 | [diff] [blame^] | 865 | remember to add a comma to the previous line. The lines can also be reordered |
| 866 | without creating a syntax error. |
Georg Brandl | d741315 | 2009-10-11 21:25:26 +0000 | [diff] [blame] | 867 | |
| 868 | Accidentally omitting the comma can lead to errors that are hard to diagnose. |
| 869 | For example:: |
| 870 | |
| 871 | x = [ |
| 872 | "fee", |
| 873 | "fie" |
| 874 | "foo", |
| 875 | "fum" |
| 876 | ] |
| 877 | |
| 878 | This list looks like it has four elements, but it actually contains three: |
| 879 | "fee", "fiefoo" and "fum". Always adding the comma avoids this source of error. |
| 880 | |
| 881 | Allowing the trailing comma may also make programmatic code generation easier. |