Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | .. _tut-brieftour: |
| 2 | |
| 3 | ********************************** |
| 4 | Brief Tour of the Standard Library |
| 5 | ********************************** |
| 6 | |
| 7 | |
| 8 | .. _tut-os-interface: |
| 9 | |
| 10 | Operating System Interface |
| 11 | ========================== |
| 12 | |
| 13 | The :mod:`os` module provides dozens of functions for interacting with the |
| 14 | operating system:: |
| 15 | |
| 16 | >>> import os |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 17 | >>> os.getcwd() # Return the current working directory |
Barry Warsaw | 97f005d | 2008-12-03 16:46:14 +0000 | [diff] [blame] | 18 | 'C:\\Python31' |
Georg Brandl | fe12390 | 2010-11-26 12:12:14 +0000 | [diff] [blame] | 19 | >>> os.chdir('/server/accesslogs') # Change current working directory |
| 20 | >>> os.system('mkdir today') # Run the command mkdir in the system shell |
| 21 | 0 |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 22 | |
| 23 | Be sure to use the ``import os`` style instead of ``from os import *``. This |
Mark Dickinson | 934896d | 2009-02-21 20:59:32 +0000 | [diff] [blame] | 24 | will keep :func:`os.open` from shadowing the built-in :func:`open` function which |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 25 | operates much differently. |
| 26 | |
| 27 | .. index:: builtin: help |
| 28 | |
Mark Dickinson | 934896d | 2009-02-21 20:59:32 +0000 | [diff] [blame] | 29 | The built-in :func:`dir` and :func:`help` functions are useful as interactive |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 30 | aids for working with large modules like :mod:`os`:: |
| 31 | |
| 32 | >>> import os |
| 33 | >>> dir(os) |
| 34 | <returns a list of all module functions> |
| 35 | >>> help(os) |
| 36 | <returns an extensive manual page created from the module's docstrings> |
| 37 | |
| 38 | For daily file and directory management tasks, the :mod:`shutil` module provides |
| 39 | a higher level interface that is easier to use:: |
| 40 | |
| 41 | >>> import shutil |
| 42 | >>> shutil.copyfile('data.db', 'archive.db') |
| 43 | >>> shutil.move('/build/executables', 'installdir') |
| 44 | |
| 45 | |
| 46 | .. _tut-file-wildcards: |
| 47 | |
| 48 | File Wildcards |
| 49 | ============== |
| 50 | |
| 51 | The :mod:`glob` module provides a function for making file lists from directory |
| 52 | wildcard searches:: |
| 53 | |
| 54 | >>> import glob |
| 55 | >>> glob.glob('*.py') |
| 56 | ['primes.py', 'random.py', 'quote.py'] |
| 57 | |
| 58 | |
| 59 | .. _tut-command-line-arguments: |
| 60 | |
| 61 | Command Line Arguments |
| 62 | ====================== |
| 63 | |
| 64 | Common utility scripts often need to process command line arguments. These |
| 65 | arguments are stored in the :mod:`sys` module's *argv* attribute as a list. For |
| 66 | instance the following output results from running ``python demo.py one two |
| 67 | three`` at the command line:: |
| 68 | |
| 69 | >>> import sys |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 70 | >>> print(sys.argv) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 71 | ['demo.py', 'one', 'two', 'three'] |
| 72 | |
| 73 | The :mod:`getopt` module processes *sys.argv* using the conventions of the Unix |
| 74 | :func:`getopt` function. More powerful and flexible command line processing is |
Steven Bethard | 6d26569 | 2010-03-02 09:22:57 +0000 | [diff] [blame] | 75 | provided by the :mod:`argparse` module. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 76 | |
| 77 | |
| 78 | .. _tut-stderr: |
| 79 | |
| 80 | Error Output Redirection and Program Termination |
| 81 | ================================================ |
| 82 | |
| 83 | The :mod:`sys` module also has attributes for *stdin*, *stdout*, and *stderr*. |
| 84 | The latter is useful for emitting warnings and error messages to make them |
| 85 | visible even when *stdout* has been redirected:: |
| 86 | |
| 87 | >>> sys.stderr.write('Warning, log file not found starting a new one\n') |
| 88 | Warning, log file not found starting a new one |
| 89 | |
| 90 | The most direct way to terminate a script is to use ``sys.exit()``. |
| 91 | |
| 92 | |
| 93 | .. _tut-string-pattern-matching: |
| 94 | |
| 95 | String Pattern Matching |
| 96 | ======================= |
| 97 | |
| 98 | The :mod:`re` module provides regular expression tools for advanced string |
| 99 | processing. For complex matching and manipulation, regular expressions offer |
| 100 | succinct, optimized solutions:: |
| 101 | |
| 102 | >>> import re |
| 103 | >>> re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest') |
| 104 | ['foot', 'fell', 'fastest'] |
| 105 | >>> re.sub(r'(\b[a-z]+) \1', r'\1', 'cat in the the hat') |
| 106 | 'cat in the hat' |
| 107 | |
| 108 | When only simple capabilities are needed, string methods are preferred because |
| 109 | they are easier to read and debug:: |
| 110 | |
| 111 | >>> 'tea for too'.replace('too', 'two') |
| 112 | 'tea for two' |
| 113 | |
| 114 | |
| 115 | .. _tut-mathematics: |
| 116 | |
| 117 | Mathematics |
| 118 | =========== |
| 119 | |
| 120 | The :mod:`math` module gives access to the underlying C library functions for |
| 121 | floating point math:: |
| 122 | |
| 123 | >>> import math |
Georg Brandl | f694518 | 2008-02-01 11:56:49 +0000 | [diff] [blame] | 124 | >>> math.cos(math.pi / 4) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 125 | 0.70710678118654757 |
| 126 | >>> math.log(1024, 2) |
| 127 | 10.0 |
| 128 | |
| 129 | The :mod:`random` module provides tools for making random selections:: |
| 130 | |
| 131 | >>> import random |
| 132 | >>> random.choice(['apple', 'pear', 'banana']) |
| 133 | 'apple' |
| 134 | >>> random.sample(range(100), 10) # sampling without replacement |
| 135 | [30, 83, 16, 4, 8, 81, 41, 50, 18, 33] |
| 136 | >>> random.random() # random float |
| 137 | 0.17970987693706186 |
| 138 | >>> random.randrange(6) # random integer chosen from range(6) |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 139 | 4 |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 140 | |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 141 | The SciPy project <http://scipy.org> has many other modules for numerical |
| 142 | computations. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 143 | |
| 144 | .. _tut-internet-access: |
| 145 | |
| 146 | Internet Access |
| 147 | =============== |
| 148 | |
| 149 | There are a number of modules for accessing the internet and processing internet |
Senthil Kumaran | aca8fd7 | 2008-06-23 04:41:59 +0000 | [diff] [blame] | 150 | protocols. Two of the simplest are :mod:`urllib.request` for retrieving data |
| 151 | from urls and :mod:`smtplib` for sending mail:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 152 | |
Georg Brandl | 0f7ede4 | 2008-06-23 11:23:31 +0000 | [diff] [blame] | 153 | >>> from urllib.request import urlopen |
| 154 | >>> for line in urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl'): |
Senthil Kumaran | ac0b889 | 2010-02-22 17:17:22 +0000 | [diff] [blame] | 155 | ... line = line.decode('utf-8') # Decoding the binary data to text. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 156 | ... if 'EST' in line or 'EDT' in line: # look for Eastern Time |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 157 | ... print(line) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 158 | |
| 159 | <BR>Nov. 25, 09:43:32 PM EST |
| 160 | |
| 161 | >>> import smtplib |
| 162 | >>> server = smtplib.SMTP('localhost') |
| 163 | >>> server.sendmail('soothsayer@example.org', 'jcaesar@example.org', |
Guido van Rossum | f10aa98 | 2007-08-17 18:30:38 +0000 | [diff] [blame] | 164 | ... """To: jcaesar@example.org |
| 165 | ... From: soothsayer@example.org |
| 166 | ... |
| 167 | ... Beware the Ides of March. |
| 168 | ... """) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 169 | >>> server.quit() |
| 170 | |
Guido van Rossum | f10aa98 | 2007-08-17 18:30:38 +0000 | [diff] [blame] | 171 | (Note that the second example needs a mailserver running on localhost.) |
| 172 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 173 | |
| 174 | .. _tut-dates-and-times: |
| 175 | |
| 176 | Dates and Times |
| 177 | =============== |
| 178 | |
| 179 | The :mod:`datetime` module supplies classes for manipulating dates and times in |
| 180 | both simple and complex ways. While date and time arithmetic is supported, the |
| 181 | focus of the implementation is on efficient member extraction for output |
| 182 | formatting and manipulation. The module also supports objects that are timezone |
| 183 | aware. :: |
| 184 | |
Ezio Melotti | 87e5d34 | 2009-11-12 10:38:55 +0000 | [diff] [blame] | 185 | >>> # dates are easily constructed and formatted |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 186 | >>> from datetime import date |
| 187 | >>> now = date.today() |
| 188 | >>> now |
| 189 | datetime.date(2003, 12, 2) |
| 190 | >>> now.strftime("%m-%d-%y. %d %b %Y is a %A on the %d day of %B.") |
| 191 | '12-02-03. 02 Dec 2003 is a Tuesday on the 02 day of December.' |
| 192 | |
Ezio Melotti | 87e5d34 | 2009-11-12 10:38:55 +0000 | [diff] [blame] | 193 | >>> # dates support calendar arithmetic |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 194 | >>> birthday = date(1964, 7, 31) |
| 195 | >>> age = now - birthday |
| 196 | >>> age.days |
| 197 | 14368 |
| 198 | |
| 199 | |
| 200 | .. _tut-data-compression: |
| 201 | |
| 202 | Data Compression |
| 203 | ================ |
| 204 | |
| 205 | Common data archiving and compression formats are directly supported by modules |
| 206 | including: :mod:`zlib`, :mod:`gzip`, :mod:`bz2`, :mod:`zipfile` and |
| 207 | :mod:`tarfile`. :: |
| 208 | |
| 209 | >>> import zlib |
Georg Brandl | 53afa6d | 2010-11-30 08:20:16 +0000 | [diff] [blame] | 210 | >>> s = b'witch which has which witches wrist watch' |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 211 | >>> len(s) |
| 212 | 41 |
| 213 | >>> t = zlib.compress(s) |
| 214 | >>> len(t) |
| 215 | 37 |
| 216 | >>> zlib.decompress(t) |
Georg Brandl | 53afa6d | 2010-11-30 08:20:16 +0000 | [diff] [blame] | 217 | b'witch which has which witches wrist watch' |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 218 | >>> zlib.crc32(s) |
| 219 | 226805979 |
| 220 | |
| 221 | |
| 222 | .. _tut-performance-measurement: |
| 223 | |
| 224 | Performance Measurement |
| 225 | ======================= |
| 226 | |
| 227 | Some Python users develop a deep interest in knowing the relative performance of |
| 228 | different approaches to the same problem. Python provides a measurement tool |
| 229 | that answers those questions immediately. |
| 230 | |
| 231 | For example, it may be tempting to use the tuple packing and unpacking feature |
| 232 | instead of the traditional approach to swapping arguments. The :mod:`timeit` |
| 233 | module quickly demonstrates a modest performance advantage:: |
| 234 | |
| 235 | >>> from timeit import Timer |
| 236 | >>> Timer('t=a; a=b; b=t', 'a=1; b=2').timeit() |
| 237 | 0.57535828626024577 |
| 238 | >>> Timer('a,b = b,a', 'a=1; b=2').timeit() |
| 239 | 0.54962537085770791 |
| 240 | |
| 241 | In contrast to :mod:`timeit`'s fine level of granularity, the :mod:`profile` and |
| 242 | :mod:`pstats` modules provide tools for identifying time critical sections in |
| 243 | larger blocks of code. |
| 244 | |
| 245 | |
| 246 | .. _tut-quality-control: |
| 247 | |
| 248 | Quality Control |
| 249 | =============== |
| 250 | |
| 251 | One approach for developing high quality software is to write tests for each |
| 252 | function as it is developed and to run those tests frequently during the |
| 253 | development process. |
| 254 | |
| 255 | The :mod:`doctest` module provides a tool for scanning a module and validating |
| 256 | tests embedded in a program's docstrings. Test construction is as simple as |
| 257 | cutting-and-pasting a typical call along with its results into the docstring. |
| 258 | This improves the documentation by providing the user with an example and it |
| 259 | allows the doctest module to make sure the code remains true to the |
| 260 | documentation:: |
| 261 | |
| 262 | def average(values): |
| 263 | """Computes the arithmetic mean of a list of numbers. |
| 264 | |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 265 | >>> print(average([20, 30, 70])) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 266 | 40.0 |
| 267 | """ |
Georg Brandl | f694518 | 2008-02-01 11:56:49 +0000 | [diff] [blame] | 268 | return sum(values) / len(values) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 269 | |
| 270 | import doctest |
| 271 | doctest.testmod() # automatically validate the embedded tests |
| 272 | |
| 273 | The :mod:`unittest` module is not as effortless as the :mod:`doctest` module, |
| 274 | but it allows a more comprehensive set of tests to be maintained in a separate |
| 275 | file:: |
| 276 | |
| 277 | import unittest |
| 278 | |
| 279 | class TestStatisticalFunctions(unittest.TestCase): |
| 280 | |
| 281 | def test_average(self): |
| 282 | self.assertEqual(average([20, 30, 70]), 40.0) |
| 283 | self.assertEqual(round(average([1, 5, 7]), 1), 4.3) |
| 284 | self.assertRaises(ZeroDivisionError, average, []) |
| 285 | self.assertRaises(TypeError, average, 20, 30, 70) |
| 286 | |
| 287 | unittest.main() # Calling from the command line invokes all tests |
| 288 | |
| 289 | |
| 290 | .. _tut-batteries-included: |
| 291 | |
| 292 | Batteries Included |
| 293 | ================== |
| 294 | |
| 295 | Python has a "batteries included" philosophy. This is best seen through the |
| 296 | sophisticated and robust capabilities of its larger packages. For example: |
| 297 | |
Georg Brandl | 38eceaa | 2008-05-26 11:14:17 +0000 | [diff] [blame] | 298 | * The :mod:`xmlrpc.client` and :mod:`xmlrpc.server` modules make implementing |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 299 | remote procedure calls into an almost trivial task. Despite the modules |
| 300 | names, no direct knowledge or handling of XML is needed. |
| 301 | |
| 302 | * The :mod:`email` package is a library for managing email messages, including |
| 303 | MIME and other RFC 2822-based message documents. Unlike :mod:`smtplib` and |
| 304 | :mod:`poplib` which actually send and receive messages, the email package has |
| 305 | a complete toolset for building or decoding complex message structures |
| 306 | (including attachments) and for implementing internet encoding and header |
| 307 | protocols. |
| 308 | |
| 309 | * The :mod:`xml.dom` and :mod:`xml.sax` packages provide robust support for |
| 310 | parsing this popular data interchange format. Likewise, the :mod:`csv` module |
| 311 | supports direct reads and writes in a common database format. Together, these |
Ezio Melotti | 0639d5a | 2009-12-19 23:26:38 +0000 | [diff] [blame] | 312 | modules and packages greatly simplify data interchange between Python |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 313 | applications and other tools. |
| 314 | |
| 315 | * Internationalization is supported by a number of modules including |
| 316 | :mod:`gettext`, :mod:`locale`, and the :mod:`codecs` package. |
| 317 | |
| 318 | |