Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`timeit` --- Measure execution time of small code snippets |
| 2 | =============================================================== |
| 3 | |
| 4 | .. module:: timeit |
| 5 | :synopsis: Measure the execution time of small code snippets. |
| 6 | |
| 7 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 8 | .. index:: |
| 9 | single: Benchmarking |
| 10 | single: Performance |
| 11 | |
Raymond Hettinger | a199368 | 2011-01-27 01:20:32 +0000 | [diff] [blame] | 12 | **Source code:** :source:`Lib/timeit.py` |
| 13 | |
| 14 | -------------- |
| 15 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 16 | This module provides a simple way to time small bits of Python code. It has both |
| 17 | command line as well as callable interfaces. It avoids a number of common traps |
| 18 | for measuring execution times. See also Tim Peters' introduction to the |
| 19 | "Algorithms" chapter in the Python Cookbook, published by O'Reilly. |
| 20 | |
| 21 | The module defines the following public class: |
| 22 | |
| 23 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 24 | .. class:: Timer(stmt='pass', setup='pass', timer=<timer function>) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 25 | |
| 26 | Class for timing execution speed of small code snippets. |
| 27 | |
| 28 | The constructor takes a statement to be timed, an additional statement used for |
| 29 | setup, and a timer function. Both statements default to ``'pass'``; the timer |
Benjamin Peterson | 0289b15 | 2009-06-28 17:22:03 +0000 | [diff] [blame] | 30 | function is platform-dependent (see the module doc string). *stmt* and *setup* |
| 31 | may also contain multiple statements separated by ``;`` or newlines, as long as |
| 32 | they don't contain multi-line string literals. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 33 | |
| 34 | To measure the execution time of the first statement, use the :meth:`timeit` |
| 35 | method. The :meth:`repeat` method is a convenience to call :meth:`timeit` |
| 36 | multiple times and return a list of results. |
| 37 | |
Georg Brandl | 55ac8f0 | 2007-09-01 13:51:09 +0000 | [diff] [blame] | 38 | The *stmt* and *setup* parameters can also take objects that are callable |
| 39 | without arguments. This will embed calls to them in a timer function that |
| 40 | will then be executed by :meth:`timeit`. Note that the timing overhead is a |
| 41 | little larger in this case because of the extra function calls. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 42 | |
| 43 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 44 | .. method:: Timer.print_exc(file=None) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 45 | |
| 46 | Helper to print a traceback from the timed code. |
| 47 | |
| 48 | Typical use:: |
| 49 | |
| 50 | t = Timer(...) # outside the try/except |
| 51 | try: |
| 52 | t.timeit(...) # or t.repeat(...) |
| 53 | except: |
| 54 | t.print_exc() |
| 55 | |
| 56 | The advantage over the standard traceback is that source lines in the compiled |
| 57 | template will be displayed. The optional *file* argument directs where the |
| 58 | traceback is sent; it defaults to ``sys.stderr``. |
| 59 | |
| 60 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 61 | .. method:: Timer.repeat(repeat=3, number=1000000) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 62 | |
| 63 | Call :meth:`timeit` a few times. |
| 64 | |
| 65 | This is a convenience function that calls the :meth:`timeit` repeatedly, |
| 66 | returning a list of results. The first argument specifies how many times to |
| 67 | call :meth:`timeit`. The second argument specifies the *number* argument for |
| 68 | :func:`timeit`. |
| 69 | |
| 70 | .. note:: |
| 71 | |
| 72 | It's tempting to calculate mean and standard deviation from the result vector |
| 73 | and report these. However, this is not very useful. In a typical case, the |
| 74 | lowest value gives a lower bound for how fast your machine can run the given |
| 75 | code snippet; higher values in the result vector are typically not caused by |
| 76 | variability in Python's speed, but by other processes interfering with your |
| 77 | timing accuracy. So the :func:`min` of the result is probably the only number |
| 78 | you should be interested in. After that, you should look at the entire vector |
| 79 | and apply common sense rather than statistics. |
| 80 | |
| 81 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 82 | .. method:: Timer.timeit(number=1000000) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 83 | |
| 84 | Time *number* executions of the main statement. This executes the setup |
| 85 | statement once, and then returns the time it takes to execute the main statement |
| 86 | a number of times, measured in seconds as a float. The argument is the number |
| 87 | of times through the loop, defaulting to one million. The main statement, the |
| 88 | setup statement and the timer function to be used are passed to the constructor. |
| 89 | |
| 90 | .. note:: |
| 91 | |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 92 | By default, :meth:`timeit` temporarily turns off :term:`garbage collection` |
| 93 | during the timing. The advantage of this approach is that it makes |
| 94 | independent timings more comparable. This disadvantage is that GC may be |
| 95 | an important component of the performance of the function being measured. |
| 96 | If so, GC can be re-enabled as the first statement in the *setup* string. |
| 97 | For example:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 98 | |
| 99 | timeit.Timer('for i in range(10): oct(i)', 'gc.enable()').timeit() |
| 100 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 101 | |
Georg Brandl | e6bcc91 | 2008-05-12 18:05:20 +0000 | [diff] [blame] | 102 | The module also defines two convenience functions: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 103 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 104 | .. function:: repeat(stmt='pass', setup='pass', timer=<default timer>, repeat=3, number=1000000) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 105 | |
| 106 | Create a :class:`Timer` instance with the given statement, setup code and timer |
| 107 | function and run its :meth:`repeat` method with the given repeat count and |
| 108 | *number* executions. |
| 109 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 110 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 111 | .. function:: timeit(stmt='pass', setup='pass', timer=<default timer>, number=1000000) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 112 | |
| 113 | Create a :class:`Timer` instance with the given statement, setup code and timer |
| 114 | function and run its :meth:`timeit` method with *number* executions. |
| 115 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 116 | |
| 117 | Command Line Interface |
| 118 | ---------------------- |
| 119 | |
| 120 | When called as a program from the command line, the following form is used:: |
| 121 | |
| 122 | python -m timeit [-n N] [-r N] [-s S] [-t] [-c] [-h] [statement ...] |
| 123 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 124 | Where the following options are understood: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 125 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 126 | .. program:: timeit |
| 127 | |
| 128 | .. cmdoption:: -n N, --number=N |
| 129 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 130 | how many times to execute 'statement' |
| 131 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 132 | .. cmdoption:: -r N, --repeat=N |
| 133 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 134 | how many times to repeat the timer (default 3) |
| 135 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 136 | .. cmdoption:: -s S, --setup=S |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 137 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 138 | statement to be executed once initially (default ``pass``) |
| 139 | |
| 140 | .. cmdoption:: -t, --time |
| 141 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 142 | use :func:`time.time` (default on all platforms but Windows) |
| 143 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 144 | .. cmdoption:: -c, --clock |
| 145 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 146 | use :func:`time.clock` (default on Windows) |
| 147 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 148 | .. cmdoption:: -v, --verbose |
| 149 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 150 | print raw timing results; repeat for more digits precision |
| 151 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 152 | .. cmdoption:: -h, --help |
| 153 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 154 | print a short usage message and exit |
| 155 | |
| 156 | A multi-line statement may be given by specifying each line as a separate |
| 157 | statement argument; indented lines are possible by enclosing an argument in |
| 158 | quotes and using leading spaces. Multiple :option:`-s` options are treated |
| 159 | similarly. |
| 160 | |
| 161 | If :option:`-n` is not given, a suitable number of loops is calculated by trying |
| 162 | successive powers of 10 until the total time is at least 0.2 seconds. |
| 163 | |
| 164 | The default timer function is platform dependent. On Windows, |
| 165 | :func:`time.clock` has microsecond granularity but :func:`time.time`'s |
| 166 | granularity is 1/60th of a second; on Unix, :func:`time.clock` has 1/100th of a |
| 167 | second granularity and :func:`time.time` is much more precise. On either |
| 168 | platform, the default timer functions measure wall clock time, not the CPU time. |
| 169 | This means that other processes running on the same computer may interfere with |
| 170 | the timing. The best thing to do when accurate timing is necessary is to repeat |
| 171 | the timing a few times and use the best time. The :option:`-r` option is good |
| 172 | for this; the default of 3 repetitions is probably enough in most cases. On |
| 173 | Unix, you can use :func:`time.clock` to measure CPU time. |
| 174 | |
| 175 | .. note:: |
| 176 | |
| 177 | There is a certain baseline overhead associated with executing a pass statement. |
| 178 | The code here doesn't try to hide it, but you should be aware of it. The |
| 179 | baseline overhead can be measured by invoking the program without arguments. |
| 180 | |
| 181 | The baseline overhead differs between Python versions! Also, to fairly compare |
| 182 | older Python versions to Python 2.3, you may want to use Python's :option:`-O` |
| 183 | option for the older versions to avoid timing ``SET_LINENO`` instructions. |
| 184 | |
| 185 | |
| 186 | Examples |
| 187 | -------- |
| 188 | |
| 189 | Here are two example sessions (one using the command line, one using the module |
| 190 | interface) that compare the cost of using :func:`hasattr` vs. |
| 191 | :keyword:`try`/:keyword:`except` to test for missing and present object |
| 192 | attributes. :: |
| 193 | |
Senthil Kumaran | 2e01535 | 2011-08-06 13:37:04 +0800 | [diff] [blame] | 194 | $ python -m timeit 'try:' ' str.__bool__' 'except AttributeError:' ' pass' |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 195 | 100000 loops, best of 3: 15.7 usec per loop |
Senthil Kumaran | 2e01535 | 2011-08-06 13:37:04 +0800 | [diff] [blame] | 196 | $ python -m timeit 'if hasattr(str, "__bool__"): pass' |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 197 | 100000 loops, best of 3: 4.26 usec per loop |
Senthil Kumaran | 2e01535 | 2011-08-06 13:37:04 +0800 | [diff] [blame] | 198 | $ python -m timeit 'try:' ' int.__bool__' 'except AttributeError:' ' pass' |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 199 | 1000000 loops, best of 3: 1.43 usec per loop |
Senthil Kumaran | 2e01535 | 2011-08-06 13:37:04 +0800 | [diff] [blame] | 200 | $ python -m timeit 'if hasattr(int, "__bool__"): pass' |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 201 | 100000 loops, best of 3: 2.23 usec per loop |
| 202 | |
| 203 | :: |
| 204 | |
| 205 | >>> import timeit |
| 206 | >>> s = """\ |
| 207 | ... try: |
| 208 | ... str.__bool__ |
| 209 | ... except AttributeError: |
| 210 | ... pass |
| 211 | ... """ |
| 212 | >>> t = timeit.Timer(stmt=s) |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 213 | >>> print("%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 214 | 17.09 usec/pass |
| 215 | >>> s = """\ |
| 216 | ... if hasattr(str, '__bool__'): pass |
| 217 | ... """ |
| 218 | >>> t = timeit.Timer(stmt=s) |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 219 | >>> print("%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 220 | 4.85 usec/pass |
| 221 | >>> s = """\ |
| 222 | ... try: |
| 223 | ... int.__bool__ |
| 224 | ... except AttributeError: |
| 225 | ... pass |
| 226 | ... """ |
| 227 | >>> t = timeit.Timer(stmt=s) |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 228 | >>> print("%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 229 | 1.97 usec/pass |
| 230 | >>> s = """\ |
| 231 | ... if hasattr(int, '__bool__'): pass |
| 232 | ... """ |
| 233 | >>> t = timeit.Timer(stmt=s) |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 234 | >>> print("%.2f usec/pass" % (1000000 * t.timeit(number=100000)/100000)) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 235 | 3.15 usec/pass |
| 236 | |
| 237 | To give the :mod:`timeit` module access to functions you define, you can pass a |
| 238 | ``setup`` parameter which contains an import statement:: |
| 239 | |
| 240 | def test(): |
Senthil Kumaran | 2e01535 | 2011-08-06 13:37:04 +0800 | [diff] [blame] | 241 | """Stupid test function""" |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 242 | L = [i for i in range(100)] |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 243 | |
Senthil Kumaran | 2e01535 | 2011-08-06 13:37:04 +0800 | [diff] [blame] | 244 | if __name__ == '__main__': |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 245 | from timeit import Timer |
| 246 | t = Timer("test()", "from __main__ import test") |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 247 | print(t.timeit()) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 248 | |