Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 1 | \section{\module{doctest} --- |
| 2 | Test docstrings represent reality} |
| 3 | |
| 4 | \declaremodule{standard}{doctest} |
| 5 | \moduleauthor{Tim Peters}{tim_one@users.sourceforge.net} |
| 6 | \sectionauthor{Tim Peters}{tim_one@users.sourceforge.net} |
| 7 | \sectionauthor{Moshe Zadka}{moshez@debian.org} |
| 8 | |
| 9 | \modulesynopsis{A framework for verifying examples in docstrings.} |
| 10 | |
| 11 | The \module{doctest} module searches a module's docstrings for text that looks |
| 12 | like an interactive Python session, then executes all such sessions to verify |
| 13 | they still work exactly as shown. Here's a complete but small example: |
| 14 | |
| 15 | \begin{verbatim} |
| 16 | """ |
| 17 | This is module example. |
| 18 | |
| 19 | Example supplies one function, factorial. For example, |
| 20 | |
| 21 | >>> factorial(5) |
| 22 | 120 |
| 23 | """ |
| 24 | |
| 25 | def factorial(n): |
| 26 | """Return the factorial of n, an exact integer >= 0. |
| 27 | |
| 28 | If the result is small enough to fit in an int, return an int. |
| 29 | Else return a long. |
| 30 | |
| 31 | >>> [factorial(n) for n in range(6)] |
| 32 | [1, 1, 2, 6, 24, 120] |
| 33 | >>> [factorial(long(n)) for n in range(6)] |
| 34 | [1, 1, 2, 6, 24, 120] |
| 35 | >>> factorial(30) |
| 36 | 265252859812191058636308480000000L |
| 37 | >>> factorial(30L) |
| 38 | 265252859812191058636308480000000L |
| 39 | >>> factorial(-1) |
| 40 | Traceback (most recent call last): |
| 41 | ... |
| 42 | ValueError: n must be >= 0 |
| 43 | |
| 44 | Factorials of floats are OK, but the float must be an exact integer: |
| 45 | >>> factorial(30.1) |
| 46 | Traceback (most recent call last): |
| 47 | ... |
| 48 | ValueError: n must be exact integer |
| 49 | >>> factorial(30.0) |
| 50 | 265252859812191058636308480000000L |
| 51 | |
| 52 | It must also not be ridiculously large: |
| 53 | >>> factorial(1e100) |
| 54 | Traceback (most recent call last): |
| 55 | ... |
| 56 | OverflowError: n too large |
| 57 | """ |
| 58 | |
| 59 | \end{verbatim} |
| 60 | % allow LaTeX to break here. |
| 61 | \begin{verbatim} |
| 62 | |
| 63 | import math |
| 64 | if not n >= 0: |
| 65 | raise ValueError("n must be >= 0") |
| 66 | if math.floor(n) != n: |
| 67 | raise ValueError("n must be exact integer") |
| 68 | if n+1 == n: # e.g., 1e300 |
| 69 | raise OverflowError("n too large") |
| 70 | result = 1 |
| 71 | factor = 2 |
| 72 | while factor <= n: |
| 73 | try: |
| 74 | result *= factor |
| 75 | except OverflowError: |
| 76 | result *= long(factor) |
| 77 | factor += 1 |
| 78 | return result |
| 79 | |
| 80 | def _test(): |
| 81 | import doctest, example |
| 82 | return doctest.testmod(example) |
| 83 | |
| 84 | if __name__ == "__main__": |
| 85 | _test() |
| 86 | \end{verbatim} |
| 87 | |
| 88 | If you run \file{example.py} directly from the command line, doctest works |
| 89 | its magic: |
| 90 | |
| 91 | \begin{verbatim} |
| 92 | $ python example.py |
| 93 | $ |
| 94 | \end{verbatim} |
| 95 | |
| 96 | There's no output! That's normal, and it means all the examples worked. |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 97 | Pass \programopt{-v} to the script, and doctest prints a detailed log |
| 98 | of what it's trying, and prints a summary at the end: |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 99 | |
| 100 | \begin{verbatim} |
| 101 | $ python example.py -v |
| 102 | Running example.__doc__ |
| 103 | Trying: factorial(5) |
| 104 | Expecting: 120 |
| 105 | ok |
| 106 | 0 of 1 examples failed in example.__doc__ |
| 107 | Running example.factorial.__doc__ |
| 108 | Trying: [factorial(n) for n in range(6)] |
| 109 | Expecting: [1, 1, 2, 6, 24, 120] |
| 110 | ok |
| 111 | Trying: [factorial(long(n)) for n in range(6)] |
| 112 | Expecting: [1, 1, 2, 6, 24, 120] |
| 113 | ok |
| 114 | Trying: factorial(30) |
| 115 | Expecting: 265252859812191058636308480000000L |
| 116 | ok |
| 117 | \end{verbatim} |
| 118 | |
| 119 | And so on, eventually ending with: |
| 120 | |
| 121 | \begin{verbatim} |
| 122 | Trying: factorial(1e100) |
| 123 | Expecting: |
| 124 | Traceback (most recent call last): |
| 125 | ... |
| 126 | OverflowError: n too large |
| 127 | ok |
| 128 | 0 of 8 examples failed in example.factorial.__doc__ |
| 129 | 2 items passed all tests: |
| 130 | 1 tests in example |
| 131 | 8 tests in example.factorial |
| 132 | 9 tests in 2 items. |
| 133 | 9 passed and 0 failed. |
| 134 | Test passed. |
| 135 | $ |
| 136 | \end{verbatim} |
| 137 | |
| 138 | That's all you need to know to start making productive use of doctest! Jump |
| 139 | in. The docstrings in doctest.py contain detailed information about all |
| 140 | aspects of doctest, and we'll just cover the more important points here. |
| 141 | |
| 142 | \subsection{Normal Usage} |
| 143 | |
| 144 | In normal use, end each module \module{M} with: |
| 145 | |
| 146 | \begin{verbatim} |
| 147 | def _test(): |
| 148 | import doctest, M # replace M with your module's name |
| 149 | return doctest.testmod(M) # ditto |
| 150 | |
| 151 | if __name__ == "__main__": |
| 152 | _test() |
| 153 | \end{verbatim} |
| 154 | |
Martin v. Löwis | 4581cfa | 2002-11-22 08:23:09 +0000 | [diff] [blame] | 155 | If you want to test the module as the main module, you don't need to |
Fred Drake | 5d2f515 | 2003-06-28 03:09:06 +0000 | [diff] [blame^] | 156 | pass M to \function{testmod()}; in this case, it will test the current |
Martin v. Löwis | 4581cfa | 2002-11-22 08:23:09 +0000 | [diff] [blame] | 157 | module. |
| 158 | |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 159 | Then running the module as a script causes the examples in the docstrings |
| 160 | to get executed and verified: |
| 161 | |
| 162 | \begin{verbatim} |
| 163 | python M.py |
| 164 | \end{verbatim} |
| 165 | |
| 166 | This won't display anything unless an example fails, in which case the |
| 167 | failing example(s) and the cause(s) of the failure(s) are printed to stdout, |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 168 | and the final line of output is \code{'Test failed.'}. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 169 | |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 170 | Run it with the \programopt{-v} switch instead: |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 171 | |
| 172 | \begin{verbatim} |
| 173 | python M.py -v |
| 174 | \end{verbatim} |
| 175 | |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 176 | and a detailed report of all examples tried is printed to \code{stdout}, |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 177 | along with assorted summaries at the end. |
| 178 | |
Fred Drake | 5d2f515 | 2003-06-28 03:09:06 +0000 | [diff] [blame^] | 179 | You can force verbose mode by passing \code{verbose=1} to |
| 180 | \function{testmod()}, or |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 181 | prohibit it by passing \code{verbose=0}. In either of those cases, |
Fred Drake | 5d2f515 | 2003-06-28 03:09:06 +0000 | [diff] [blame^] | 182 | \code{sys.argv} is not examined by \function{testmod()}. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 183 | |
Fred Drake | 5d2f515 | 2003-06-28 03:09:06 +0000 | [diff] [blame^] | 184 | In any case, \function{testmod()} returns a 2-tuple of ints \code{(\var{f}, |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 185 | \var{t})}, where \var{f} is the number of docstring examples that |
| 186 | failed and \var{t} is the total number of docstring examples |
| 187 | attempted. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 188 | |
| 189 | \subsection{Which Docstrings Are Examined?} |
| 190 | |
| 191 | See \file{docstring.py} for all the details. They're unsurprising: the |
| 192 | module docstring, and all function, class and method docstrings are |
| 193 | searched, with the exception of docstrings attached to objects with private |
Tim Peters | 0481d24 | 2001-10-02 21:01:22 +0000 | [diff] [blame] | 194 | names. Objects imported into the module are not searched. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 195 | |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 196 | In addition, if \code{M.__test__} exists and "is true", it must be a |
| 197 | dict, and each entry maps a (string) name to a function object, class |
| 198 | object, or string. Function and class object docstrings found from |
| 199 | \code{M.__test__} are searched even if the name is private, and |
| 200 | strings are searched directly as if they were docstrings. In output, |
| 201 | a key \code{K} in \code{M.__test__} appears with name |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 202 | |
| 203 | \begin{verbatim} |
| 204 | <name of M>.__test__.K |
| 205 | \end{verbatim} |
| 206 | |
| 207 | Any classes found are recursively searched similarly, to test docstrings in |
| 208 | their contained methods and nested classes. While private names reached |
| 209 | from \module{M}'s globals are skipped, all names reached from |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 210 | \code{M.__test__} are searched. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 211 | |
| 212 | \subsection{What's the Execution Context?} |
| 213 | |
Fred Drake | 5d2f515 | 2003-06-28 03:09:06 +0000 | [diff] [blame^] | 214 | By default, each time \function{testmod()} finds a docstring to test, it uses |
| 215 | a \emph{copy} of \module{M}'s globals, so that running tests on a module |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 216 | doesn't change the module's real globals, and so that one test in |
| 217 | \module{M} can't leave behind crumbs that accidentally allow another test |
| 218 | to work. This means examples can freely use any names defined at top-level |
Tim Peters | 0481d24 | 2001-10-02 21:01:22 +0000 | [diff] [blame] | 219 | in \module{M}, and names defined earlier in the docstring being run. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 220 | |
| 221 | You can force use of your own dict as the execution context by passing |
| 222 | \code{globs=your_dict} to \function{testmod()} instead. Presumably this |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 223 | would be a copy of \code{M.__dict__} merged with the globals from other |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 224 | imported modules. |
| 225 | |
| 226 | \subsection{What About Exceptions?} |
| 227 | |
| 228 | No problem, as long as the only output generated by the example is the |
| 229 | traceback itself. For example: |
| 230 | |
| 231 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 232 | >>> [1, 2, 3].remove(42) |
| 233 | Traceback (most recent call last): |
| 234 | File "<stdin>", line 1, in ? |
| 235 | ValueError: list.remove(x): x not in list |
| 236 | >>> |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 237 | \end{verbatim} |
| 238 | |
| 239 | Note that only the exception type and value are compared (specifically, |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 240 | only the last line in the traceback). The various ``File'' lines in |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 241 | between can be left out (unless they add significantly to the documentation |
| 242 | value of the example). |
| 243 | |
| 244 | \subsection{Advanced Usage} |
| 245 | |
| 246 | \function{testmod()} actually creates a local instance of class |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 247 | \class{Tester}, runs appropriate methods of that class, and merges |
| 248 | the results into global \class{Tester} instance \code{master}. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 249 | |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 250 | You can create your own instances of \class{Tester}, and so build your |
| 251 | own policies, or even run methods of \code{master} directly. See |
| 252 | \code{Tester.__doc__} for details. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 253 | |
| 254 | |
| 255 | \subsection{How are Docstring Examples Recognized?} |
| 256 | |
| 257 | In most cases a copy-and-paste of an interactive console session works fine |
| 258 | --- just make sure the leading whitespace is rigidly consistent (you can mix |
| 259 | tabs and spaces if you're too lazy to do it right, but doctest is not in |
| 260 | the business of guessing what you think a tab means). |
| 261 | |
| 262 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 263 | >>> # comments are ignored |
| 264 | >>> x = 12 |
| 265 | >>> x |
| 266 | 12 |
| 267 | >>> if x == 13: |
| 268 | ... print "yes" |
| 269 | ... else: |
| 270 | ... print "no" |
| 271 | ... print "NO" |
| 272 | ... print "NO!!!" |
| 273 | ... |
| 274 | no |
| 275 | NO |
| 276 | NO!!! |
| 277 | >>> |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 278 | \end{verbatim} |
| 279 | |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 280 | Any expected output must immediately follow the final |
| 281 | \code{'>\code{>}>~'} or \code{'...~'} line containing the code, and |
| 282 | the expected output (if any) extends to the next \code{'>\code{>}>~'} |
| 283 | or all-whitespace line. |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 284 | |
| 285 | The fine print: |
| 286 | |
| 287 | \begin{itemize} |
| 288 | |
| 289 | \item Expected output cannot contain an all-whitespace line, since such a |
| 290 | line is taken to signal the end of expected output. |
| 291 | |
| 292 | \item Output to stdout is captured, but not output to stderr (exception |
| 293 | tracebacks are captured via a different means). |
| 294 | |
| 295 | \item If you continue a line via backslashing in an interactive session, or |
| 296 | for any other reason use a backslash, you need to double the backslash in |
| 297 | the docstring version. This is simply because you're in a string, and so |
| 298 | the backslash must be escaped for it to survive intact. Like: |
| 299 | |
| 300 | \begin{verbatim} |
| 301 | >>> if "yes" == \\ |
| 302 | ... "y" + \\ |
| 303 | ... "es": |
| 304 | ... print 'yes' |
| 305 | yes |
| 306 | \end{verbatim} |
| 307 | |
Tim Peters | f0768c8 | 2001-02-20 10:57:30 +0000 | [diff] [blame] | 308 | \item The starting column doesn't matter: |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 309 | |
| 310 | \begin{verbatim} |
Tim Peters | c4089d8 | 2001-02-17 18:03:25 +0000 | [diff] [blame] | 311 | >>> assert "Easy!" |
| 312 | >>> import math |
| 313 | >>> math.floor(1.9) |
| 314 | 1.0 |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 315 | \end{verbatim} |
| 316 | |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 317 | and as many leading whitespace characters are stripped from the |
| 318 | expected output as appeared in the initial \code{'>\code{>}>~'} line |
| 319 | that triggered it. |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 320 | \end{itemize} |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 321 | |
| 322 | \subsection{Warnings} |
| 323 | |
| 324 | \begin{enumerate} |
| 325 | |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 326 | \item \module{doctest} is serious about requiring exact matches in expected |
| 327 | output. If even a single character doesn't match, the test fails. This |
| 328 | will probably surprise you a few times, as you learn exactly what Python |
| 329 | does and doesn't guarantee about output. For example, when printing a |
| 330 | dict, Python doesn't guarantee that the key-value pairs will be printed |
| 331 | in any particular order, so a test like |
| 332 | |
| 333 | % Hey! What happened to Monty Python examples? |
Tim Peters | f0768c8 | 2001-02-20 10:57:30 +0000 | [diff] [blame] | 334 | % Tim: ask Guido -- it's his example! |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 335 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 336 | >>> foo() |
| 337 | {"Hermione": "hippogryph", "Harry": "broomstick"} |
| 338 | >>> |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 339 | \end{verbatim} |
| 340 | |
| 341 | is vulnerable! One workaround is to do |
| 342 | |
| 343 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 344 | >>> foo() == {"Hermione": "hippogryph", "Harry": "broomstick"} |
| 345 | 1 |
| 346 | >>> |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 347 | \end{verbatim} |
| 348 | |
| 349 | instead. Another is to do |
| 350 | |
| 351 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 352 | >>> d = foo().items() |
| 353 | >>> d.sort() |
| 354 | >>> d |
| 355 | [('Harry', 'broomstick'), ('Hermione', 'hippogryph')] |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 356 | \end{verbatim} |
| 357 | |
| 358 | There are others, but you get the idea. |
| 359 | |
| 360 | Another bad idea is to print things that embed an object address, like |
| 361 | |
| 362 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 363 | >>> id(1.0) # certain to fail some of the time |
| 364 | 7948648 |
| 365 | >>> |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 366 | \end{verbatim} |
| 367 | |
| 368 | Floating-point numbers are also subject to small output variations across |
| 369 | platforms, because Python defers to the platform C library for float |
| 370 | formatting, and C libraries vary widely in quality here. |
| 371 | |
| 372 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 373 | >>> 1./7 # risky |
| 374 | 0.14285714285714285 |
| 375 | >>> print 1./7 # safer |
| 376 | 0.142857142857 |
| 377 | >>> print round(1./7, 6) # much safer |
| 378 | 0.142857 |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 379 | \end{verbatim} |
| 380 | |
| 381 | Numbers of the form \code{I/2.**J} are safe across all platforms, and I |
| 382 | often contrive doctest examples to produce numbers of that form: |
| 383 | |
| 384 | \begin{verbatim} |
Fred Drake | 19f3c52 | 2001-02-22 23:15:05 +0000 | [diff] [blame] | 385 | >>> 3./4 # utterly safe |
| 386 | 0.75 |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 387 | \end{verbatim} |
| 388 | |
| 389 | Simple fractions are also easier for people to understand, and that makes |
| 390 | for better documentation. |
| 391 | |
Skip Montanaro | 1dc98c4 | 2001-06-08 14:40:28 +0000 | [diff] [blame] | 392 | \item Be careful if you have code that must only execute once. |
| 393 | |
| 394 | If you have module-level code that must only execute once, a more foolproof |
Fred Drake | c115835 | 2001-06-11 14:55:01 +0000 | [diff] [blame] | 395 | definition of \function{_test()} is |
Skip Montanaro | 1dc98c4 | 2001-06-08 14:40:28 +0000 | [diff] [blame] | 396 | |
| 397 | \begin{verbatim} |
| 398 | def _test(): |
| 399 | import doctest, sys |
Martin v. Löwis | 4581cfa | 2002-11-22 08:23:09 +0000 | [diff] [blame] | 400 | doctest.testmod() |
Skip Montanaro | 1dc98c4 | 2001-06-08 14:40:28 +0000 | [diff] [blame] | 401 | \end{verbatim} |
Tim Peters | 6ebe61f | 2003-06-27 20:48:05 +0000 | [diff] [blame] | 402 | |
| 403 | \item WYSIWYG isn't always the case, starting in Python 2.3. The |
Fred Drake | 5d2f515 | 2003-06-28 03:09:06 +0000 | [diff] [blame^] | 404 | string form of boolean results changed from \code{'0'} and |
| 405 | \code{'1'} to \code{'False'} and \code{'True'} in Python 2.3. |
Tim Peters | 6ebe61f | 2003-06-27 20:48:05 +0000 | [diff] [blame] | 406 | This makes it clumsy to write a doctest showing boolean results that |
| 407 | passes under multiple versions of Python. In Python 2.3, by default, |
| 408 | and as a special case, if an expected output block consists solely |
Fred Drake | 5d2f515 | 2003-06-28 03:09:06 +0000 | [diff] [blame^] | 409 | of \code{'0'} and the actual output block consists solely of |
| 410 | \code{'False'}, that's accepted as an exact match, and similarly for |
| 411 | \code{'1'} versus \code{'True'}. This behavior can be turned off by |
Tim Peters | 6ebe61f | 2003-06-27 20:48:05 +0000 | [diff] [blame] | 412 | passing the new (in 2.3) module constant |
| 413 | \constant{DONT_ACCEPT_TRUE_FOR_1} as the value of \function{testmod()}'s |
| 414 | new (in 2.3) optional \var{optionflags} argument. Some years after |
| 415 | the integer spellings of booleans are history, this hack will |
| 416 | probably be removed again. |
| 417 | |
Fred Drake | c115835 | 2001-06-11 14:55:01 +0000 | [diff] [blame] | 418 | \end{enumerate} |
| 419 | |
Tim Peters | 7688229 | 2001-02-17 05:58:44 +0000 | [diff] [blame] | 420 | |
| 421 | \subsection{Soapbox} |
| 422 | |
| 423 | The first word in doctest is "doc", and that's why the author wrote |
| 424 | doctest: to keep documentation up to date. It so happens that doctest |
| 425 | makes a pleasant unit testing environment, but that's not its primary |
| 426 | purpose. |
| 427 | |
| 428 | Choose docstring examples with care. There's an art to this that needs to |
| 429 | be learned --- it may not be natural at first. Examples should add genuine |
| 430 | value to the documentation. A good example can often be worth many words. |
| 431 | If possible, show just a few normal cases, show endcases, show interesting |
| 432 | subtle cases, and show an example of each kind of exception that can be |
| 433 | raised. You're probably testing for endcases and subtle cases anyway in an |
| 434 | interactive shell: doctest wants to make it as easy as possible to capture |
| 435 | those sessions, and will verify they continue to work as designed forever |
| 436 | after. |
| 437 | |
| 438 | If done with care, the examples will be invaluable for your users, and will |
| 439 | pay back the time it takes to collect them many times over as the years go |
| 440 | by and "things change". I'm still amazed at how often one of my doctest |
| 441 | examples stops working after a "harmless" change. |
| 442 | |
| 443 | For exhaustive testing, or testing boring cases that add no value to the |
Fred Drake | 7eb1463 | 2001-02-17 17:32:41 +0000 | [diff] [blame] | 444 | docs, define a \code{__test__} dict instead. That's what it's for. |