blob: 21cbc324b147700a5a86e2b8f702f0d848fef090 [file] [log] [blame]
Tim Peters4fd9e2f2001-08-18 00:05:50 +00001# Module doctest.
Tim Peters8485b562004-08-04 18:46:34 +00002# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org).
Tim Peters19397e52004-08-06 22:02:59 +00003# Major enhancements and refactoring by:
Tim Peters8485b562004-08-04 18:46:34 +00004# Jim Fulton
5# Edward Loper
Tim Peters8a7d2d52001-01-16 07:10:57 +00006
7# Provided as-is; use at your own risk; no warranty; no promises; enjoy!
8
Martin v. Löwis92816de2004-05-31 19:01:00 +00009r"""Module doctest -- a framework for running examples in docstrings.
Tim Peters8a7d2d52001-01-16 07:10:57 +000010
11NORMAL USAGE
12
Tim Peters80e53142004-08-09 04:34:45 +000013In simplest use, end each module M to be tested with:
Tim Peters8a7d2d52001-01-16 07:10:57 +000014
15def _test():
Tim Peters80e53142004-08-09 04:34:45 +000016 import doctest
17 return doctest.testmod()
Tim Peters8a7d2d52001-01-16 07:10:57 +000018
19if __name__ == "__main__":
20 _test()
21
22Then running the module as a script will cause the examples in the
23docstrings to get executed and verified:
24
25python M.py
26
27This won't display anything unless an example fails, in which case the
28failing example(s) and the cause(s) of the failure(s) are printed to stdout
29(why not stderr? because stderr is a lame hack <0.2 wink>), and the final
30line of output is "Test failed.".
31
32Run it with the -v switch instead:
33
34python M.py -v
35
36and a detailed report of all examples tried is printed to stdout, along
37with assorted summaries at the end.
38
Tim Peters80e53142004-08-09 04:34:45 +000039You can force verbose mode by passing "verbose=True" to testmod, or prohibit
40it by passing "verbose=False". In either of those cases, sys.argv is not
Tim Peters8a7d2d52001-01-16 07:10:57 +000041examined by testmod.
42
43In any case, testmod returns a 2-tuple of ints (f, t), where f is the
44number of docstring examples that failed and t is the total number of
45docstring examples attempted.
46
Tim Peters80e53142004-08-09 04:34:45 +000047There are a variety of other ways to run doctests, including integration
48with the unittest framework, and support for running non-Python text
49files containing doctests. There are also many ways to override parts
50of doctest's default behaviors. See the Library Reference Manual for
51details.
52
Tim Peters8a7d2d52001-01-16 07:10:57 +000053
54WHICH DOCSTRINGS ARE EXAMINED?
55
56+ M.__doc__.
57
58+ f.__doc__ for all functions f in M.__dict__.values(), except those
Raymond Hettinger71adf7e2003-07-16 19:25:22 +000059 defined in other modules.
Tim Peters8a7d2d52001-01-16 07:10:57 +000060
Raymond Hettinger71adf7e2003-07-16 19:25:22 +000061+ C.__doc__ for all classes C in M.__dict__.values(), except those
62 defined in other modules.
Tim Peters8a7d2d52001-01-16 07:10:57 +000063
64+ If M.__test__ exists and "is true", it must be a dict, and
65 each entry maps a (string) name to a function object, class object, or
66 string. Function and class object docstrings found from M.__test__
Tim Peters80e53142004-08-09 04:34:45 +000067 are searched, and strings are searched directly as if they were docstrings.
68 In output, a key K in M.__test__ appears with name
Tim Peters8a7d2d52001-01-16 07:10:57 +000069 <name of M>.__test__.K
70
71Any classes found are recursively searched similarly, to test docstrings in
Tim Peters80e53142004-08-09 04:34:45 +000072their contained methods and nested classes.
Tim Peters8a7d2d52001-01-16 07:10:57 +000073
Tim Peters8a7d2d52001-01-16 07:10:57 +000074
Tim Peters8a7d2d52001-01-16 07:10:57 +000075WHAT'S THE EXECUTION CONTEXT?
76
77By default, each time testmod finds a docstring to test, it uses a *copy*
78of M's globals (so that running tests on a module doesn't change the
79module's real globals, and so that one test in M can't leave behind crumbs
80that accidentally allow another test to work). This means examples can
81freely use any names defined at top-level in M. It also means that sloppy
82imports (see above) can cause examples in external docstrings to use
83globals inappropriate for them.
84
85You can force use of your own dict as the execution context by passing
86"globs=your_dict" to testmod instead. Presumably this would be a copy of
87M.__dict__ merged with the globals from other imported modules.
88
89
Tim Peters8a7d2d52001-01-16 07:10:57 +000090WHAT ABOUT EXCEPTIONS?
91
92No problem, as long as the only output generated by the example is the
93traceback itself. For example:
94
Tim Peters60e23f42001-02-14 00:43:21 +000095 >>> [1, 2, 3].remove(42)
Tim Petersea4f9312001-02-13 20:54:42 +000096 Traceback (most recent call last):
Tim Peters8a7d2d52001-01-16 07:10:57 +000097 File "<stdin>", line 1, in ?
Tim Peters60e23f42001-02-14 00:43:21 +000098 ValueError: list.remove(x): x not in list
Tim Peters8a7d2d52001-01-16 07:10:57 +000099 >>>
100
Tim Peters80e53142004-08-09 04:34:45 +0000101Note that only the exception type and value are compared.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000102
103
Tim Peters80e53142004-08-09 04:34:45 +0000104SO WHAT DOES A DOCTEST EXAMPLE LOOK LIKE ALREADY!?
Tim Peters8a7d2d52001-01-16 07:10:57 +0000105
106Oh ya. It's easy! In most cases a copy-and-paste of an interactive
107console session works fine -- just make sure the leading whitespace is
108rigidly consistent (you can mix tabs and spaces if you're too lazy to do it
109right, but doctest is not in the business of guessing what you think a tab
110means).
111
112 >>> # comments are ignored
113 >>> x = 12
114 >>> x
115 12
116 >>> if x == 13:
117 ... print "yes"
118 ... else:
119 ... print "no"
120 ... print "NO"
121 ... print "NO!!!"
122 ...
123 no
124 NO
125 NO!!!
126 >>>
127
128Any expected output must immediately follow the final ">>>" or "..." line
129containing the code, and the expected output (if any) extends to the next
130">>>" or all-whitespace line. That's it.
131
132Bummers:
133
Tim Peters8a7d2d52001-01-16 07:10:57 +0000134+ Output to stdout is captured, but not output to stderr (exception
135 tracebacks are captured via a different means).
136
Martin v. Löwis92816de2004-05-31 19:01:00 +0000137+ If you continue a line via backslashing in an interactive session,
138 or for any other reason use a backslash, you should use a raw
139 docstring, which will preserve your backslahses exactly as you type
140 them:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000141
Tim Peters4e0e1b62004-07-07 20:54:48 +0000142 >>> def f(x):
Martin v. Löwis92816de2004-05-31 19:01:00 +0000143 ... r'''Backslashes in a raw docstring: m\n'''
144 >>> print f.__doc__
145 Backslashes in a raw docstring: m\n
Tim Peters8a7d2d52001-01-16 07:10:57 +0000146
Martin v. Löwis92816de2004-05-31 19:01:00 +0000147 Otherwise, the backslash will be interpreted as part of the string.
148 E.g., the "\n" above would be interpreted as a newline character.
149 Alternatively, you can double each backslash in the doctest version
150 (and not use a raw string):
151
Tim Peters4e0e1b62004-07-07 20:54:48 +0000152 >>> def f(x):
Martin v. Löwis92816de2004-05-31 19:01:00 +0000153 ... '''Backslashes in a raw docstring: m\\n'''
154 >>> print f.__doc__
155 Backslashes in a raw docstring: m\n
Tim Peters4e0e1b62004-07-07 20:54:48 +0000156
Tim Peters8a7d2d52001-01-16 07:10:57 +0000157The starting column doesn't matter:
158
159>>> assert "Easy!"
160 >>> import math
161 >>> math.floor(1.9)
162 1.0
163
164and as many leading whitespace characters are stripped from the expected
165output as appeared in the initial ">>>" line that triggered it.
166
167If you execute this very file, the examples above will be found and
Tim Peters80e53142004-08-09 04:34:45 +0000168executed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000169"""
170
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000171__all__ = [
Tim Peters8485b562004-08-04 18:46:34 +0000172 'is_private',
173 'Example',
174 'DocTest',
175 'DocTestFinder',
176 'DocTestRunner',
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000177 'testmod',
178 'run_docstring_examples',
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000179 'Tester',
Tim Peters19397e52004-08-06 22:02:59 +0000180 'DocTestCase',
Tim Petersdb3756d2003-06-29 05:30:48 +0000181 'DocTestSuite',
182 'testsource',
183 'debug',
Tim Peters8485b562004-08-04 18:46:34 +0000184# 'master',
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000185]
Tim Peters8a7d2d52001-01-16 07:10:57 +0000186
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000187import __future__
Tim Peters8a7d2d52001-01-16 07:10:57 +0000188
Tim Peters19397e52004-08-06 22:02:59 +0000189import sys, traceback, inspect, linecache, os, re, types
Tim Peters8485b562004-08-04 18:46:34 +0000190import unittest, difflib, tempfile
Tim Petersf727c6c2004-08-08 01:48:59 +0000191import warnings
Tim Peters8485b562004-08-04 18:46:34 +0000192from StringIO import StringIO
Tim Peters7402f792001-10-02 03:53:41 +0000193
Tim Peters6ebe61f2003-06-27 20:48:05 +0000194# Option constants.
195DONT_ACCEPT_TRUE_FOR_1 = 1 << 0
Tim Peters8485b562004-08-04 18:46:34 +0000196DONT_ACCEPT_BLANKLINE = 1 << 1
197NORMALIZE_WHITESPACE = 1 << 2
198ELLIPSIS = 1 << 3
199UNIFIED_DIFF = 1 << 4
200CONTEXT_DIFF = 1 << 5
Tim Peters6ebe61f2003-06-27 20:48:05 +0000201
Tim Peters8485b562004-08-04 18:46:34 +0000202OPTIONFLAGS_BY_NAME = {
203 'DONT_ACCEPT_TRUE_FOR_1': DONT_ACCEPT_TRUE_FOR_1,
204 'DONT_ACCEPT_BLANKLINE': DONT_ACCEPT_BLANKLINE,
205 'NORMALIZE_WHITESPACE': NORMALIZE_WHITESPACE,
206 'ELLIPSIS': ELLIPSIS,
207 'UNIFIED_DIFF': UNIFIED_DIFF,
208 'CONTEXT_DIFF': CONTEXT_DIFF,
209 }
Tim Peters8a7d2d52001-01-16 07:10:57 +0000210
Tim Peters8485b562004-08-04 18:46:34 +0000211# Special string markers for use in `want` strings:
212BLANKLINE_MARKER = '<BLANKLINE>'
213ELLIPSIS_MARKER = '...'
Tim Peters8a7d2d52001-01-16 07:10:57 +0000214
Tim Peters19397e52004-08-06 22:02:59 +0000215
216# There are 4 basic classes:
217# - Example: a <source, want> pair, plus an intra-docstring line number.
218# - DocTest: a collection of examples, parsed from a docstring, plus
219# info about where the docstring came from (name, filename, lineno).
220# - DocTestFinder: extracts DocTests from a given object's docstring and
221# its contained objects' docstrings.
222# - DocTestRunner: runs DocTest cases, and accumulates statistics.
223#
224# So the basic picture is:
225#
226# list of:
227# +------+ +---------+ +-------+
228# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results|
229# +------+ +---------+ +-------+
230# | Example |
231# | ... |
232# | Example |
233# +---------+
234
Tim Peters8485b562004-08-04 18:46:34 +0000235######################################################################
236## Table of Contents
237######################################################################
Edward Loper7c748462004-08-09 02:06:06 +0000238# 1. Utility Functions
239# 2. Example & DocTest -- store test cases
240# 3. DocTest Parser -- extracts examples from strings
241# 4. DocTest Finder -- extracts test cases from objects
242# 5. DocTest Runner -- runs test cases
243# 6. Test Functions -- convenient wrappers for testing
244# 7. Tester Class -- for backwards compatibility
245# 8. Unittest Support
246# 9. Debugging Support
247# 10. Example Usage
Tim Peters8a7d2d52001-01-16 07:10:57 +0000248
Tim Peters8485b562004-08-04 18:46:34 +0000249######################################################################
250## 1. Utility Functions
251######################################################################
Tim Peters8a7d2d52001-01-16 07:10:57 +0000252
253def is_private(prefix, base):
254 """prefix, base -> true iff name prefix + "." + base is "private".
255
256 Prefix may be an empty string, and base does not contain a period.
257 Prefix is ignored (although functions you write conforming to this
258 protocol may make use of it).
259 Return true iff base begins with an (at least one) underscore, but
260 does not both begin and end with (at least) two underscores.
261
Tim Petersbafb1fe2004-08-08 01:52:57 +0000262 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning,
263 ... "doctest", 0)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000264 >>> is_private("a.b", "my_func")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000265 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000266 >>> is_private("____", "_my_func")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000267 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000268 >>> is_private("someclass", "__init__")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000269 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000270 >>> is_private("sometypo", "__init_")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000271 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000272 >>> is_private("x.y.z", "_")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000273 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000274 >>> is_private("_x.y.z", "__")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000275 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000276 >>> is_private("", "") # senseless but consistent
Guido van Rossum77f6a652002-04-03 22:41:51 +0000277 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000278 """
Tim Petersbafb1fe2004-08-08 01:52:57 +0000279 warnings.warn("is_private is deprecated; it wasn't useful; "
280 "examine DocTestFinder.find() lists instead",
Tim Peters3ddd60a2004-08-08 02:43:33 +0000281 DeprecationWarning, stacklevel=2)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000282 return base[:1] == "_" and not base[:2] == "__" == base[-2:]
283
Tim Peters8485b562004-08-04 18:46:34 +0000284def _extract_future_flags(globs):
285 """
286 Return the compiler-flags associated with the future features that
287 have been imported into the given namespace (globs).
288 """
289 flags = 0
290 for fname in __future__.all_feature_names:
291 feature = globs.get(fname, None)
292 if feature is getattr(__future__, fname):
293 flags |= feature.compiler_flag
294 return flags
Tim Peters7402f792001-10-02 03:53:41 +0000295
Tim Peters8485b562004-08-04 18:46:34 +0000296def _normalize_module(module, depth=2):
297 """
298 Return the module specified by `module`. In particular:
299 - If `module` is a module, then return module.
300 - If `module` is a string, then import and return the
301 module with that name.
302 - If `module` is None, then return the calling module.
303 The calling module is assumed to be the module of
304 the stack frame at the given depth in the call stack.
305 """
306 if inspect.ismodule(module):
307 return module
308 elif isinstance(module, (str, unicode)):
309 return __import__(module, globals(), locals(), ["*"])
310 elif module is None:
311 return sys.modules[sys._getframe(depth).f_globals['__name__']]
312 else:
313 raise TypeError("Expected a module, string, or None")
Tim Peters7402f792001-10-02 03:53:41 +0000314
Tim Peters8485b562004-08-04 18:46:34 +0000315def _tag_msg(tag, msg, indent_msg=True):
316 """
317 Return a string that displays a tag-and-message pair nicely,
318 keeping the tag and its message on the same line when that
319 makes sense. If `indent_msg` is true, then messages that are
320 put on separate lines will be indented.
321 """
322 # What string should we use to indent contents?
323 INDENT = ' '
Tim Peters8a7d2d52001-01-16 07:10:57 +0000324
Tim Peters8485b562004-08-04 18:46:34 +0000325 # If the message doesn't end in a newline, then add one.
326 if msg[-1:] != '\n':
327 msg += '\n'
328 # If the message is short enough, and contains no internal
329 # newlines, then display it on the same line as the tag.
330 # Otherwise, display the tag on its own line.
331 if (len(tag) + len(msg) < 75 and
332 msg.find('\n', 0, len(msg)-1) == -1):
333 return '%s: %s' % (tag, msg)
334 else:
335 if indent_msg:
336 msg = '\n'.join([INDENT+l for l in msg.split('\n')])
337 msg = msg[:-len(INDENT)]
338 return '%s:\n%s' % (tag, msg)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000339
Tim Peters8485b562004-08-04 18:46:34 +0000340# Override some StringIO methods.
341class _SpoofOut(StringIO):
342 def getvalue(self):
343 result = StringIO.getvalue(self)
344 # If anything at all was written, make sure there's a trailing
345 # newline. There's no way for the expected output to indicate
346 # that a trailing newline is missing.
347 if result and not result.endswith("\n"):
348 result += "\n"
349 # Prevent softspace from screwing up the next test case, in
350 # case they used print with a trailing comma in an example.
351 if hasattr(self, "softspace"):
352 del self.softspace
353 return result
Tim Peters8a7d2d52001-01-16 07:10:57 +0000354
Tim Peters8485b562004-08-04 18:46:34 +0000355 def truncate(self, size=None):
356 StringIO.truncate(self, size)
357 if hasattr(self, "softspace"):
358 del self.softspace
Tim Peters8a7d2d52001-01-16 07:10:57 +0000359
Tim Peters8485b562004-08-04 18:46:34 +0000360######################################################################
361## 2. Example & DocTest
362######################################################################
363## - An "example" is a <source, want> pair, where "source" is a
364## fragment of source code, and "want" is the expected output for
365## "source." The Example class also includes information about
366## where the example was extracted from.
367##
368## - A "doctest" is a collection of examples extracted from a string
369## (such as an object's docstring). The DocTest class also includes
370## information about where the string was extracted from.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000371
Tim Peters8485b562004-08-04 18:46:34 +0000372class Example:
373 """
374 A single doctest example, consisting of source code and expected
375 output. Example defines the following attributes:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000376
Tim Petersbb431472004-08-09 03:51:46 +0000377 - source: A single Python statement, always ending with a newline.
378 The constructor adds a newline if needed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000379
Tim Petersbb431472004-08-09 03:51:46 +0000380 - want: The expected output from running the source code (either
381 from stdout, or a traceback in case of exception). `want` ends
382 with a newline unless it's empty, in which case it's an empty
383 string. The constructor adds a newline if needed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000384
Tim Petersbb431472004-08-09 03:51:46 +0000385 - lineno: The line number within the DocTest string containing
Tim Peters8485b562004-08-04 18:46:34 +0000386 this Example where the Example begins. This line number is
387 zero-based, with respect to the beginning of the DocTest.
388 """
389 def __init__(self, source, want, lineno):
Tim Petersbb431472004-08-09 03:51:46 +0000390 # Normalize inputs.
391 if not source.endswith('\n'):
392 source += '\n'
393 if want and not want.endswith('\n'):
394 want += '\n'
Tim Peters8485b562004-08-04 18:46:34 +0000395 # Store properties.
396 self.source = source
397 self.want = want
398 self.lineno = lineno
Tim Peters8a7d2d52001-01-16 07:10:57 +0000399
Tim Peters8485b562004-08-04 18:46:34 +0000400class DocTest:
401 """
402 A collection of doctest examples that should be run in a single
403 namespace. Each DocTest defines the following attributes:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000404
Tim Peters8485b562004-08-04 18:46:34 +0000405 - examples: the list of examples.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000406
Tim Peters8485b562004-08-04 18:46:34 +0000407 - globs: The namespace (aka globals) that the examples should
408 be run in.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000409
Tim Peters8485b562004-08-04 18:46:34 +0000410 - name: A name identifying the DocTest (typically, the name of
411 the object whose docstring this DocTest was extracted from).
Tim Peters8a7d2d52001-01-16 07:10:57 +0000412
Tim Peters19397e52004-08-06 22:02:59 +0000413 - docstring: The docstring being tested
414
Tim Peters8485b562004-08-04 18:46:34 +0000415 - filename: The name of the file that this DocTest was extracted
416 from.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000417
Tim Peters8485b562004-08-04 18:46:34 +0000418 - lineno: The line number within filename where this DocTest
419 begins. This line number is zero-based, with respect to the
420 beginning of the file.
421 """
422 def __init__(self, docstring, globs, name, filename, lineno):
423 """
424 Create a new DocTest, by extracting examples from `docstring`.
425 The DocTest's globals are initialized with a copy of `globs`.
426 """
427 # Store a copy of the globals
428 self.globs = globs.copy()
429 # Store identifying information
430 self.name = name
431 self.filename = filename
432 self.lineno = lineno
433 # Parse the docstring.
Tim Peters19397e52004-08-06 22:02:59 +0000434 self.docstring = docstring
Edward Loper78b58f32004-08-09 02:56:02 +0000435 self.examples = Parser(name, docstring).get_examples()
Tim Peters8485b562004-08-04 18:46:34 +0000436
437 def __repr__(self):
438 if len(self.examples) == 0:
439 examples = 'no examples'
440 elif len(self.examples) == 1:
441 examples = '1 example'
442 else:
443 examples = '%d examples' % len(self.examples)
444 return ('<DocTest %s from %s:%s (%s)>' %
445 (self.name, self.filename, self.lineno, examples))
446
447
448 # This lets us sort tests by name:
449 def __cmp__(self, other):
450 if not isinstance(other, DocTest):
451 return -1
452 return cmp((self.name, self.filename, self.lineno, id(self)),
453 (other.name, other.filename, other.lineno, id(other)))
454
455######################################################################
Edward Loper7c748462004-08-09 02:06:06 +0000456## 2. Example Parser
457######################################################################
458
459class Parser:
460 """
461 Extract doctests from a string.
462 """
463 def __init__(self, name, string):
464 """
465 Prepare to extract doctests from string `string`.
466
467 `name` is an arbitrary (string) name associated with the string,
468 and is used only in error messages.
469 """
470 self.name = name
471 self.string = string.expandtabs()
472
473 _EXAMPLE_RE = re.compile(r'''
Tim Petersd40a92b2004-08-09 03:28:45 +0000474 # Source consists of a PS1 line followed by zero or more PS2 lines.
475 (?P<source>
476 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line
477 (?:\n [ ]* \.\.\. .*)*) # PS2 lines
478 \n?
479 # Want consists of any non-blank lines that do not start with PS1.
480 (?P<want> (?:(?![ ]*$) # Not a blank line
481 (?![ ]*>>>) # Not a line starting with PS1
482 .*$\n? # But any other line
483 )*)
484 ''', re.MULTILINE | re.VERBOSE)
485 _IS_BLANK_OR_COMMENT = re.compile('^[ ]*(#.*)?$').match
Edward Loper7c748462004-08-09 02:06:06 +0000486
487 def get_examples(self):
488 """
Edward Loper78b58f32004-08-09 02:56:02 +0000489 Extract all doctest examples, from the string, and return them
490 as a list of `Example` objects. Line numbers are 0-based,
491 because it's most common in doctests that nothing interesting
492 appears on the same line as opening triple-quote, and so the
493 first interesting line is called \"line 1\" then.
Edward Loper7c748462004-08-09 02:06:06 +0000494
495 >>> text = '''
496 ... >>> x, y = 2, 3 # no output expected
497 ... >>> if 1:
498 ... ... print x
499 ... ... print y
500 ... 2
501 ... 3
502 ...
503 ... Some text.
504 ... >>> x+y
505 ... 5
506 ... '''
507 >>> for x in Parser('<string>', text).get_examples():
Edward Loper78b58f32004-08-09 02:56:02 +0000508 ... print (x.source, x.want, x.lineno)
Tim Petersbb431472004-08-09 03:51:46 +0000509 ('x, y = 2, 3 # no output expected\\n', '', 1)
Edward Loper7c748462004-08-09 02:06:06 +0000510 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2)
Tim Petersbb431472004-08-09 03:51:46 +0000511 ('x+y\\n', '5\\n', 9)
Edward Loper7c748462004-08-09 02:06:06 +0000512 """
513 examples = []
514 charno, lineno = 0, 0
515 # Find all doctest examples in the string:
516 for m in self._EXAMPLE_RE.finditer(self.string):
517 # Update lineno (lines before this example)
518 lineno += self.string.count('\n', charno, m.start())
519
520 # Extract source/want from the regexp match.
521 (source, want) = self._parse_example(m, lineno)
Tim Petersd40a92b2004-08-09 03:28:45 +0000522 if self._IS_BLANK_OR_COMMENT(source):
Edward Loper7c748462004-08-09 02:06:06 +0000523 continue
Edward Loper78b58f32004-08-09 02:56:02 +0000524 examples.append( Example(source, want, lineno) )
Edward Loper7c748462004-08-09 02:06:06 +0000525
526 # Update lineno (lines inside this example)
527 lineno += self.string.count('\n', m.start(), m.end())
528 # Update charno.
529 charno = m.end()
530 return examples
531
532 def get_program(self):
533 """
534 Return an executable program from the string, as a string.
535
536 The format of this isn't rigidly defined. In general, doctest
537 examples become the executable statements in the result, and
538 their expected outputs become comments, preceded by an \"#Expected:\"
539 comment. Everything else (text, comments, everything not part of
540 a doctest test) is also placed in comments.
541
542 >>> text = '''
543 ... >>> x, y = 2, 3 # no output expected
544 ... >>> if 1:
545 ... ... print x
546 ... ... print y
547 ... 2
548 ... 3
549 ...
550 ... Some text.
551 ... >>> x+y
552 ... 5
553 ... '''
554 >>> print Parser('<string>', text).get_program()
555 x, y = 2, 3 # no output expected
556 if 1:
557 print x
558 print y
559 # Expected:
560 # 2
561 # 3
562 #
563 # Some text.
564 x+y
565 # Expected:
566 # 5
567 """
568 output = []
569 charnum, lineno = 0, 0
570 # Find all doctest examples in the string:
571 for m in self._EXAMPLE_RE.finditer(self.string):
572 # Add any text before this example, as a comment.
573 if m.start() > charnum:
574 lines = self.string[charnum:m.start()-1].split('\n')
575 output.extend([self._comment_line(l) for l in lines])
576 lineno += len(lines)
577
578 # Extract source/want from the regexp match.
579 (source, want) = self._parse_example(m, lineno, False)
580 # Display the source
581 output.append(source)
582 # Display the expected output, if any
583 if want:
584 output.append('# Expected:')
585 output.extend(['# '+l for l in want.split('\n')])
586
587 # Update the line number & char number.
588 lineno += self.string.count('\n', m.start(), m.end())
589 charnum = m.end()
590 # Add any remaining text, as comments.
591 output.extend([self._comment_line(l)
592 for l in self.string[charnum:].split('\n')])
593 # Trim junk on both ends.
594 while output and output[-1] == '#':
595 output.pop()
596 while output and output[0] == '#':
597 output.pop(0)
598 # Combine the output, and return it.
599 return '\n'.join(output)
600
601 def _parse_example(self, m, lineno, add_newlines=True):
602 # Get the example's indentation level.
603 indent = len(m.group('indent'))
604
605 # Divide source into lines; check that they're properly
606 # indented; and then strip their indentation & prompts.
607 source_lines = m.group('source').split('\n')
608 self._check_prompt_blank(source_lines, indent, lineno)
609 self._check_prefix(source_lines[1:], ' '*indent+'.', lineno)
610 source = '\n'.join([sl[indent+4:] for sl in source_lines])
611 if len(source_lines) > 1 and add_newlines:
612 source += '\n'
613
614 # Divide want into lines; check that it's properly
615 # indented; and then strip the indentation.
616 want_lines = m.group('want').rstrip().split('\n')
617 self._check_prefix(want_lines, ' '*indent,
618 lineno+len(source_lines))
619 want = '\n'.join([wl[indent:] for wl in want_lines])
620 if len(want) > 0 and add_newlines:
621 want += '\n'
622
623 return source, want
624
625 def _comment_line(self, line):
626 line = line.rstrip()
Tim Petersdd0e4752004-08-09 03:31:56 +0000627 if line:
628 return '# '+line
629 else:
630 return '#'
Edward Loper7c748462004-08-09 02:06:06 +0000631
632 def _check_prompt_blank(self, lines, indent, lineno):
633 for i, line in enumerate(lines):
634 if len(line) >= indent+4 and line[indent+3] != ' ':
635 raise ValueError('line %r of the docstring for %s '
636 'lacks blank after %s: %r' %
637 (lineno+i+1, self.name,
638 line[indent:indent+3], line))
639
640 def _check_prefix(self, lines, prefix, lineno):
641 for i, line in enumerate(lines):
642 if line and not line.startswith(prefix):
643 raise ValueError('line %r of the docstring for %s has '
644 'inconsistent leading whitespace: %r' %
645 (lineno+i+1, self.name, line))
646
647
648######################################################################
649## 4. DocTest Finder
Tim Peters8485b562004-08-04 18:46:34 +0000650######################################################################
651
652class DocTestFinder:
653 """
654 A class used to extract the DocTests that are relevant to a given
655 object, from its docstring and the docstrings of its contained
656 objects. Doctests can currently be extracted from the following
657 object types: modules, functions, classes, methods, staticmethods,
658 classmethods, and properties.
Tim Peters8485b562004-08-04 18:46:34 +0000659 """
660
Tim Peters19397e52004-08-06 22:02:59 +0000661 def __init__(self, verbose=False, doctest_factory=DocTest,
Tim Petersf727c6c2004-08-08 01:48:59 +0000662 recurse=True, _namefilter=None):
Tim Peters8485b562004-08-04 18:46:34 +0000663 """
664 Create a new doctest finder.
665
Tim Peters19397e52004-08-06 22:02:59 +0000666 The optional argument `doctest_factory` specifies a class or
667 function that should be used to create new DocTest objects (or
Tim Peters161c9632004-08-08 03:38:33 +0000668 objects that implement the same interface as DocTest). The
Tim Peters19397e52004-08-06 22:02:59 +0000669 signature for this factory function should match the signature
670 of the DocTest constructor.
671
Tim Peters8485b562004-08-04 18:46:34 +0000672 If the optional argument `recurse` is false, then `find` will
673 only examine the given object, and not any contained objects.
674 """
Tim Peters19397e52004-08-06 22:02:59 +0000675 self._doctest_factory = doctest_factory
Tim Peters8485b562004-08-04 18:46:34 +0000676 self._verbose = verbose
Tim Peters8485b562004-08-04 18:46:34 +0000677 self._recurse = recurse
Tim Petersf727c6c2004-08-08 01:48:59 +0000678 # _namefilter is undocumented, and exists only for temporary backward-
679 # compatibility support of testmod's deprecated isprivate mess.
680 self._namefilter = _namefilter
Tim Peters8485b562004-08-04 18:46:34 +0000681
682 def find(self, obj, name=None, module=None, globs=None,
Tim Petersf3f57472004-08-08 06:11:48 +0000683 extraglobs=None):
Tim Peters8485b562004-08-04 18:46:34 +0000684 """
685 Return a list of the DocTests that are defined by the given
686 object's docstring, or by any of its contained objects'
687 docstrings.
688
689 The optional parameter `module` is the module that contains
Tim Petersf3f57472004-08-08 06:11:48 +0000690 the given object. If the module is not specified or is None, then
691 the test finder will attempt to automatically determine the
Tim Peters8485b562004-08-04 18:46:34 +0000692 correct module. The object's module is used:
693
694 - As a default namespace, if `globs` is not specified.
695 - To prevent the DocTestFinder from extracting DocTests
Tim Petersf3f57472004-08-08 06:11:48 +0000696 from objects that are imported from other modules.
Tim Peters8485b562004-08-04 18:46:34 +0000697 - To find the name of the file containing the object.
698 - To help find the line number of the object within its
699 file.
700
Tim Petersf3f57472004-08-08 06:11:48 +0000701 Contained objects whose module does not match `module` are ignored.
702
703 If `module` is False, no attempt to find the module will be made.
704 This is obscure, of use mostly in tests: if `module` is False, or
705 is None but cannot be found automatically, then all objects are
706 considered to belong to the (non-existent) module, so all contained
707 objects will (recursively) be searched for doctests.
708
Tim Peters8485b562004-08-04 18:46:34 +0000709 The globals for each DocTest is formed by combining `globs`
710 and `extraglobs` (bindings in `extraglobs` override bindings
711 in `globs`). A new copy of the globals dictionary is created
712 for each DocTest. If `globs` is not specified, then it
713 defaults to the module's `__dict__`, if specified, or {}
714 otherwise. If `extraglobs` is not specified, then it defaults
715 to {}.
716
Tim Peters8485b562004-08-04 18:46:34 +0000717 """
718 # If name was not specified, then extract it from the object.
719 if name is None:
720 name = getattr(obj, '__name__', None)
721 if name is None:
722 raise ValueError("DocTestFinder.find: name must be given "
723 "when obj.__name__ doesn't exist: %r" %
724 (type(obj),))
725
726 # Find the module that contains the given object (if obj is
727 # a module, then module=obj.). Note: this may fail, in which
728 # case module will be None.
Tim Petersf3f57472004-08-08 06:11:48 +0000729 if module is False:
730 module = None
731 elif module is None:
Tim Peters8485b562004-08-04 18:46:34 +0000732 module = inspect.getmodule(obj)
733
734 # Read the module's source code. This is used by
735 # DocTestFinder._find_lineno to find the line number for a
736 # given object's docstring.
737 try:
738 file = inspect.getsourcefile(obj) or inspect.getfile(obj)
739 source_lines = linecache.getlines(file)
740 if not source_lines:
741 source_lines = None
742 except TypeError:
743 source_lines = None
744
745 # Initialize globals, and merge in extraglobs.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000746 if globs is None:
Tim Peters8485b562004-08-04 18:46:34 +0000747 if module is None:
748 globs = {}
749 else:
750 globs = module.__dict__.copy()
751 else:
752 globs = globs.copy()
753 if extraglobs is not None:
754 globs.update(extraglobs)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000755
Tim Peters8485b562004-08-04 18:46:34 +0000756 # Recursively expore `obj`, extracting DocTests.
757 tests = []
Tim Petersf3f57472004-08-08 06:11:48 +0000758 self._find(tests, obj, name, module, source_lines, globs, {})
Tim Peters8485b562004-08-04 18:46:34 +0000759 return tests
760
761 def _filter(self, obj, prefix, base):
762 """
763 Return true if the given object should not be examined.
764 """
Tim Petersf727c6c2004-08-08 01:48:59 +0000765 return (self._namefilter is not None and
766 self._namefilter(prefix, base))
Tim Peters8485b562004-08-04 18:46:34 +0000767
768 def _from_module(self, module, object):
769 """
770 Return true if the given object is defined in the given
771 module.
772 """
773 if module is None:
774 return True
775 elif inspect.isfunction(object):
776 return module.__dict__ is object.func_globals
777 elif inspect.isclass(object):
778 return module.__name__ == object.__module__
779 elif inspect.getmodule(object) is not None:
780 return module is inspect.getmodule(object)
781 elif hasattr(object, '__module__'):
782 return module.__name__ == object.__module__
783 elif isinstance(object, property):
784 return True # [XX] no way not be sure.
785 else:
786 raise ValueError("object must be a class or function")
787
Tim Petersf3f57472004-08-08 06:11:48 +0000788 def _find(self, tests, obj, name, module, source_lines, globs, seen):
Tim Peters8485b562004-08-04 18:46:34 +0000789 """
790 Find tests for the given object and any contained objects, and
791 add them to `tests`.
792 """
793 if self._verbose:
794 print 'Finding tests in %s' % name
795
796 # If we've already processed this object, then ignore it.
797 if id(obj) in seen:
798 return
799 seen[id(obj)] = 1
800
801 # Find a test for this object, and add it to the list of tests.
802 test = self._get_test(obj, name, module, globs, source_lines)
803 if test is not None:
804 tests.append(test)
805
806 # Look for tests in a module's contained objects.
807 if inspect.ismodule(obj) and self._recurse:
808 for valname, val in obj.__dict__.items():
809 # Check if this contained object should be ignored.
810 if self._filter(val, name, valname):
811 continue
812 valname = '%s.%s' % (name, valname)
813 # Recurse to functions & classes.
814 if ((inspect.isfunction(val) or inspect.isclass(val)) and
Tim Petersf3f57472004-08-08 06:11:48 +0000815 self._from_module(module, val)):
Tim Peters8485b562004-08-04 18:46:34 +0000816 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +0000817 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +0000818
819 # Look for tests in a module's __test__ dictionary.
820 if inspect.ismodule(obj) and self._recurse:
821 for valname, val in getattr(obj, '__test__', {}).items():
822 if not isinstance(valname, basestring):
823 raise ValueError("DocTestFinder.find: __test__ keys "
824 "must be strings: %r" %
825 (type(valname),))
826 if not (inspect.isfunction(val) or inspect.isclass(val) or
827 inspect.ismethod(val) or inspect.ismodule(val) or
828 isinstance(val, basestring)):
829 raise ValueError("DocTestFinder.find: __test__ values "
830 "must be strings, functions, methods, "
831 "classes, or modules: %r" %
832 (type(val),))
833 valname = '%s.%s' % (name, valname)
834 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +0000835 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +0000836
837 # Look for tests in a class's contained objects.
838 if inspect.isclass(obj) and self._recurse:
839 for valname, val in obj.__dict__.items():
840 # Check if this contained object should be ignored.
841 if self._filter(val, name, valname):
842 continue
843 # Special handling for staticmethod/classmethod.
844 if isinstance(val, staticmethod):
845 val = getattr(obj, valname)
846 if isinstance(val, classmethod):
847 val = getattr(obj, valname).im_func
848
849 # Recurse to methods, properties, and nested classes.
850 if ((inspect.isfunction(val) or inspect.isclass(val) or
Tim Petersf3f57472004-08-08 06:11:48 +0000851 isinstance(val, property)) and
852 self._from_module(module, val)):
Tim Peters8485b562004-08-04 18:46:34 +0000853 valname = '%s.%s' % (name, valname)
854 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +0000855 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +0000856
857 def _get_test(self, obj, name, module, globs, source_lines):
858 """
859 Return a DocTest for the given object, if it defines a docstring;
860 otherwise, return None.
861 """
862 # Extract the object's docstring. If it doesn't have one,
863 # then return None (no test for this object).
864 if isinstance(obj, basestring):
865 docstring = obj
866 else:
867 try:
868 if obj.__doc__ is None:
869 return None
870 docstring = str(obj.__doc__)
871 except (TypeError, AttributeError):
872 return None
873
874 # Don't bother if the docstring is empty.
875 if not docstring:
876 return None
877
878 # Find the docstring's location in the file.
879 lineno = self._find_lineno(obj, source_lines)
880
881 # Return a DocTest for this object.
882 if module is None:
883 filename = None
884 else:
885 filename = getattr(module, '__file__', module.__name__)
Tim Peters19397e52004-08-06 22:02:59 +0000886 return self._doctest_factory(docstring, globs, name, filename, lineno)
Tim Peters8485b562004-08-04 18:46:34 +0000887
888 def _find_lineno(self, obj, source_lines):
889 """
890 Return a line number of the given object's docstring. Note:
891 this method assumes that the object has a docstring.
892 """
893 lineno = None
894
895 # Find the line number for modules.
896 if inspect.ismodule(obj):
897 lineno = 0
898
899 # Find the line number for classes.
900 # Note: this could be fooled if a class is defined multiple
901 # times in a single file.
902 if inspect.isclass(obj):
903 if source_lines is None:
904 return None
905 pat = re.compile(r'^\s*class\s*%s\b' %
906 getattr(obj, '__name__', '-'))
907 for i, line in enumerate(source_lines):
908 if pat.match(line):
909 lineno = i
910 break
911
912 # Find the line number for functions & methods.
913 if inspect.ismethod(obj): obj = obj.im_func
914 if inspect.isfunction(obj): obj = obj.func_code
915 if inspect.istraceback(obj): obj = obj.tb_frame
916 if inspect.isframe(obj): obj = obj.f_code
917 if inspect.iscode(obj):
918 lineno = getattr(obj, 'co_firstlineno', None)-1
919
920 # Find the line number where the docstring starts. Assume
921 # that it's the first line that begins with a quote mark.
922 # Note: this could be fooled by a multiline function
923 # signature, where a continuation line begins with a quote
924 # mark.
925 if lineno is not None:
926 if source_lines is None:
927 return lineno+1
928 pat = re.compile('(^|.*:)\s*\w*("|\')')
929 for lineno in range(lineno, len(source_lines)):
930 if pat.match(source_lines[lineno]):
931 return lineno
932
933 # We couldn't find the line number.
934 return None
935
936######################################################################
Edward Loper7c748462004-08-09 02:06:06 +0000937## 5. DocTest Runner
Tim Peters8485b562004-08-04 18:46:34 +0000938######################################################################
939
Tim Peters8485b562004-08-04 18:46:34 +0000940class DocTestRunner:
941 """
942 A class used to run DocTest test cases, and accumulate statistics.
943 The `run` method is used to process a single DocTest case. It
944 returns a tuple `(f, t)`, where `t` is the number of test cases
945 tried, and `f` is the number of test cases that failed.
946
947 >>> tests = DocTestFinder().find(_TestClass)
948 >>> runner = DocTestRunner(verbose=False)
949 >>> for test in tests:
950 ... print runner.run(test)
951 (0, 2)
952 (0, 1)
953 (0, 2)
954 (0, 2)
955
956 The `summarize` method prints a summary of all the test cases that
957 have been run by the runner, and returns an aggregated `(f, t)`
958 tuple:
959
960 >>> runner.summarize(verbose=1)
961 4 items passed all tests:
962 2 tests in _TestClass
963 2 tests in _TestClass.__init__
964 2 tests in _TestClass.get
965 1 tests in _TestClass.square
966 7 tests in 4 items.
967 7 passed and 0 failed.
968 Test passed.
969 (0, 7)
970
971 The aggregated number of tried examples and failed examples is
972 also available via the `tries` and `failures` attributes:
973
974 >>> runner.tries
975 7
976 >>> runner.failures
977 0
978
979 The comparison between expected outputs and actual outputs is done
Edward Loper34fcb142004-08-09 02:45:41 +0000980 by an `OutputChecker`. This comparison may be customized with a
981 number of option flags; see the documentation for `testmod` for
982 more information. If the option flags are insufficient, then the
983 comparison may also be customized by passing a subclass of
984 `OutputChecker` to the constructor.
Tim Peters8485b562004-08-04 18:46:34 +0000985
986 The test runner's display output can be controlled in two ways.
987 First, an output function (`out) can be passed to
988 `TestRunner.run`; this function will be called with strings that
989 should be displayed. It defaults to `sys.stdout.write`. If
990 capturing the output is not sufficient, then the display output
991 can be also customized by subclassing DocTestRunner, and
992 overriding the methods `report_start`, `report_success`,
993 `report_unexpected_exception`, and `report_failure`.
994 """
995 # This divider string is used to separate failure messages, and to
996 # separate sections of the summary.
997 DIVIDER = "*" * 70
998
Edward Loper34fcb142004-08-09 02:45:41 +0000999 def __init__(self, checker=None, verbose=None, optionflags=0):
Tim Peters8485b562004-08-04 18:46:34 +00001000 """
1001 Create a new test runner.
1002
Edward Loper34fcb142004-08-09 02:45:41 +00001003 Optional keyword arg `checker` is the `OutputChecker` that
1004 should be used to compare the expected outputs and actual
1005 outputs of doctest examples.
1006
Tim Peters8485b562004-08-04 18:46:34 +00001007 Optional keyword arg 'verbose' prints lots of stuff if true,
1008 only failures if false; by default, it's true iff '-v' is in
1009 sys.argv.
1010
1011 Optional argument `optionflags` can be used to control how the
1012 test runner compares expected output to actual output, and how
1013 it displays failures. See the documentation for `testmod` for
1014 more information.
1015 """
Edward Loper34fcb142004-08-09 02:45:41 +00001016 self._checker = checker or OutputChecker()
Tim Peters8a7d2d52001-01-16 07:10:57 +00001017 if verbose is None:
Tim Peters8485b562004-08-04 18:46:34 +00001018 verbose = '-v' in sys.argv
1019 self._verbose = verbose
Tim Peters6ebe61f2003-06-27 20:48:05 +00001020 self.optionflags = optionflags
1021
Tim Peters8485b562004-08-04 18:46:34 +00001022 # Keep track of the examples we've run.
1023 self.tries = 0
1024 self.failures = 0
1025 self._name2ft = {}
Tim Peters8a7d2d52001-01-16 07:10:57 +00001026
Tim Peters8485b562004-08-04 18:46:34 +00001027 # Create a fake output target for capturing doctest output.
1028 self._fakeout = _SpoofOut()
Tim Peters4fd9e2f2001-08-18 00:05:50 +00001029
Tim Peters8485b562004-08-04 18:46:34 +00001030 #/////////////////////////////////////////////////////////////////
Tim Peters8485b562004-08-04 18:46:34 +00001031 # Reporting methods
1032 #/////////////////////////////////////////////////////////////////
Tim Peters17111f32001-10-03 04:08:26 +00001033
Tim Peters8485b562004-08-04 18:46:34 +00001034 def report_start(self, out, test, example):
Tim Peters8a7d2d52001-01-16 07:10:57 +00001035 """
Tim Peters8485b562004-08-04 18:46:34 +00001036 Report that the test runner is about to process the given
1037 example. (Only displays a message if verbose=True)
1038 """
1039 if self._verbose:
1040 out(_tag_msg("Trying", example.source) +
1041 _tag_msg("Expecting", example.want or "nothing"))
Tim Peters8a7d2d52001-01-16 07:10:57 +00001042
Tim Peters8485b562004-08-04 18:46:34 +00001043 def report_success(self, out, test, example, got):
1044 """
1045 Report that the given example ran successfully. (Only
1046 displays a message if verbose=True)
1047 """
1048 if self._verbose:
1049 out("ok\n")
Tim Peters8a7d2d52001-01-16 07:10:57 +00001050
Tim Peters8485b562004-08-04 18:46:34 +00001051 def report_failure(self, out, test, example, got):
1052 """
1053 Report that the given example failed.
1054 """
1055 # Print an error message.
1056 out(self.__failure_header(test, example) +
Edward Loper34fcb142004-08-09 02:45:41 +00001057 self._checker.output_difference(example.want, got,
1058 self.optionflags))
Tim Peters7402f792001-10-02 03:53:41 +00001059
Tim Peters8485b562004-08-04 18:46:34 +00001060 def report_unexpected_exception(self, out, test, example, exc_info):
1061 """
1062 Report that the given example raised an unexpected exception.
1063 """
1064 # Get a traceback message.
1065 excout = StringIO()
1066 exc_type, exc_val, exc_tb = exc_info
1067 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout)
1068 exception_tb = excout.getvalue()
1069 # Print an error message.
1070 out(self.__failure_header(test, example) +
1071 _tag_msg("Exception raised", exception_tb))
Tim Peters7402f792001-10-02 03:53:41 +00001072
Tim Peters8485b562004-08-04 18:46:34 +00001073 def __failure_header(self, test, example):
1074 s = (self.DIVIDER + "\n" +
1075 _tag_msg("Failure in example", example.source))
1076 if test.filename is None:
1077 # [XX] I'm not putting +1 here, to give the same output
1078 # as the old version. But I think it *should* go here.
1079 return s + ("from line #%s of %s\n" %
1080 (example.lineno, test.name))
1081 elif test.lineno is None:
1082 return s + ("from line #%s of %s in %s\n" %
1083 (example.lineno+1, test.name, test.filename))
1084 else:
1085 lineno = test.lineno+example.lineno+1
1086 return s + ("from line #%s of %s (%s)\n" %
1087 (lineno, test.filename, test.name))
Tim Peters7402f792001-10-02 03:53:41 +00001088
Tim Peters8485b562004-08-04 18:46:34 +00001089 #/////////////////////////////////////////////////////////////////
1090 # DocTest Running
1091 #/////////////////////////////////////////////////////////////////
Tim Peters7402f792001-10-02 03:53:41 +00001092
Tim Peters8485b562004-08-04 18:46:34 +00001093 # A regular expression for handling `want` strings that contain
1094 # expected exceptions. It divides `want` into two pieces: the
1095 # pre-exception output (`out`) and the exception message (`exc`),
1096 # as generated by traceback.format_exception_only(). (I assume
1097 # that the exception_only message is the first non-indented line
1098 # starting with word characters after the "Traceback ...".)
1099 _EXCEPTION_RE = re.compile(('^(?P<out>.*)'
1100 '^(?P<hdr>Traceback \((?:%s|%s)\):)\s*$.*?'
1101 '^(?P<exc>\w+.*)') %
1102 ('most recent call last', 'innermost last'),
1103 re.MULTILINE | re.DOTALL)
Tim Peters7402f792001-10-02 03:53:41 +00001104
Tim Peters8485b562004-08-04 18:46:34 +00001105 _OPTION_DIRECTIVE_RE = re.compile('\s*doctest:\s*(?P<flags>[^#\n]*)')
Tim Peters7402f792001-10-02 03:53:41 +00001106
Tim Peters8485b562004-08-04 18:46:34 +00001107 def __handle_directive(self, example):
1108 """
1109 Check if the given example is actually a directive to doctest
1110 (to turn an optionflag on or off); and if it is, then handle
1111 the directive.
Tim Peters7402f792001-10-02 03:53:41 +00001112
Tim Peters8485b562004-08-04 18:46:34 +00001113 Return true iff the example is actually a directive (and so
1114 should not be executed).
Tim Peters4a9ac4a2001-10-02 22:47:08 +00001115
Tim Peters8a7d2d52001-01-16 07:10:57 +00001116 """
Tim Peters8485b562004-08-04 18:46:34 +00001117 m = self._OPTION_DIRECTIVE_RE.match(example.source)
1118 if m is None:
1119 return False
Tim Peters8a7d2d52001-01-16 07:10:57 +00001120
Tim Peters8485b562004-08-04 18:46:34 +00001121 for flag in m.group('flags').upper().split():
1122 if (flag[:1] not in '+-' or
1123 flag[1:] not in OPTIONFLAGS_BY_NAME):
1124 raise ValueError('Bad doctest option directive: '+flag)
1125 if flag[0] == '+':
1126 self.optionflags |= OPTIONFLAGS_BY_NAME[flag[1:]]
1127 else:
1128 self.optionflags &= ~OPTIONFLAGS_BY_NAME[flag[1:]]
1129 return True
Tim Peters8a7d2d52001-01-16 07:10:57 +00001130
Tim Peters8485b562004-08-04 18:46:34 +00001131 def __run(self, test, compileflags, out):
Tim Peters8a7d2d52001-01-16 07:10:57 +00001132 """
Tim Peters8485b562004-08-04 18:46:34 +00001133 Run the examples in `test`. Write the outcome of each example
1134 with one of the `DocTestRunner.report_*` methods, using the
1135 writer function `out`. `compileflags` is the set of compiler
1136 flags that should be used to execute examples. Return a tuple
1137 `(f, t)`, where `t` is the number of examples tried, and `f`
1138 is the number of examples that failed. The examples are run
1139 in the namespace `test.globs`.
1140 """
1141 # Keep track of the number of failures and tries.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001142 failures = tries = 0
Tim Peters8485b562004-08-04 18:46:34 +00001143
1144 # Save the option flags (since option directives can be used
1145 # to modify them).
1146 original_optionflags = self.optionflags
1147
1148 # Process each example.
1149 for example in test.examples:
1150 # Check if it's an option directive. If it is, then handle
1151 # it, and go on to the next example.
1152 if self.__handle_directive(example):
1153 continue
1154
1155 # Record that we started this example.
1156 tries += 1
1157 self.report_start(out, test, example)
1158
1159 # Run the example in the given context (globs), and record
1160 # any exception that gets raised. (But don't intercept
1161 # keyboard interrupts.)
1162 try:
Tim Peters208ca702004-08-09 04:12:36 +00001163 # Don't blink! This is where the user's code gets run.
Tim Petersbb431472004-08-09 03:51:46 +00001164 exec compile(example.source, "<string>", "single",
Tim Peters8485b562004-08-04 18:46:34 +00001165 compileflags, 1) in test.globs
1166 exception = None
1167 except KeyboardInterrupt:
1168 raise
1169 except:
1170 exception = sys.exc_info()
1171
Tim Peters208ca702004-08-09 04:12:36 +00001172 got = self._fakeout.getvalue() # the actual output
Tim Peters8485b562004-08-04 18:46:34 +00001173 self._fakeout.truncate(0)
1174
1175 # If the example executed without raising any exceptions,
1176 # then verify its output and report its outcome.
1177 if exception is None:
Edward Loper34fcb142004-08-09 02:45:41 +00001178 if self._checker.check_output(example.want, got,
1179 self.optionflags):
Tim Peters8485b562004-08-04 18:46:34 +00001180 self.report_success(out, test, example, got)
Tim Peters8a7d2d52001-01-16 07:10:57 +00001181 else:
Tim Peters8485b562004-08-04 18:46:34 +00001182 self.report_failure(out, test, example, got)
1183 failures += 1
1184
1185 # If the example raised an exception, then check if it was
1186 # expected.
1187 else:
1188 exc_info = sys.exc_info()
1189 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1]
1190
1191 # Search the `want` string for an exception. If we don't
1192 # find one, then report an unexpected exception.
1193 m = self._EXCEPTION_RE.match(example.want)
1194 if m is None:
1195 self.report_unexpected_exception(out, test, example,
1196 exc_info)
1197 failures += 1
1198 else:
1199 exc_hdr = m.group('hdr')+'\n' # Exception header
1200 # The test passes iff the pre-exception output and
1201 # the exception description match the values given
1202 # in `want`.
Edward Loper34fcb142004-08-09 02:45:41 +00001203 if (self._checker.check_output(m.group('out'), got,
1204 self.optionflags) and
1205 self._checker.check_output(m.group('exc'), exc_msg,
1206 self.optionflags)):
Tim Peters8485b562004-08-04 18:46:34 +00001207 # Is +exc_msg the right thing here??
1208 self.report_success(out, test, example,
1209 got+exc_hdr+exc_msg)
1210 else:
1211 self.report_failure(out, test, example,
1212 got+exc_hdr+exc_msg)
1213 failures += 1
1214
1215 # Restore the option flags (in case they were modified)
1216 self.optionflags = original_optionflags
1217
1218 # Record and return the number of failures and tries.
1219 self.__record_outcome(test, failures, tries)
Tim Peters8a7d2d52001-01-16 07:10:57 +00001220 return failures, tries
1221
Tim Peters8485b562004-08-04 18:46:34 +00001222 def __record_outcome(self, test, f, t):
1223 """
1224 Record the fact that the given DocTest (`test`) generated `f`
1225 failures out of `t` tried examples.
1226 """
1227 f2, t2 = self._name2ft.get(test.name, (0,0))
1228 self._name2ft[test.name] = (f+f2, t+t2)
1229 self.failures += f
1230 self.tries += t
1231
1232 def run(self, test, compileflags=None, out=None, clear_globs=True):
1233 """
1234 Run the examples in `test`, and display the results using the
1235 writer function `out`.
1236
1237 The examples are run in the namespace `test.globs`. If
1238 `clear_globs` is true (the default), then this namespace will
1239 be cleared after the test runs, to help with garbage
1240 collection. If you would like to examine the namespace after
1241 the test completes, then use `clear_globs=False`.
1242
1243 `compileflags` gives the set of flags that should be used by
1244 the Python compiler when running the examples. If not
1245 specified, then it will default to the set of future-import
1246 flags that apply to `globs`.
1247
1248 The output of each example is checked using
1249 `DocTestRunner.check_output`, and the results are formatted by
1250 the `DocTestRunner.report_*` methods.
1251 """
1252 if compileflags is None:
1253 compileflags = _extract_future_flags(test.globs)
1254 if out is None:
1255 out = sys.stdout.write
1256 saveout = sys.stdout
1257
1258 try:
1259 sys.stdout = self._fakeout
1260 return self.__run(test, compileflags, out)
1261 finally:
1262 sys.stdout = saveout
Tim Peters8485b562004-08-04 18:46:34 +00001263 if clear_globs:
1264 test.globs.clear()
1265
1266 #/////////////////////////////////////////////////////////////////
1267 # Summarization
1268 #/////////////////////////////////////////////////////////////////
Tim Peters8a7d2d52001-01-16 07:10:57 +00001269 def summarize(self, verbose=None):
1270 """
Tim Peters8485b562004-08-04 18:46:34 +00001271 Print a summary of all the test cases that have been run by
1272 this DocTestRunner, and return a tuple `(f, t)`, where `f` is
1273 the total number of failed examples, and `t` is the total
1274 number of tried examples.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001275
Tim Peters8485b562004-08-04 18:46:34 +00001276 The optional `verbose` argument controls how detailed the
1277 summary is. If the verbosity is not specified, then the
1278 DocTestRunner's verbosity is used.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001279 """
Tim Peters8a7d2d52001-01-16 07:10:57 +00001280 if verbose is None:
Tim Peters8485b562004-08-04 18:46:34 +00001281 verbose = self._verbose
Tim Peters8a7d2d52001-01-16 07:10:57 +00001282 notests = []
1283 passed = []
1284 failed = []
1285 totalt = totalf = 0
Tim Peters8485b562004-08-04 18:46:34 +00001286 for x in self._name2ft.items():
Tim Peters8a7d2d52001-01-16 07:10:57 +00001287 name, (f, t) = x
1288 assert f <= t
Tim Peters8485b562004-08-04 18:46:34 +00001289 totalt += t
1290 totalf += f
Tim Peters8a7d2d52001-01-16 07:10:57 +00001291 if t == 0:
1292 notests.append(name)
1293 elif f == 0:
1294 passed.append( (name, t) )
1295 else:
1296 failed.append(x)
1297 if verbose:
1298 if notests:
1299 print len(notests), "items had no tests:"
1300 notests.sort()
1301 for thing in notests:
1302 print " ", thing
1303 if passed:
1304 print len(passed), "items passed all tests:"
1305 passed.sort()
1306 for thing, count in passed:
1307 print " %3d tests in %s" % (count, thing)
1308 if failed:
Tim Peters8485b562004-08-04 18:46:34 +00001309 print self.DIVIDER
Tim Peters8a7d2d52001-01-16 07:10:57 +00001310 print len(failed), "items had failures:"
1311 failed.sort()
1312 for thing, (f, t) in failed:
1313 print " %3d of %3d in %s" % (f, t, thing)
1314 if verbose:
Tim Peters8485b562004-08-04 18:46:34 +00001315 print totalt, "tests in", len(self._name2ft), "items."
Tim Peters8a7d2d52001-01-16 07:10:57 +00001316 print totalt - totalf, "passed and", totalf, "failed."
1317 if totalf:
1318 print "***Test Failed***", totalf, "failures."
1319 elif verbose:
1320 print "Test passed."
1321 return totalf, totalt
1322
Edward Loper34fcb142004-08-09 02:45:41 +00001323class OutputChecker:
1324 """
1325 A class used to check the whether the actual output from a doctest
1326 example matches the expected output. `OutputChecker` defines two
1327 methods: `check_output`, which compares a given pair of outputs,
1328 and returns true if they match; and `output_difference`, which
1329 returns a string describing the differences between two outputs.
1330 """
1331 def check_output(self, want, got, optionflags):
1332 """
1333 Return True iff the actual output (`got`) matches the expected
1334 output (`want`). These strings are always considered to match
1335 if they are identical; but depending on what option flags the
1336 test runner is using, several non-exact match types are also
1337 possible. See the documentation for `TestRunner` for more
1338 information about option flags.
1339 """
1340 # Handle the common case first, for efficiency:
1341 # if they're string-identical, always return true.
1342 if got == want:
1343 return True
1344
1345 # The values True and False replaced 1 and 0 as the return
1346 # value for boolean comparisons in Python 2.3.
1347 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1):
1348 if (got,want) == ("True\n", "1\n"):
1349 return True
1350 if (got,want) == ("False\n", "0\n"):
1351 return True
1352
1353 # <BLANKLINE> can be used as a special sequence to signify a
1354 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used.
1355 if not (optionflags & DONT_ACCEPT_BLANKLINE):
1356 # Replace <BLANKLINE> in want with a blank line.
1357 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER),
1358 '', want)
1359 # If a line in got contains only spaces, then remove the
1360 # spaces.
1361 got = re.sub('(?m)^\s*?$', '', got)
1362 if got == want:
1363 return True
1364
1365 # This flag causes doctest to ignore any differences in the
1366 # contents of whitespace strings. Note that this can be used
1367 # in conjunction with the ELLISPIS flag.
1368 if (optionflags & NORMALIZE_WHITESPACE):
1369 got = ' '.join(got.split())
1370 want = ' '.join(want.split())
1371 if got == want:
1372 return True
1373
1374 # The ELLIPSIS flag says to let the sequence "..." in `want`
1375 # match any substring in `got`. We implement this by
1376 # transforming `want` into a regular expression.
1377 if (optionflags & ELLIPSIS):
1378 # Escape any special regexp characters
1379 want_re = re.escape(want)
1380 # Replace ellipsis markers ('...') with .*
1381 want_re = want_re.replace(re.escape(ELLIPSIS_MARKER), '.*')
1382 # Require that it matches the entire string; and set the
1383 # re.DOTALL flag (with '(?s)').
1384 want_re = '(?s)^%s$' % want_re
1385 # Check if the `want_re` regexp matches got.
1386 if re.match(want_re, got):
1387 return True
1388
1389 # We didn't find any match; return false.
1390 return False
1391
1392 def output_difference(self, want, got, optionflags):
1393 """
1394 Return a string describing the differences between the
1395 expected output (`want`) and the actual output (`got`).
1396 """
1397 # If <BLANKLINE>s are being used, then replace <BLANKLINE>
1398 # with blank lines in the expected output string.
1399 if not (optionflags & DONT_ACCEPT_BLANKLINE):
1400 want = re.sub('(?m)^%s$' % re.escape(BLANKLINE_MARKER), '', want)
1401
1402 # Check if we should use diff. Don't use diff if the actual
1403 # or expected outputs are too short, or if the expected output
1404 # contains an ellipsis marker.
1405 if ((optionflags & (UNIFIED_DIFF | CONTEXT_DIFF)) and
1406 want.count('\n') > 2 and got.count('\n') > 2 and
1407 not (optionflags & ELLIPSIS and '...' in want)):
1408 # Split want & got into lines.
1409 want_lines = [l+'\n' for l in want.split('\n')]
1410 got_lines = [l+'\n' for l in got.split('\n')]
1411 # Use difflib to find their differences.
1412 if optionflags & UNIFIED_DIFF:
1413 diff = difflib.unified_diff(want_lines, got_lines, n=2,
1414 fromfile='Expected', tofile='Got')
1415 kind = 'unified'
1416 elif optionflags & CONTEXT_DIFF:
1417 diff = difflib.context_diff(want_lines, got_lines, n=2,
1418 fromfile='Expected', tofile='Got')
1419 kind = 'context'
1420 else:
1421 assert 0, 'Bad diff option'
1422 # Remove trailing whitespace on diff output.
1423 diff = [line.rstrip() + '\n' for line in diff]
1424 return _tag_msg("Differences (" + kind + " diff)",
1425 ''.join(diff))
1426
1427 # If we're not using diff, then simply list the expected
1428 # output followed by the actual output.
1429 return (_tag_msg("Expected", want or "Nothing") +
1430 _tag_msg("Got", got))
1431
Tim Peters19397e52004-08-06 22:02:59 +00001432class DocTestFailure(Exception):
1433 """A DocTest example has failed in debugging mode.
1434
1435 The exception instance has variables:
1436
1437 - test: the DocTest object being run
1438
1439 - excample: the Example object that failed
1440
1441 - got: the actual output
1442 """
1443 def __init__(self, test, example, got):
1444 self.test = test
1445 self.example = example
1446 self.got = got
1447
1448 def __str__(self):
1449 return str(self.test)
1450
1451class UnexpectedException(Exception):
1452 """A DocTest example has encountered an unexpected exception
1453
1454 The exception instance has variables:
1455
1456 - test: the DocTest object being run
1457
1458 - excample: the Example object that failed
1459
1460 - exc_info: the exception info
1461 """
1462 def __init__(self, test, example, exc_info):
1463 self.test = test
1464 self.example = example
1465 self.exc_info = exc_info
1466
1467 def __str__(self):
1468 return str(self.test)
Tim Petersd1b78272004-08-07 06:03:09 +00001469
Tim Peters19397e52004-08-06 22:02:59 +00001470class DebugRunner(DocTestRunner):
1471 r"""Run doc tests but raise an exception as soon as there is a failure.
1472
1473 If an unexpected exception occurs, an UnexpectedException is raised.
1474 It contains the test, the example, and the original exception:
1475
1476 >>> runner = DebugRunner(verbose=False)
1477 >>> test = DocTest('>>> raise KeyError\n42', {}, 'foo', 'foo.py', 0)
1478 >>> try:
1479 ... runner.run(test)
1480 ... except UnexpectedException, failure:
1481 ... pass
1482
1483 >>> failure.test is test
1484 True
1485
1486 >>> failure.example.want
1487 '42\n'
1488
1489 >>> exc_info = failure.exc_info
1490 >>> raise exc_info[0], exc_info[1], exc_info[2]
1491 Traceback (most recent call last):
1492 ...
1493 KeyError
1494
1495 We wrap the original exception to give the calling application
1496 access to the test and example information.
1497
1498 If the output doesn't match, then a DocTestFailure is raised:
1499
1500 >>> test = DocTest('''
1501 ... >>> x = 1
1502 ... >>> x
1503 ... 2
1504 ... ''', {}, 'foo', 'foo.py', 0)
1505
1506 >>> try:
1507 ... runner.run(test)
1508 ... except DocTestFailure, failure:
1509 ... pass
1510
1511 DocTestFailure objects provide access to the test:
1512
1513 >>> failure.test is test
1514 True
1515
1516 As well as to the example:
1517
1518 >>> failure.example.want
1519 '2\n'
1520
1521 and the actual output:
1522
1523 >>> failure.got
1524 '1\n'
1525
1526 If a failure or error occurs, the globals are left intact:
1527
1528 >>> del test.globs['__builtins__']
1529 >>> test.globs
1530 {'x': 1}
1531
1532 >>> test = DocTest('''
1533 ... >>> x = 2
1534 ... >>> raise KeyError
1535 ... ''', {}, 'foo', 'foo.py', 0)
1536
1537 >>> runner.run(test)
1538 Traceback (most recent call last):
1539 ...
1540 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)>
Tim Petersd1b78272004-08-07 06:03:09 +00001541
Tim Peters19397e52004-08-06 22:02:59 +00001542 >>> del test.globs['__builtins__']
1543 >>> test.globs
1544 {'x': 2}
1545
1546 But the globals are cleared if there is no error:
1547
1548 >>> test = DocTest('''
1549 ... >>> x = 2
1550 ... ''', {}, 'foo', 'foo.py', 0)
1551
1552 >>> runner.run(test)
1553 (0, 1)
1554
1555 >>> test.globs
1556 {}
1557
1558 """
1559
1560 def run(self, test, compileflags=None, out=None, clear_globs=True):
1561 r = DocTestRunner.run(self, test, compileflags, out, False)
1562 if clear_globs:
1563 test.globs.clear()
1564 return r
1565
1566 def report_unexpected_exception(self, out, test, example, exc_info):
1567 raise UnexpectedException(test, example, exc_info)
1568
1569 def report_failure(self, out, test, example, got):
1570 raise DocTestFailure(test, example, got)
1571
Tim Peters8485b562004-08-04 18:46:34 +00001572######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001573## 6. Test Functions
Tim Peters8485b562004-08-04 18:46:34 +00001574######################################################################
1575# These should be backwards compatible.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001576
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001577def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None,
Tim Peters19397e52004-08-06 22:02:59 +00001578 report=True, optionflags=0, extraglobs=None,
1579 raise_on_error=False):
Tim Peters6ebe61f2003-06-27 20:48:05 +00001580 """m=None, name=None, globs=None, verbose=None, isprivate=None,
Tim Peters8485b562004-08-04 18:46:34 +00001581 report=True, optionflags=0, extraglobs=None
Tim Peters8a7d2d52001-01-16 07:10:57 +00001582
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001583 Test examples in docstrings in functions and classes reachable
1584 from module m (or the current module if m is not supplied), starting
Raymond Hettinger71adf7e2003-07-16 19:25:22 +00001585 with m.__doc__. Unless isprivate is specified, private names
1586 are not skipped.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001587
1588 Also test examples reachable from dict m.__test__ if it exists and is
1589 not None. m.__dict__ maps names to functions, classes and strings;
1590 function and class docstrings are tested even if the name is private;
1591 strings are tested directly, as if they were docstrings.
1592
1593 Return (#failures, #tests).
1594
1595 See doctest.__doc__ for an overview.
1596
1597 Optional keyword arg "name" gives the name of the module; by default
1598 use m.__name__.
1599
1600 Optional keyword arg "globs" gives a dict to be used as the globals
1601 when executing examples; by default, use m.__dict__. A copy of this
1602 dict is actually used for each docstring, so that each docstring's
1603 examples start with a clean slate.
1604
Tim Peters8485b562004-08-04 18:46:34 +00001605 Optional keyword arg "extraglobs" gives a dictionary that should be
1606 merged into the globals that are used to execute examples. By
1607 default, no extra globals are used. This is new in 2.4.
1608
Tim Peters8a7d2d52001-01-16 07:10:57 +00001609 Optional keyword arg "verbose" prints lots of stuff if true, prints
1610 only failures if false; by default, it's true iff "-v" is in sys.argv.
1611
Tim Peters8a7d2d52001-01-16 07:10:57 +00001612 Optional keyword arg "report" prints a summary at the end when true,
1613 else prints nothing at the end. In verbose mode, the summary is
1614 detailed, else very brief (in fact, empty if all tests passed).
1615
Tim Peters6ebe61f2003-06-27 20:48:05 +00001616 Optional keyword arg "optionflags" or's together module constants,
1617 and defaults to 0. This is new in 2.3. Possible values:
1618
1619 DONT_ACCEPT_TRUE_FOR_1
1620 By default, if an expected output block contains just "1",
1621 an actual output block containing just "True" is considered
1622 to be a match, and similarly for "0" versus "False". When
1623 DONT_ACCEPT_TRUE_FOR_1 is specified, neither substitution
1624 is allowed.
1625
Tim Peters8485b562004-08-04 18:46:34 +00001626 DONT_ACCEPT_BLANKLINE
1627 By default, if an expected output block contains a line
1628 containing only the string "<BLANKLINE>", then that line
1629 will match a blank line in the actual output. When
1630 DONT_ACCEPT_BLANKLINE is specified, this substitution is
1631 not allowed.
1632
1633 NORMALIZE_WHITESPACE
1634 When NORMALIZE_WHITESPACE is specified, all sequences of
1635 whitespace are treated as equal. I.e., any sequence of
1636 whitespace within the expected output will match any
1637 sequence of whitespace within the actual output.
1638
1639 ELLIPSIS
1640 When ELLIPSIS is specified, then an ellipsis marker
1641 ("...") in the expected output can match any substring in
1642 the actual output.
1643
1644 UNIFIED_DIFF
1645 When UNIFIED_DIFF is specified, failures that involve
1646 multi-line expected and actual outputs will be displayed
1647 using a unified diff.
1648
1649 CONTEXT_DIFF
1650 When CONTEXT_DIFF is specified, failures that involve
1651 multi-line expected and actual outputs will be displayed
1652 using a context diff.
Tim Peters19397e52004-08-06 22:02:59 +00001653
1654 Optional keyword arg "raise_on_error" raises an exception on the
1655 first unexpected exception or failure. This allows failures to be
1656 post-mortem debugged.
1657
Tim Petersf727c6c2004-08-08 01:48:59 +00001658 Deprecated in Python 2.4:
1659 Optional keyword arg "isprivate" specifies a function used to
1660 determine whether a name is private. The default function is
1661 treat all functions as public. Optionally, "isprivate" can be
1662 set to doctest.is_private to skip over functions marked as private
1663 using the underscore naming convention; see its docs for details.
Tim Peters8485b562004-08-04 18:46:34 +00001664 """
1665
1666 """ [XX] This is no longer true:
Tim Peters8a7d2d52001-01-16 07:10:57 +00001667 Advanced tomfoolery: testmod runs methods of a local instance of
1668 class doctest.Tester, then merges the results into (or creates)
1669 global Tester instance doctest.master. Methods of doctest.master
1670 can be called directly too, if you want to do something unusual.
1671 Passing report=0 to testmod is especially useful then, to delay
1672 displaying a summary. Invoke doctest.master.summarize(verbose)
1673 when you're done fiddling.
1674 """
Tim Petersf727c6c2004-08-08 01:48:59 +00001675 if isprivate is not None:
1676 warnings.warn("the isprivate argument is deprecated; "
1677 "examine DocTestFinder.find() lists instead",
1678 DeprecationWarning)
1679
Tim Peters8485b562004-08-04 18:46:34 +00001680 # If no module was given, then use __main__.
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001681 if m is None:
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001682 # DWA - m will still be None if this wasn't invoked from the command
1683 # line, in which case the following TypeError is about as good an error
1684 # as we should expect
1685 m = sys.modules.get('__main__')
1686
Tim Peters8485b562004-08-04 18:46:34 +00001687 # Check that we were actually given a module.
1688 if not inspect.ismodule(m):
Walter Dörwald70a6b492004-02-12 17:35:32 +00001689 raise TypeError("testmod: module required; %r" % (m,))
Tim Peters8485b562004-08-04 18:46:34 +00001690
1691 # If no name was given, then use the module's name.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001692 if name is None:
1693 name = m.__name__
Tim Peters8485b562004-08-04 18:46:34 +00001694
1695 # Find, parse, and run all tests in the given module.
Tim Petersf727c6c2004-08-08 01:48:59 +00001696 finder = DocTestFinder(_namefilter=isprivate)
Tim Peters19397e52004-08-06 22:02:59 +00001697
1698 if raise_on_error:
1699 runner = DebugRunner(verbose=verbose, optionflags=optionflags)
1700 else:
1701 runner = DocTestRunner(verbose=verbose, optionflags=optionflags)
1702
Tim Peters8485b562004-08-04 18:46:34 +00001703 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
1704 runner.run(test)
1705
Tim Peters8a7d2d52001-01-16 07:10:57 +00001706 if report:
Tim Peters8485b562004-08-04 18:46:34 +00001707 runner.summarize()
Tim Peters8a7d2d52001-01-16 07:10:57 +00001708
Tim Peters8485b562004-08-04 18:46:34 +00001709 return runner.failures, runner.tries
Tim Petersdb3756d2003-06-29 05:30:48 +00001710
Tim Peters8485b562004-08-04 18:46:34 +00001711def run_docstring_examples(f, globs, verbose=False, name="NoName",
1712 compileflags=None, optionflags=0):
1713 """
1714 Test examples in the given object's docstring (`f`), using `globs`
1715 as globals. Optional argument `name` is used in failure messages.
1716 If the optional argument `verbose` is true, then generate output
1717 even if there are no failures.
Tim Petersdb3756d2003-06-29 05:30:48 +00001718
Tim Peters8485b562004-08-04 18:46:34 +00001719 `compileflags` gives the set of flags that should be used by the
1720 Python compiler when running the examples. If not specified, then
1721 it will default to the set of future-import flags that apply to
1722 `globs`.
Tim Petersdb3756d2003-06-29 05:30:48 +00001723
Tim Peters8485b562004-08-04 18:46:34 +00001724 Optional keyword arg `optionflags` specifies options for the
1725 testing and output. See the documentation for `testmod` for more
1726 information.
1727 """
1728 # Find, parse, and run all tests in the given module.
1729 finder = DocTestFinder(verbose=verbose, recurse=False)
1730 runner = DocTestRunner(verbose=verbose, optionflags=optionflags)
1731 for test in finder.find(f, name, globs=globs):
1732 runner.run(test, compileflags=compileflags)
Tim Petersdb3756d2003-06-29 05:30:48 +00001733
Tim Peters8485b562004-08-04 18:46:34 +00001734######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001735## 7. Tester
Tim Peters8485b562004-08-04 18:46:34 +00001736######################################################################
1737# This is provided only for backwards compatibility. It's not
1738# actually used in any way.
Tim Petersdb3756d2003-06-29 05:30:48 +00001739
Tim Peters8485b562004-08-04 18:46:34 +00001740class Tester:
1741 def __init__(self, mod=None, globs=None, verbose=None,
1742 isprivate=None, optionflags=0):
Tim Peters3ddd60a2004-08-08 02:43:33 +00001743
1744 warnings.warn("class Tester is deprecated; "
1745 "use class doctest.DocTestRunner instead",
1746 DeprecationWarning, stacklevel=2)
Tim Peters8485b562004-08-04 18:46:34 +00001747 if mod is None and globs is None:
1748 raise TypeError("Tester.__init__: must specify mod or globs")
1749 if mod is not None and not _ismodule(mod):
1750 raise TypeError("Tester.__init__: mod must be a module; %r" %
1751 (mod,))
1752 if globs is None:
1753 globs = mod.__dict__
1754 self.globs = globs
Tim Petersdb3756d2003-06-29 05:30:48 +00001755
Tim Peters8485b562004-08-04 18:46:34 +00001756 self.verbose = verbose
1757 self.isprivate = isprivate
1758 self.optionflags = optionflags
Tim Petersf727c6c2004-08-08 01:48:59 +00001759 self.testfinder = DocTestFinder(_namefilter=isprivate)
Tim Peters8485b562004-08-04 18:46:34 +00001760 self.testrunner = DocTestRunner(verbose=verbose,
1761 optionflags=optionflags)
Tim Petersdb3756d2003-06-29 05:30:48 +00001762
Tim Peters8485b562004-08-04 18:46:34 +00001763 def runstring(self, s, name):
1764 test = DocTest(s, self.globs, name, None, None)
1765 if self.verbose:
1766 print "Running string", name
1767 (f,t) = self.testrunner.run(test)
1768 if self.verbose:
1769 print f, "of", t, "examples failed in string", name
1770 return (f,t)
Tim Petersdb3756d2003-06-29 05:30:48 +00001771
Tim Petersf3f57472004-08-08 06:11:48 +00001772 def rundoc(self, object, name=None, module=None):
Tim Peters8485b562004-08-04 18:46:34 +00001773 f = t = 0
1774 tests = self.testfinder.find(object, name, module=module,
Tim Petersf3f57472004-08-08 06:11:48 +00001775 globs=self.globs)
Tim Peters8485b562004-08-04 18:46:34 +00001776 for test in tests:
1777 (f2, t2) = self.testrunner.run(test)
1778 (f,t) = (f+f2, t+t2)
1779 return (f,t)
Tim Petersdb3756d2003-06-29 05:30:48 +00001780
Tim Peters8485b562004-08-04 18:46:34 +00001781 def rundict(self, d, name, module=None):
1782 import new
1783 m = new.module(name)
1784 m.__dict__.update(d)
Tim Petersf3f57472004-08-08 06:11:48 +00001785 if module is None:
1786 module = False
1787 return self.rundoc(m, name, module)
Tim Petersdb3756d2003-06-29 05:30:48 +00001788
Tim Peters8485b562004-08-04 18:46:34 +00001789 def run__test__(self, d, name):
1790 import new
1791 m = new.module(name)
1792 m.__test__ = d
1793 return self.rundoc(m, name, module)
Tim Petersdb3756d2003-06-29 05:30:48 +00001794
Tim Peters8485b562004-08-04 18:46:34 +00001795 def summarize(self, verbose=None):
1796 return self.testrunner.summarize(verbose)
Tim Petersdb3756d2003-06-29 05:30:48 +00001797
Tim Peters8485b562004-08-04 18:46:34 +00001798 def merge(self, other):
1799 d = self.testrunner._name2ft
1800 for name, (f, t) in other.testrunner._name2ft.items():
1801 if name in d:
1802 print "*** Tester.merge: '" + name + "' in both" \
1803 " testers; summing outcomes."
1804 f2, t2 = d[name]
1805 f = f + f2
1806 t = t + t2
1807 d[name] = f, t
Tim Petersdb3756d2003-06-29 05:30:48 +00001808
Tim Peters8485b562004-08-04 18:46:34 +00001809######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001810## 8. Unittest Support
Tim Peters8485b562004-08-04 18:46:34 +00001811######################################################################
Tim Petersdb3756d2003-06-29 05:30:48 +00001812
Tim Peters19397e52004-08-06 22:02:59 +00001813class DocTestCase(unittest.TestCase):
Tim Petersdb3756d2003-06-29 05:30:48 +00001814
Edward Loper34fcb142004-08-09 02:45:41 +00001815 def __init__(self, test, optionflags=0, setUp=None, tearDown=None,
1816 checker=None):
Jim Fultona643b652004-07-14 19:06:50 +00001817 unittest.TestCase.__init__(self)
Tim Peters19397e52004-08-06 22:02:59 +00001818 self._dt_optionflags = optionflags
Edward Loper34fcb142004-08-09 02:45:41 +00001819 self._dt_checker = checker
Tim Peters19397e52004-08-06 22:02:59 +00001820 self._dt_test = test
1821 self._dt_setUp = setUp
1822 self._dt_tearDown = tearDown
Tim Petersdb3756d2003-06-29 05:30:48 +00001823
Jim Fultona643b652004-07-14 19:06:50 +00001824 def setUp(self):
Tim Peters19397e52004-08-06 22:02:59 +00001825 if self._dt_setUp is not None:
1826 self._dt_setUp()
Jim Fultona643b652004-07-14 19:06:50 +00001827
1828 def tearDown(self):
Tim Peters19397e52004-08-06 22:02:59 +00001829 if self._dt_tearDown is not None:
1830 self._dt_tearDown()
Jim Fultona643b652004-07-14 19:06:50 +00001831
1832 def runTest(self):
Tim Peters19397e52004-08-06 22:02:59 +00001833 test = self._dt_test
Jim Fultona643b652004-07-14 19:06:50 +00001834 old = sys.stdout
1835 new = StringIO()
Edward Loper34fcb142004-08-09 02:45:41 +00001836 runner = DocTestRunner(optionflags=self._dt_optionflags,
1837 checker=self._dt_checker, verbose=False)
Tim Peters19397e52004-08-06 22:02:59 +00001838
Jim Fultona643b652004-07-14 19:06:50 +00001839 try:
Tim Peters19397e52004-08-06 22:02:59 +00001840 runner.DIVIDER = "-"*70
1841 failures, tries = runner.run(test, out=new.write)
Jim Fultona643b652004-07-14 19:06:50 +00001842 finally:
1843 sys.stdout = old
1844
1845 if failures:
Tim Peters19397e52004-08-06 22:02:59 +00001846 raise self.failureException(self.format_failure(new.getvalue()))
Tim Peters8485b562004-08-04 18:46:34 +00001847
Tim Peters19397e52004-08-06 22:02:59 +00001848 def format_failure(self, err):
1849 test = self._dt_test
1850 if test.lineno is None:
1851 lineno = 'unknown line number'
1852 else:
1853 lineno = 'line %s' % test.lineno
1854 lname = '.'.join(test.name.split('.')[-1:])
1855 return ('Failed doctest test for %s\n'
1856 ' File "%s", line %s, in %s\n\n%s'
1857 % (test.name, test.filename, lineno, lname, err)
1858 )
1859
1860 def debug(self):
1861 r"""Run the test case without results and without catching exceptions
1862
1863 The unit test framework includes a debug method on test cases
1864 and test suites to support post-mortem debugging. The test code
1865 is run in such a way that errors are not caught. This way a
1866 caller can catch the errors and initiate post-mortem debugging.
1867
1868 The DocTestCase provides a debug method that raises
1869 UnexpectedException errors if there is an unexepcted
1870 exception:
1871
1872 >>> test = DocTest('>>> raise KeyError\n42',
1873 ... {}, 'foo', 'foo.py', 0)
1874 >>> case = DocTestCase(test)
1875 >>> try:
1876 ... case.debug()
1877 ... except UnexpectedException, failure:
1878 ... pass
1879
1880 The UnexpectedException contains the test, the example, and
1881 the original exception:
1882
1883 >>> failure.test is test
1884 True
1885
1886 >>> failure.example.want
1887 '42\n'
1888
1889 >>> exc_info = failure.exc_info
1890 >>> raise exc_info[0], exc_info[1], exc_info[2]
1891 Traceback (most recent call last):
1892 ...
1893 KeyError
1894
1895 If the output doesn't match, then a DocTestFailure is raised:
1896
1897 >>> test = DocTest('''
1898 ... >>> x = 1
1899 ... >>> x
1900 ... 2
1901 ... ''', {}, 'foo', 'foo.py', 0)
1902 >>> case = DocTestCase(test)
1903
1904 >>> try:
1905 ... case.debug()
1906 ... except DocTestFailure, failure:
1907 ... pass
1908
1909 DocTestFailure objects provide access to the test:
1910
1911 >>> failure.test is test
1912 True
1913
1914 As well as to the example:
1915
1916 >>> failure.example.want
1917 '2\n'
1918
1919 and the actual output:
1920
1921 >>> failure.got
1922 '1\n'
1923
1924 """
1925
Edward Loper34fcb142004-08-09 02:45:41 +00001926 runner = DebugRunner(optionflags=self._dt_optionflags,
1927 checker=self._dt_checker, verbose=False)
Tim Peters19397e52004-08-06 22:02:59 +00001928 runner.run(self._dt_test, out=nooutput)
Jim Fultona643b652004-07-14 19:06:50 +00001929
1930 def id(self):
Tim Peters19397e52004-08-06 22:02:59 +00001931 return self._dt_test.name
Jim Fultona643b652004-07-14 19:06:50 +00001932
1933 def __repr__(self):
Tim Peters19397e52004-08-06 22:02:59 +00001934 name = self._dt_test.name.split('.')
Jim Fultona643b652004-07-14 19:06:50 +00001935 return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
1936
1937 __str__ = __repr__
1938
1939 def shortDescription(self):
Tim Peters19397e52004-08-06 22:02:59 +00001940 return "Doctest: " + self._dt_test.name
Jim Fultona643b652004-07-14 19:06:50 +00001941
Tim Peters19397e52004-08-06 22:02:59 +00001942def nooutput(*args):
1943 pass
Jim Fultona643b652004-07-14 19:06:50 +00001944
Tim Peters19397e52004-08-06 22:02:59 +00001945def DocTestSuite(module=None, globs=None, extraglobs=None,
1946 optionflags=0, test_finder=None,
Edward Loper34fcb142004-08-09 02:45:41 +00001947 setUp=lambda: None, tearDown=lambda: None,
1948 checker=None):
Tim Peters8485b562004-08-04 18:46:34 +00001949 """
Tim Peters19397e52004-08-06 22:02:59 +00001950 Convert doctest tests for a mudule to a unittest test suite.
Jim Fultona643b652004-07-14 19:06:50 +00001951
Tim Peters19397e52004-08-06 22:02:59 +00001952 This converts each documentation string in a module that
1953 contains doctest tests to a unittest test case. If any of the
1954 tests in a doc string fail, then the test case fails. An exception
1955 is raised showing the name of the file containing the test and a
Jim Fultona643b652004-07-14 19:06:50 +00001956 (sometimes approximate) line number.
1957
Tim Peters19397e52004-08-06 22:02:59 +00001958 The `module` argument provides the module to be tested. The argument
Jim Fultona643b652004-07-14 19:06:50 +00001959 can be either a module or a module name.
1960
1961 If no argument is given, the calling module is used.
Jim Fultona643b652004-07-14 19:06:50 +00001962 """
Jim Fultona643b652004-07-14 19:06:50 +00001963
Tim Peters8485b562004-08-04 18:46:34 +00001964 if test_finder is None:
1965 test_finder = DocTestFinder()
Tim Peters8485b562004-08-04 18:46:34 +00001966
Tim Peters19397e52004-08-06 22:02:59 +00001967 module = _normalize_module(module)
1968 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs)
1969 if globs is None:
1970 globs = module.__dict__
1971 if not tests: # [XX] why do we want to do this?
1972 raise ValueError(module, "has no tests")
Tim Petersdb3756d2003-06-29 05:30:48 +00001973
1974 tests.sort()
1975 suite = unittest.TestSuite()
Tim Peters8485b562004-08-04 18:46:34 +00001976 for test in tests:
Tim Peters19397e52004-08-06 22:02:59 +00001977 if len(test.examples) == 0:
1978 continue
Tim Peters8485b562004-08-04 18:46:34 +00001979 if not test.filename:
Tim Petersdb3756d2003-06-29 05:30:48 +00001980 filename = module.__file__
1981 if filename.endswith(".pyc"):
1982 filename = filename[:-1]
1983 elif filename.endswith(".pyo"):
1984 filename = filename[:-1]
Tim Peters8485b562004-08-04 18:46:34 +00001985 test.filename = filename
Edward Loper34fcb142004-08-09 02:45:41 +00001986 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown,
1987 checker))
Tim Peters19397e52004-08-06 22:02:59 +00001988
1989 return suite
1990
1991class DocFileCase(DocTestCase):
1992
1993 def id(self):
1994 return '_'.join(self._dt_test.name.split('.'))
1995
1996 def __repr__(self):
1997 return self._dt_test.filename
1998 __str__ = __repr__
1999
2000 def format_failure(self, err):
2001 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s'
2002 % (self._dt_test.name, self._dt_test.filename, err)
2003 )
2004
2005def DocFileTest(path, package=None, globs=None,
2006 setUp=None, tearDown=None,
2007 optionflags=0):
2008 package = _normalize_module(package)
2009 name = path.split('/')[-1]
2010 dir = os.path.split(package.__file__)[0]
2011 path = os.path.join(dir, *(path.split('/')))
2012 doc = open(path).read()
2013
2014 if globs is None:
2015 globs = {}
2016
2017 test = DocTest(doc, globs, name, path, 0)
2018
2019 return DocFileCase(test, optionflags, setUp, tearDown)
2020
2021def DocFileSuite(*paths, **kw):
2022 """Creates a suite of doctest files.
2023
2024 One or more text file paths are given as strings. These should
2025 use "/" characters to separate path segments. Paths are relative
2026 to the directory of the calling module, or relative to the package
2027 passed as a keyword argument.
2028
2029 A number of options may be provided as keyword arguments:
2030
2031 package
2032 The name of a Python package. Text-file paths will be
2033 interpreted relative to the directory containing this package.
2034 The package may be supplied as a package object or as a dotted
2035 package name.
2036
2037 setUp
2038 The name of a set-up function. This is called before running the
2039 tests in each file.
2040
2041 tearDown
2042 The name of a tear-down function. This is called after running the
2043 tests in each file.
2044
2045 globs
2046 A dictionary containing initial global variables for the tests.
2047 """
2048 suite = unittest.TestSuite()
2049
2050 # We do this here so that _normalize_module is called at the right
2051 # level. If it were called in DocFileTest, then this function
2052 # would be the caller and we might guess the package incorrectly.
2053 kw['package'] = _normalize_module(kw.get('package'))
2054
2055 for path in paths:
2056 suite.addTest(DocFileTest(path, **kw))
Jim Fultona643b652004-07-14 19:06:50 +00002057
Tim Petersdb3756d2003-06-29 05:30:48 +00002058 return suite
2059
Tim Peters8485b562004-08-04 18:46:34 +00002060######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00002061## 9. Debugging Support
Tim Peters8485b562004-08-04 18:46:34 +00002062######################################################################
Jim Fultona643b652004-07-14 19:06:50 +00002063
Tim Peters19397e52004-08-06 22:02:59 +00002064def script_from_examples(s):
2065 r"""Extract script from text with examples.
2066
2067 Converts text with examples to a Python script. Example input is
2068 converted to regular code. Example output and all other words
2069 are converted to comments:
2070
2071 >>> text = '''
2072 ... Here are examples of simple math.
2073 ...
2074 ... Python has super accurate integer addition
2075 ...
2076 ... >>> 2 + 2
2077 ... 5
2078 ...
2079 ... And very friendly error messages:
2080 ...
2081 ... >>> 1/0
2082 ... To Infinity
2083 ... And
2084 ... Beyond
2085 ...
2086 ... You can use logic if you want:
2087 ...
2088 ... >>> if 0:
2089 ... ... blah
2090 ... ... blah
2091 ... ...
2092 ...
2093 ... Ho hum
2094 ... '''
2095
2096 >>> print script_from_examples(text)
2097 # Here are examples of simple math.
2098 #
2099 # Python has super accurate integer addition
2100 #
2101 2 + 2
2102 # Expected:
2103 # 5
2104 #
2105 # And very friendly error messages:
2106 #
2107 1/0
2108 # Expected:
2109 # To Infinity
2110 # And
2111 # Beyond
2112 #
2113 # You can use logic if you want:
2114 #
2115 if 0:
2116 blah
2117 blah
2118 <BLANKLINE>
2119 #
2120 # Ho hum
2121 """
2122
2123 return Parser('<string>', s).get_program()
2124
Tim Peters8485b562004-08-04 18:46:34 +00002125def _want_comment(example):
2126 """
Tim Peters19397e52004-08-06 22:02:59 +00002127 Return a comment containing the expected output for the given example.
Tim Peters8485b562004-08-04 18:46:34 +00002128 """
Jim Fultona643b652004-07-14 19:06:50 +00002129 # Return the expected output, if any
Tim Peters8485b562004-08-04 18:46:34 +00002130 want = example.want
2131 if want:
Tim Peters19397e52004-08-06 22:02:59 +00002132 if want[-1] == '\n':
2133 want = want[:-1]
Tim Peters8485b562004-08-04 18:46:34 +00002134 want = "\n# ".join(want.split("\n"))
2135 want = "\n# Expected:\n# %s" % want
2136 return want
Tim Petersdb3756d2003-06-29 05:30:48 +00002137
2138def testsource(module, name):
Tim Peters19397e52004-08-06 22:02:59 +00002139 """Extract the test sources from a doctest docstring as a script.
Tim Petersdb3756d2003-06-29 05:30:48 +00002140
2141 Provide the module (or dotted name of the module) containing the
Jim Fultona643b652004-07-14 19:06:50 +00002142 test to be debugged and the name (within the module) of the object
2143 with the doc string with tests to be debugged.
Tim Petersdb3756d2003-06-29 05:30:48 +00002144 """
Tim Peters8485b562004-08-04 18:46:34 +00002145 module = _normalize_module(module)
2146 tests = DocTestFinder().find(module)
2147 test = [t for t in tests if t.name == name]
Tim Petersdb3756d2003-06-29 05:30:48 +00002148 if not test:
2149 raise ValueError(name, "not found in tests")
2150 test = test[0]
Tim Peters19397e52004-08-06 22:02:59 +00002151 testsrc = script_from_examples(test.docstring)
Jim Fultona643b652004-07-14 19:06:50 +00002152 return testsrc
Tim Petersdb3756d2003-06-29 05:30:48 +00002153
Jim Fultona643b652004-07-14 19:06:50 +00002154def debug_src(src, pm=False, globs=None):
Tim Peters19397e52004-08-06 22:02:59 +00002155 """Debug a single doctest docstring, in argument `src`'"""
2156 testsrc = script_from_examples(src)
Tim Peters8485b562004-08-04 18:46:34 +00002157 debug_script(testsrc, pm, globs)
Tim Petersdb3756d2003-06-29 05:30:48 +00002158
Jim Fultona643b652004-07-14 19:06:50 +00002159def debug_script(src, pm=False, globs=None):
Tim Peters19397e52004-08-06 22:02:59 +00002160 "Debug a test script. `src` is the script, as a string."
Tim Petersdb3756d2003-06-29 05:30:48 +00002161 import pdb
Tim Petersdb3756d2003-06-29 05:30:48 +00002162
Tim Petersdb3756d2003-06-29 05:30:48 +00002163 srcfilename = tempfile.mktemp("doctestdebug.py")
Tim Peters8485b562004-08-04 18:46:34 +00002164 f = open(srcfilename, 'w')
2165 f.write(src)
2166 f.close()
2167
Jim Fultona643b652004-07-14 19:06:50 +00002168 if globs:
2169 globs = globs.copy()
2170 else:
2171 globs = {}
Tim Petersdb3756d2003-06-29 05:30:48 +00002172
Tim Peters8485b562004-08-04 18:46:34 +00002173 if pm:
2174 try:
2175 execfile(srcfilename, globs, globs)
2176 except:
2177 print sys.exc_info()[1]
2178 pdb.post_mortem(sys.exc_info()[2])
2179 else:
2180 # Note that %r is vital here. '%s' instead can, e.g., cause
2181 # backslashes to get treated as metacharacters on Windows.
2182 pdb.run("execfile(%r)" % srcfilename, globs, globs)
Tim Petersdb3756d2003-06-29 05:30:48 +00002183
Jim Fultona643b652004-07-14 19:06:50 +00002184def debug(module, name, pm=False):
Tim Peters19397e52004-08-06 22:02:59 +00002185 """Debug a single doctest docstring.
Jim Fultona643b652004-07-14 19:06:50 +00002186
2187 Provide the module (or dotted name of the module) containing the
2188 test to be debugged and the name (within the module) of the object
Tim Peters19397e52004-08-06 22:02:59 +00002189 with the docstring with tests to be debugged.
Jim Fultona643b652004-07-14 19:06:50 +00002190 """
Tim Peters8485b562004-08-04 18:46:34 +00002191 module = _normalize_module(module)
Jim Fultona643b652004-07-14 19:06:50 +00002192 testsrc = testsource(module, name)
2193 debug_script(testsrc, pm, module.__dict__)
2194
Tim Peters8485b562004-08-04 18:46:34 +00002195######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00002196## 10. Example Usage
Tim Peters8485b562004-08-04 18:46:34 +00002197######################################################################
Tim Peters8a7d2d52001-01-16 07:10:57 +00002198class _TestClass:
2199 """
2200 A pointless class, for sanity-checking of docstring testing.
2201
2202 Methods:
2203 square()
2204 get()
2205
2206 >>> _TestClass(13).get() + _TestClass(-12).get()
2207 1
2208 >>> hex(_TestClass(13).square().get())
2209 '0xa9'
2210 """
2211
2212 def __init__(self, val):
2213 """val -> _TestClass object with associated value val.
2214
2215 >>> t = _TestClass(123)
2216 >>> print t.get()
2217 123
2218 """
2219
2220 self.val = val
2221
2222 def square(self):
2223 """square() -> square TestClass's associated value
2224
2225 >>> _TestClass(13).square().get()
2226 169
2227 """
2228
2229 self.val = self.val ** 2
2230 return self
2231
2232 def get(self):
2233 """get() -> return TestClass's associated value.
2234
2235 >>> x = _TestClass(-42)
2236 >>> print x.get()
2237 -42
2238 """
2239
2240 return self.val
2241
2242__test__ = {"_TestClass": _TestClass,
2243 "string": r"""
2244 Example of a string object, searched as-is.
2245 >>> x = 1; y = 2
2246 >>> x + y, x * y
2247 (3, 2)
Tim Peters6ebe61f2003-06-27 20:48:05 +00002248 """,
2249 "bool-int equivalence": r"""
2250 In 2.2, boolean expressions displayed
2251 0 or 1. By default, we still accept
2252 them. This can be disabled by passing
2253 DONT_ACCEPT_TRUE_FOR_1 to the new
2254 optionflags argument.
2255 >>> 4 == 4
2256 1
2257 >>> 4 == 4
2258 True
2259 >>> 4 > 4
2260 0
2261 >>> 4 > 4
2262 False
2263 """,
Tim Peters8485b562004-08-04 18:46:34 +00002264 "blank lines": r"""
2265 Blank lines can be marked with <BLANKLINE>:
2266 >>> print 'foo\n\nbar\n'
2267 foo
2268 <BLANKLINE>
2269 bar
2270 <BLANKLINE>
2271 """,
2272 }
2273# "ellipsis": r"""
2274# If the ellipsis flag is used, then '...' can be used to
2275# elide substrings in the desired output:
2276# >>> print range(1000)
2277# [0, 1, 2, ..., 999]
2278# """,
2279# "whitespace normalization": r"""
2280# If the whitespace normalization flag is used, then
2281# differences in whitespace are ignored.
2282# >>> print range(30)
2283# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
2284# 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
2285# 27, 28, 29]
2286# """,
2287# }
2288
2289def test1(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002290>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning,
2291... "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002292>>> from doctest import Tester
2293>>> t = Tester(globs={'x': 42}, verbose=0)
2294>>> t.runstring(r'''
2295... >>> x = x * 2
2296... >>> print x
2297... 42
2298... ''', 'XYZ')
2299**********************************************************************
2300Failure in example: print x
2301from line #2 of XYZ
2302Expected: 42
2303Got: 84
2304(1, 2)
2305>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2')
2306(0, 2)
2307>>> t.summarize()
2308**********************************************************************
23091 items had failures:
2310 1 of 2 in XYZ
2311***Test Failed*** 1 failures.
2312(1, 4)
2313>>> t.summarize(verbose=1)
23141 items passed all tests:
2315 2 tests in example2
2316**********************************************************************
23171 items had failures:
2318 1 of 2 in XYZ
23194 tests in 2 items.
23203 passed and 1 failed.
2321***Test Failed*** 1 failures.
2322(1, 4)
2323"""
2324
2325def test2(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002326 >>> warnings.filterwarnings("ignore", "class Tester",
2327 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002328 >>> t = Tester(globs={}, verbose=1)
2329 >>> test = r'''
2330 ... # just an example
2331 ... >>> x = 1 + 2
2332 ... >>> x
2333 ... 3
2334 ... '''
2335 >>> t.runstring(test, "Example")
2336 Running string Example
2337 Trying: x = 1 + 2
2338 Expecting: nothing
2339 ok
2340 Trying: x
2341 Expecting: 3
2342 ok
2343 0 of 2 examples failed in string Example
2344 (0, 2)
2345"""
2346def test3(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002347 >>> warnings.filterwarnings("ignore", "class Tester",
2348 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002349 >>> t = Tester(globs={}, verbose=0)
2350 >>> def _f():
2351 ... '''Trivial docstring example.
2352 ... >>> assert 2 == 2
2353 ... '''
2354 ... return 32
2355 ...
2356 >>> t.rundoc(_f) # expect 0 failures in 1 example
2357 (0, 1)
2358"""
2359def test4(): """
2360 >>> import new
2361 >>> m1 = new.module('_m1')
2362 >>> m2 = new.module('_m2')
2363 >>> test_data = \"""
2364 ... def _f():
2365 ... '''>>> assert 1 == 1
2366 ... '''
2367 ... def g():
2368 ... '''>>> assert 2 != 1
2369 ... '''
2370 ... class H:
2371 ... '''>>> assert 2 > 1
2372 ... '''
2373 ... def bar(self):
2374 ... '''>>> assert 1 < 2
2375 ... '''
2376 ... \"""
2377 >>> exec test_data in m1.__dict__
2378 >>> exec test_data in m2.__dict__
2379 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H})
2380
2381 Tests that objects outside m1 are excluded:
2382
Tim Peters3ddd60a2004-08-08 02:43:33 +00002383 >>> warnings.filterwarnings("ignore", "class Tester",
2384 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002385 >>> t = Tester(globs={}, verbose=0)
Tim Petersf727c6c2004-08-08 01:48:59 +00002386 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped
Tim Peters8485b562004-08-04 18:46:34 +00002387 (0, 4)
2388
Tim Petersf727c6c2004-08-08 01:48:59 +00002389 Once more, not excluding stuff outside m1:
Tim Peters8485b562004-08-04 18:46:34 +00002390
2391 >>> t = Tester(globs={}, verbose=0)
2392 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped.
2393 (0, 8)
2394
2395 The exclusion of objects from outside the designated module is
2396 meant to be invoked automagically by testmod.
2397
Tim Petersf727c6c2004-08-08 01:48:59 +00002398 >>> testmod(m1, verbose=False)
2399 (0, 4)
Tim Peters8485b562004-08-04 18:46:34 +00002400"""
Tim Peters8a7d2d52001-01-16 07:10:57 +00002401
2402def _test():
Tim Peters8485b562004-08-04 18:46:34 +00002403 #import doctest
2404 #doctest.testmod(doctest, verbose=False,
2405 # optionflags=ELLIPSIS | NORMALIZE_WHITESPACE |
2406 # UNIFIED_DIFF)
2407 #print '~'*70
2408 r = unittest.TextTestRunner()
2409 r.run(DocTestSuite())
Tim Peters8a7d2d52001-01-16 07:10:57 +00002410
2411if __name__ == "__main__":
2412 _test()