blob: 5dd0bae40be13ce81706420776cd4e6a44120a5a [file] [log] [blame]
Tim Peters4fd9e2f2001-08-18 00:05:50 +00001# Module doctest.
Tim Peters8485b562004-08-04 18:46:34 +00002# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org).
Tim Peters19397e52004-08-06 22:02:59 +00003# Major enhancements and refactoring by:
Tim Peters8485b562004-08-04 18:46:34 +00004# Jim Fulton
5# Edward Loper
Tim Peters8a7d2d52001-01-16 07:10:57 +00006
7# Provided as-is; use at your own risk; no warranty; no promises; enjoy!
8
Martin v. Löwis92816de2004-05-31 19:01:00 +00009r"""Module doctest -- a framework for running examples in docstrings.
Tim Peters8a7d2d52001-01-16 07:10:57 +000010
11NORMAL USAGE
12
Tim Peters80e53142004-08-09 04:34:45 +000013In simplest use, end each module M to be tested with:
Tim Peters8a7d2d52001-01-16 07:10:57 +000014
15def _test():
Tim Peters80e53142004-08-09 04:34:45 +000016 import doctest
17 return doctest.testmod()
Tim Peters8a7d2d52001-01-16 07:10:57 +000018
19if __name__ == "__main__":
20 _test()
21
22Then running the module as a script will cause the examples in the
23docstrings to get executed and verified:
24
25python M.py
26
27This won't display anything unless an example fails, in which case the
28failing example(s) and the cause(s) of the failure(s) are printed to stdout
29(why not stderr? because stderr is a lame hack <0.2 wink>), and the final
30line of output is "Test failed.".
31
32Run it with the -v switch instead:
33
34python M.py -v
35
36and a detailed report of all examples tried is printed to stdout, along
37with assorted summaries at the end.
38
Tim Peters80e53142004-08-09 04:34:45 +000039You can force verbose mode by passing "verbose=True" to testmod, or prohibit
40it by passing "verbose=False". In either of those cases, sys.argv is not
Tim Peters8a7d2d52001-01-16 07:10:57 +000041examined by testmod.
42
43In any case, testmod returns a 2-tuple of ints (f, t), where f is the
44number of docstring examples that failed and t is the total number of
45docstring examples attempted.
46
Tim Peters80e53142004-08-09 04:34:45 +000047There are a variety of other ways to run doctests, including integration
48with the unittest framework, and support for running non-Python text
49files containing doctests. There are also many ways to override parts
50of doctest's default behaviors. See the Library Reference Manual for
51details.
52
Tim Peters8a7d2d52001-01-16 07:10:57 +000053
54WHICH DOCSTRINGS ARE EXAMINED?
55
56+ M.__doc__.
57
58+ f.__doc__ for all functions f in M.__dict__.values(), except those
Raymond Hettinger71adf7e2003-07-16 19:25:22 +000059 defined in other modules.
Tim Peters8a7d2d52001-01-16 07:10:57 +000060
Raymond Hettinger71adf7e2003-07-16 19:25:22 +000061+ C.__doc__ for all classes C in M.__dict__.values(), except those
62 defined in other modules.
Tim Peters8a7d2d52001-01-16 07:10:57 +000063
64+ If M.__test__ exists and "is true", it must be a dict, and
65 each entry maps a (string) name to a function object, class object, or
66 string. Function and class object docstrings found from M.__test__
Tim Peters80e53142004-08-09 04:34:45 +000067 are searched, and strings are searched directly as if they were docstrings.
68 In output, a key K in M.__test__ appears with name
Tim Peters8a7d2d52001-01-16 07:10:57 +000069 <name of M>.__test__.K
70
71Any classes found are recursively searched similarly, to test docstrings in
Tim Peters80e53142004-08-09 04:34:45 +000072their contained methods and nested classes.
Tim Peters8a7d2d52001-01-16 07:10:57 +000073
Tim Peters8a7d2d52001-01-16 07:10:57 +000074
Tim Peters8a7d2d52001-01-16 07:10:57 +000075WHAT'S THE EXECUTION CONTEXT?
76
77By default, each time testmod finds a docstring to test, it uses a *copy*
78of M's globals (so that running tests on a module doesn't change the
79module's real globals, and so that one test in M can't leave behind crumbs
80that accidentally allow another test to work). This means examples can
81freely use any names defined at top-level in M. It also means that sloppy
82imports (see above) can cause examples in external docstrings to use
83globals inappropriate for them.
84
85You can force use of your own dict as the execution context by passing
86"globs=your_dict" to testmod instead. Presumably this would be a copy of
87M.__dict__ merged with the globals from other imported modules.
88
89
Tim Peters8a7d2d52001-01-16 07:10:57 +000090WHAT ABOUT EXCEPTIONS?
91
92No problem, as long as the only output generated by the example is the
93traceback itself. For example:
94
Tim Peters60e23f42001-02-14 00:43:21 +000095 >>> [1, 2, 3].remove(42)
Tim Petersea4f9312001-02-13 20:54:42 +000096 Traceback (most recent call last):
Tim Peters8a7d2d52001-01-16 07:10:57 +000097 File "<stdin>", line 1, in ?
Tim Peters60e23f42001-02-14 00:43:21 +000098 ValueError: list.remove(x): x not in list
Tim Peters8a7d2d52001-01-16 07:10:57 +000099 >>>
100
Tim Peters80e53142004-08-09 04:34:45 +0000101Note that only the exception type and value are compared.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000102
103
Tim Peters80e53142004-08-09 04:34:45 +0000104SO WHAT DOES A DOCTEST EXAMPLE LOOK LIKE ALREADY!?
Tim Peters8a7d2d52001-01-16 07:10:57 +0000105
106Oh ya. It's easy! In most cases a copy-and-paste of an interactive
107console session works fine -- just make sure the leading whitespace is
108rigidly consistent (you can mix tabs and spaces if you're too lazy to do it
109right, but doctest is not in the business of guessing what you think a tab
110means).
111
112 >>> # comments are ignored
113 >>> x = 12
114 >>> x
115 12
116 >>> if x == 13:
117 ... print "yes"
118 ... else:
119 ... print "no"
120 ... print "NO"
121 ... print "NO!!!"
122 ...
123 no
124 NO
125 NO!!!
126 >>>
127
128Any expected output must immediately follow the final ">>>" or "..." line
129containing the code, and the expected output (if any) extends to the next
130">>>" or all-whitespace line. That's it.
131
132Bummers:
133
Tim Peters8a7d2d52001-01-16 07:10:57 +0000134+ Output to stdout is captured, but not output to stderr (exception
135 tracebacks are captured via a different means).
136
Martin v. Löwis92816de2004-05-31 19:01:00 +0000137+ If you continue a line via backslashing in an interactive session,
138 or for any other reason use a backslash, you should use a raw
139 docstring, which will preserve your backslahses exactly as you type
140 them:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000141
Tim Peters4e0e1b62004-07-07 20:54:48 +0000142 >>> def f(x):
Martin v. Löwis92816de2004-05-31 19:01:00 +0000143 ... r'''Backslashes in a raw docstring: m\n'''
144 >>> print f.__doc__
145 Backslashes in a raw docstring: m\n
Tim Peters8a7d2d52001-01-16 07:10:57 +0000146
Martin v. Löwis92816de2004-05-31 19:01:00 +0000147 Otherwise, the backslash will be interpreted as part of the string.
148 E.g., the "\n" above would be interpreted as a newline character.
149 Alternatively, you can double each backslash in the doctest version
150 (and not use a raw string):
151
Tim Peters4e0e1b62004-07-07 20:54:48 +0000152 >>> def f(x):
Martin v. Löwis92816de2004-05-31 19:01:00 +0000153 ... '''Backslashes in a raw docstring: m\\n'''
154 >>> print f.__doc__
155 Backslashes in a raw docstring: m\n
Tim Peters4e0e1b62004-07-07 20:54:48 +0000156
Tim Peters8a7d2d52001-01-16 07:10:57 +0000157The starting column doesn't matter:
158
159>>> assert "Easy!"
160 >>> import math
161 >>> math.floor(1.9)
162 1.0
163
164and as many leading whitespace characters are stripped from the expected
165output as appeared in the initial ">>>" line that triggered it.
166
167If you execute this very file, the examples above will be found and
Tim Peters80e53142004-08-09 04:34:45 +0000168executed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000169"""
170
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000171__all__ = [
Tim Peters8485b562004-08-04 18:46:34 +0000172 'is_private',
173 'Example',
174 'DocTest',
175 'DocTestFinder',
176 'DocTestRunner',
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000177 'testmod',
178 'run_docstring_examples',
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000179 'Tester',
Tim Peters19397e52004-08-06 22:02:59 +0000180 'DocTestCase',
Tim Petersdb3756d2003-06-29 05:30:48 +0000181 'DocTestSuite',
182 'testsource',
183 'debug',
Tim Peters8485b562004-08-04 18:46:34 +0000184# 'master',
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000185]
Tim Peters8a7d2d52001-01-16 07:10:57 +0000186
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000187import __future__
Tim Peters8a7d2d52001-01-16 07:10:57 +0000188
Tim Peters19397e52004-08-06 22:02:59 +0000189import sys, traceback, inspect, linecache, os, re, types
Jim Fulton356fd192004-08-09 11:34:47 +0000190import unittest, difflib, pdb, tempfile
Tim Petersf727c6c2004-08-08 01:48:59 +0000191import warnings
Tim Peters8485b562004-08-04 18:46:34 +0000192from StringIO import StringIO
Tim Peters7402f792001-10-02 03:53:41 +0000193
Jim Fulton356fd192004-08-09 11:34:47 +0000194real_pdb_set_trace = pdb.set_trace
195
Tim Peters6ebe61f2003-06-27 20:48:05 +0000196# Option constants.
197DONT_ACCEPT_TRUE_FOR_1 = 1 << 0
Tim Peters8485b562004-08-04 18:46:34 +0000198DONT_ACCEPT_BLANKLINE = 1 << 1
199NORMALIZE_WHITESPACE = 1 << 2
200ELLIPSIS = 1 << 3
201UNIFIED_DIFF = 1 << 4
202CONTEXT_DIFF = 1 << 5
Tim Peters6ebe61f2003-06-27 20:48:05 +0000203
Tim Peters8485b562004-08-04 18:46:34 +0000204OPTIONFLAGS_BY_NAME = {
205 'DONT_ACCEPT_TRUE_FOR_1': DONT_ACCEPT_TRUE_FOR_1,
206 'DONT_ACCEPT_BLANKLINE': DONT_ACCEPT_BLANKLINE,
207 'NORMALIZE_WHITESPACE': NORMALIZE_WHITESPACE,
208 'ELLIPSIS': ELLIPSIS,
209 'UNIFIED_DIFF': UNIFIED_DIFF,
210 'CONTEXT_DIFF': CONTEXT_DIFF,
211 }
Tim Peters8a7d2d52001-01-16 07:10:57 +0000212
Tim Peters8485b562004-08-04 18:46:34 +0000213# Special string markers for use in `want` strings:
214BLANKLINE_MARKER = '<BLANKLINE>'
215ELLIPSIS_MARKER = '...'
Tim Peters8a7d2d52001-01-16 07:10:57 +0000216
Tim Peters19397e52004-08-06 22:02:59 +0000217
218# There are 4 basic classes:
219# - Example: a <source, want> pair, plus an intra-docstring line number.
220# - DocTest: a collection of examples, parsed from a docstring, plus
221# info about where the docstring came from (name, filename, lineno).
222# - DocTestFinder: extracts DocTests from a given object's docstring and
223# its contained objects' docstrings.
224# - DocTestRunner: runs DocTest cases, and accumulates statistics.
225#
226# So the basic picture is:
227#
228# list of:
229# +------+ +---------+ +-------+
230# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results|
231# +------+ +---------+ +-------+
232# | Example |
233# | ... |
234# | Example |
235# +---------+
236
Tim Peters8485b562004-08-04 18:46:34 +0000237######################################################################
238## Table of Contents
239######################################################################
Edward Loper7c748462004-08-09 02:06:06 +0000240# 1. Utility Functions
241# 2. Example & DocTest -- store test cases
242# 3. DocTest Parser -- extracts examples from strings
243# 4. DocTest Finder -- extracts test cases from objects
244# 5. DocTest Runner -- runs test cases
245# 6. Test Functions -- convenient wrappers for testing
246# 7. Tester Class -- for backwards compatibility
247# 8. Unittest Support
248# 9. Debugging Support
249# 10. Example Usage
Tim Peters8a7d2d52001-01-16 07:10:57 +0000250
Tim Peters8485b562004-08-04 18:46:34 +0000251######################################################################
252## 1. Utility Functions
253######################################################################
Tim Peters8a7d2d52001-01-16 07:10:57 +0000254
255def is_private(prefix, base):
256 """prefix, base -> true iff name prefix + "." + base is "private".
257
258 Prefix may be an empty string, and base does not contain a period.
259 Prefix is ignored (although functions you write conforming to this
260 protocol may make use of it).
261 Return true iff base begins with an (at least one) underscore, but
262 does not both begin and end with (at least) two underscores.
263
Tim Petersbafb1fe2004-08-08 01:52:57 +0000264 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning,
265 ... "doctest", 0)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000266 >>> is_private("a.b", "my_func")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000267 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000268 >>> is_private("____", "_my_func")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000269 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000270 >>> is_private("someclass", "__init__")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000271 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000272 >>> is_private("sometypo", "__init_")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000273 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000274 >>> is_private("x.y.z", "_")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000275 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000276 >>> is_private("_x.y.z", "__")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000277 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000278 >>> is_private("", "") # senseless but consistent
Guido van Rossum77f6a652002-04-03 22:41:51 +0000279 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000280 """
Tim Petersbafb1fe2004-08-08 01:52:57 +0000281 warnings.warn("is_private is deprecated; it wasn't useful; "
282 "examine DocTestFinder.find() lists instead",
Tim Peters3ddd60a2004-08-08 02:43:33 +0000283 DeprecationWarning, stacklevel=2)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000284 return base[:1] == "_" and not base[:2] == "__" == base[-2:]
285
Tim Peters8485b562004-08-04 18:46:34 +0000286def _extract_future_flags(globs):
287 """
288 Return the compiler-flags associated with the future features that
289 have been imported into the given namespace (globs).
290 """
291 flags = 0
292 for fname in __future__.all_feature_names:
293 feature = globs.get(fname, None)
294 if feature is getattr(__future__, fname):
295 flags |= feature.compiler_flag
296 return flags
Tim Peters7402f792001-10-02 03:53:41 +0000297
Tim Peters8485b562004-08-04 18:46:34 +0000298def _normalize_module(module, depth=2):
299 """
300 Return the module specified by `module`. In particular:
301 - If `module` is a module, then return module.
302 - If `module` is a string, then import and return the
303 module with that name.
304 - If `module` is None, then return the calling module.
305 The calling module is assumed to be the module of
306 the stack frame at the given depth in the call stack.
307 """
308 if inspect.ismodule(module):
309 return module
310 elif isinstance(module, (str, unicode)):
311 return __import__(module, globals(), locals(), ["*"])
312 elif module is None:
313 return sys.modules[sys._getframe(depth).f_globals['__name__']]
314 else:
315 raise TypeError("Expected a module, string, or None")
Tim Peters7402f792001-10-02 03:53:41 +0000316
Tim Peters8485b562004-08-04 18:46:34 +0000317def _tag_msg(tag, msg, indent_msg=True):
318 """
319 Return a string that displays a tag-and-message pair nicely,
320 keeping the tag and its message on the same line when that
321 makes sense. If `indent_msg` is true, then messages that are
322 put on separate lines will be indented.
323 """
324 # What string should we use to indent contents?
325 INDENT = ' '
Tim Peters8a7d2d52001-01-16 07:10:57 +0000326
Tim Peters8485b562004-08-04 18:46:34 +0000327 # If the message doesn't end in a newline, then add one.
328 if msg[-1:] != '\n':
329 msg += '\n'
330 # If the message is short enough, and contains no internal
331 # newlines, then display it on the same line as the tag.
332 # Otherwise, display the tag on its own line.
333 if (len(tag) + len(msg) < 75 and
334 msg.find('\n', 0, len(msg)-1) == -1):
335 return '%s: %s' % (tag, msg)
336 else:
337 if indent_msg:
338 msg = '\n'.join([INDENT+l for l in msg.split('\n')])
339 msg = msg[:-len(INDENT)]
340 return '%s:\n%s' % (tag, msg)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000341
Tim Peters8485b562004-08-04 18:46:34 +0000342# Override some StringIO methods.
343class _SpoofOut(StringIO):
344 def getvalue(self):
345 result = StringIO.getvalue(self)
346 # If anything at all was written, make sure there's a trailing
347 # newline. There's no way for the expected output to indicate
348 # that a trailing newline is missing.
349 if result and not result.endswith("\n"):
350 result += "\n"
351 # Prevent softspace from screwing up the next test case, in
352 # case they used print with a trailing comma in an example.
353 if hasattr(self, "softspace"):
354 del self.softspace
355 return result
Tim Peters8a7d2d52001-01-16 07:10:57 +0000356
Tim Peters8485b562004-08-04 18:46:34 +0000357 def truncate(self, size=None):
358 StringIO.truncate(self, size)
359 if hasattr(self, "softspace"):
360 del self.softspace
Tim Peters8a7d2d52001-01-16 07:10:57 +0000361
Tim Peters8485b562004-08-04 18:46:34 +0000362######################################################################
363## 2. Example & DocTest
364######################################################################
365## - An "example" is a <source, want> pair, where "source" is a
366## fragment of source code, and "want" is the expected output for
367## "source." The Example class also includes information about
368## where the example was extracted from.
369##
370## - A "doctest" is a collection of examples extracted from a string
371## (such as an object's docstring). The DocTest class also includes
372## information about where the string was extracted from.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000373
Tim Peters8485b562004-08-04 18:46:34 +0000374class Example:
375 """
376 A single doctest example, consisting of source code and expected
377 output. Example defines the following attributes:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000378
Tim Petersbb431472004-08-09 03:51:46 +0000379 - source: A single Python statement, always ending with a newline.
380 The constructor adds a newline if needed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000381
Tim Petersbb431472004-08-09 03:51:46 +0000382 - want: The expected output from running the source code (either
383 from stdout, or a traceback in case of exception). `want` ends
384 with a newline unless it's empty, in which case it's an empty
385 string. The constructor adds a newline if needed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000386
Tim Petersbb431472004-08-09 03:51:46 +0000387 - lineno: The line number within the DocTest string containing
Tim Peters8485b562004-08-04 18:46:34 +0000388 this Example where the Example begins. This line number is
389 zero-based, with respect to the beginning of the DocTest.
390 """
391 def __init__(self, source, want, lineno):
Tim Petersbb431472004-08-09 03:51:46 +0000392 # Normalize inputs.
393 if not source.endswith('\n'):
394 source += '\n'
395 if want and not want.endswith('\n'):
396 want += '\n'
Tim Peters8485b562004-08-04 18:46:34 +0000397 # Store properties.
398 self.source = source
399 self.want = want
400 self.lineno = lineno
Tim Peters8a7d2d52001-01-16 07:10:57 +0000401
Tim Peters8485b562004-08-04 18:46:34 +0000402class DocTest:
403 """
404 A collection of doctest examples that should be run in a single
405 namespace. Each DocTest defines the following attributes:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000406
Tim Peters8485b562004-08-04 18:46:34 +0000407 - examples: the list of examples.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000408
Tim Peters8485b562004-08-04 18:46:34 +0000409 - globs: The namespace (aka globals) that the examples should
410 be run in.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000411
Tim Peters8485b562004-08-04 18:46:34 +0000412 - name: A name identifying the DocTest (typically, the name of
413 the object whose docstring this DocTest was extracted from).
Tim Peters8a7d2d52001-01-16 07:10:57 +0000414
Tim Peters19397e52004-08-06 22:02:59 +0000415 - docstring: The docstring being tested
416
Tim Peters8485b562004-08-04 18:46:34 +0000417 - filename: The name of the file that this DocTest was extracted
418 from.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000419
Tim Peters8485b562004-08-04 18:46:34 +0000420 - lineno: The line number within filename where this DocTest
421 begins. This line number is zero-based, with respect to the
422 beginning of the file.
423 """
424 def __init__(self, docstring, globs, name, filename, lineno):
425 """
426 Create a new DocTest, by extracting examples from `docstring`.
427 The DocTest's globals are initialized with a copy of `globs`.
428 """
429 # Store a copy of the globals
430 self.globs = globs.copy()
431 # Store identifying information
432 self.name = name
433 self.filename = filename
434 self.lineno = lineno
435 # Parse the docstring.
Tim Peters19397e52004-08-06 22:02:59 +0000436 self.docstring = docstring
Edward Loper78b58f32004-08-09 02:56:02 +0000437 self.examples = Parser(name, docstring).get_examples()
Tim Peters8485b562004-08-04 18:46:34 +0000438
439 def __repr__(self):
440 if len(self.examples) == 0:
441 examples = 'no examples'
442 elif len(self.examples) == 1:
443 examples = '1 example'
444 else:
445 examples = '%d examples' % len(self.examples)
446 return ('<DocTest %s from %s:%s (%s)>' %
447 (self.name, self.filename, self.lineno, examples))
448
449
450 # This lets us sort tests by name:
451 def __cmp__(self, other):
452 if not isinstance(other, DocTest):
453 return -1
454 return cmp((self.name, self.filename, self.lineno, id(self)),
455 (other.name, other.filename, other.lineno, id(other)))
456
457######################################################################
Edward Loper7c748462004-08-09 02:06:06 +0000458## 2. Example Parser
459######################################################################
460
461class Parser:
462 """
463 Extract doctests from a string.
464 """
465 def __init__(self, name, string):
466 """
467 Prepare to extract doctests from string `string`.
468
469 `name` is an arbitrary (string) name associated with the string,
470 and is used only in error messages.
471 """
472 self.name = name
473 self.string = string.expandtabs()
474
475 _EXAMPLE_RE = re.compile(r'''
Tim Petersd40a92b2004-08-09 03:28:45 +0000476 # Source consists of a PS1 line followed by zero or more PS2 lines.
477 (?P<source>
478 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line
479 (?:\n [ ]* \.\.\. .*)*) # PS2 lines
480 \n?
481 # Want consists of any non-blank lines that do not start with PS1.
482 (?P<want> (?:(?![ ]*$) # Not a blank line
483 (?![ ]*>>>) # Not a line starting with PS1
484 .*$\n? # But any other line
485 )*)
486 ''', re.MULTILINE | re.VERBOSE)
487 _IS_BLANK_OR_COMMENT = re.compile('^[ ]*(#.*)?$').match
Edward Loper7c748462004-08-09 02:06:06 +0000488
489 def get_examples(self):
490 """
Edward Loper78b58f32004-08-09 02:56:02 +0000491 Extract all doctest examples, from the string, and return them
492 as a list of `Example` objects. Line numbers are 0-based,
493 because it's most common in doctests that nothing interesting
494 appears on the same line as opening triple-quote, and so the
495 first interesting line is called \"line 1\" then.
Edward Loper7c748462004-08-09 02:06:06 +0000496
497 >>> text = '''
498 ... >>> x, y = 2, 3 # no output expected
499 ... >>> if 1:
500 ... ... print x
501 ... ... print y
502 ... 2
503 ... 3
504 ...
505 ... Some text.
506 ... >>> x+y
507 ... 5
508 ... '''
509 >>> for x in Parser('<string>', text).get_examples():
Edward Loper78b58f32004-08-09 02:56:02 +0000510 ... print (x.source, x.want, x.lineno)
Tim Petersbb431472004-08-09 03:51:46 +0000511 ('x, y = 2, 3 # no output expected\\n', '', 1)
Edward Loper7c748462004-08-09 02:06:06 +0000512 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2)
Tim Petersbb431472004-08-09 03:51:46 +0000513 ('x+y\\n', '5\\n', 9)
Edward Loper7c748462004-08-09 02:06:06 +0000514 """
515 examples = []
516 charno, lineno = 0, 0
517 # Find all doctest examples in the string:
518 for m in self._EXAMPLE_RE.finditer(self.string):
519 # Update lineno (lines before this example)
520 lineno += self.string.count('\n', charno, m.start())
521
522 # Extract source/want from the regexp match.
523 (source, want) = self._parse_example(m, lineno)
Tim Petersd40a92b2004-08-09 03:28:45 +0000524 if self._IS_BLANK_OR_COMMENT(source):
Edward Loper7c748462004-08-09 02:06:06 +0000525 continue
Edward Loper78b58f32004-08-09 02:56:02 +0000526 examples.append( Example(source, want, lineno) )
Edward Loper7c748462004-08-09 02:06:06 +0000527
528 # Update lineno (lines inside this example)
529 lineno += self.string.count('\n', m.start(), m.end())
530 # Update charno.
531 charno = m.end()
532 return examples
533
534 def get_program(self):
535 """
536 Return an executable program from the string, as a string.
537
538 The format of this isn't rigidly defined. In general, doctest
539 examples become the executable statements in the result, and
540 their expected outputs become comments, preceded by an \"#Expected:\"
541 comment. Everything else (text, comments, everything not part of
542 a doctest test) is also placed in comments.
543
544 >>> text = '''
545 ... >>> x, y = 2, 3 # no output expected
546 ... >>> if 1:
547 ... ... print x
548 ... ... print y
549 ... 2
550 ... 3
551 ...
552 ... Some text.
553 ... >>> x+y
554 ... 5
555 ... '''
556 >>> print Parser('<string>', text).get_program()
557 x, y = 2, 3 # no output expected
558 if 1:
559 print x
560 print y
561 # Expected:
562 # 2
563 # 3
564 #
565 # Some text.
566 x+y
567 # Expected:
568 # 5
569 """
570 output = []
571 charnum, lineno = 0, 0
572 # Find all doctest examples in the string:
573 for m in self._EXAMPLE_RE.finditer(self.string):
574 # Add any text before this example, as a comment.
575 if m.start() > charnum:
576 lines = self.string[charnum:m.start()-1].split('\n')
577 output.extend([self._comment_line(l) for l in lines])
578 lineno += len(lines)
579
580 # Extract source/want from the regexp match.
581 (source, want) = self._parse_example(m, lineno, False)
582 # Display the source
583 output.append(source)
584 # Display the expected output, if any
585 if want:
586 output.append('# Expected:')
587 output.extend(['# '+l for l in want.split('\n')])
588
589 # Update the line number & char number.
590 lineno += self.string.count('\n', m.start(), m.end())
591 charnum = m.end()
592 # Add any remaining text, as comments.
593 output.extend([self._comment_line(l)
594 for l in self.string[charnum:].split('\n')])
595 # Trim junk on both ends.
596 while output and output[-1] == '#':
597 output.pop()
598 while output and output[0] == '#':
599 output.pop(0)
600 # Combine the output, and return it.
601 return '\n'.join(output)
602
603 def _parse_example(self, m, lineno, add_newlines=True):
604 # Get the example's indentation level.
605 indent = len(m.group('indent'))
606
607 # Divide source into lines; check that they're properly
608 # indented; and then strip their indentation & prompts.
609 source_lines = m.group('source').split('\n')
610 self._check_prompt_blank(source_lines, indent, lineno)
611 self._check_prefix(source_lines[1:], ' '*indent+'.', lineno)
612 source = '\n'.join([sl[indent+4:] for sl in source_lines])
613 if len(source_lines) > 1 and add_newlines:
614 source += '\n'
615
616 # Divide want into lines; check that it's properly
617 # indented; and then strip the indentation.
618 want_lines = m.group('want').rstrip().split('\n')
619 self._check_prefix(want_lines, ' '*indent,
620 lineno+len(source_lines))
621 want = '\n'.join([wl[indent:] for wl in want_lines])
622 if len(want) > 0 and add_newlines:
623 want += '\n'
624
625 return source, want
626
627 def _comment_line(self, line):
628 line = line.rstrip()
Tim Petersdd0e4752004-08-09 03:31:56 +0000629 if line:
630 return '# '+line
631 else:
632 return '#'
Edward Loper7c748462004-08-09 02:06:06 +0000633
634 def _check_prompt_blank(self, lines, indent, lineno):
635 for i, line in enumerate(lines):
636 if len(line) >= indent+4 and line[indent+3] != ' ':
637 raise ValueError('line %r of the docstring for %s '
638 'lacks blank after %s: %r' %
639 (lineno+i+1, self.name,
640 line[indent:indent+3], line))
641
642 def _check_prefix(self, lines, prefix, lineno):
643 for i, line in enumerate(lines):
644 if line and not line.startswith(prefix):
645 raise ValueError('line %r of the docstring for %s has '
646 'inconsistent leading whitespace: %r' %
647 (lineno+i+1, self.name, line))
648
649
650######################################################################
651## 4. DocTest Finder
Tim Peters8485b562004-08-04 18:46:34 +0000652######################################################################
653
654class DocTestFinder:
655 """
656 A class used to extract the DocTests that are relevant to a given
657 object, from its docstring and the docstrings of its contained
658 objects. Doctests can currently be extracted from the following
659 object types: modules, functions, classes, methods, staticmethods,
660 classmethods, and properties.
Tim Peters8485b562004-08-04 18:46:34 +0000661 """
662
Tim Peters19397e52004-08-06 22:02:59 +0000663 def __init__(self, verbose=False, doctest_factory=DocTest,
Tim Petersf727c6c2004-08-08 01:48:59 +0000664 recurse=True, _namefilter=None):
Tim Peters8485b562004-08-04 18:46:34 +0000665 """
666 Create a new doctest finder.
667
Tim Peters19397e52004-08-06 22:02:59 +0000668 The optional argument `doctest_factory` specifies a class or
669 function that should be used to create new DocTest objects (or
Tim Peters161c9632004-08-08 03:38:33 +0000670 objects that implement the same interface as DocTest). The
Tim Peters19397e52004-08-06 22:02:59 +0000671 signature for this factory function should match the signature
672 of the DocTest constructor.
673
Tim Peters8485b562004-08-04 18:46:34 +0000674 If the optional argument `recurse` is false, then `find` will
675 only examine the given object, and not any contained objects.
676 """
Tim Peters19397e52004-08-06 22:02:59 +0000677 self._doctest_factory = doctest_factory
Tim Peters8485b562004-08-04 18:46:34 +0000678 self._verbose = verbose
Tim Peters8485b562004-08-04 18:46:34 +0000679 self._recurse = recurse
Tim Petersf727c6c2004-08-08 01:48:59 +0000680 # _namefilter is undocumented, and exists only for temporary backward-
681 # compatibility support of testmod's deprecated isprivate mess.
682 self._namefilter = _namefilter
Tim Peters8485b562004-08-04 18:46:34 +0000683
684 def find(self, obj, name=None, module=None, globs=None,
Tim Petersf3f57472004-08-08 06:11:48 +0000685 extraglobs=None):
Tim Peters8485b562004-08-04 18:46:34 +0000686 """
687 Return a list of the DocTests that are defined by the given
688 object's docstring, or by any of its contained objects'
689 docstrings.
690
691 The optional parameter `module` is the module that contains
Tim Petersf3f57472004-08-08 06:11:48 +0000692 the given object. If the module is not specified or is None, then
693 the test finder will attempt to automatically determine the
Tim Peters8485b562004-08-04 18:46:34 +0000694 correct module. The object's module is used:
695
696 - As a default namespace, if `globs` is not specified.
697 - To prevent the DocTestFinder from extracting DocTests
Tim Petersf3f57472004-08-08 06:11:48 +0000698 from objects that are imported from other modules.
Tim Peters8485b562004-08-04 18:46:34 +0000699 - To find the name of the file containing the object.
700 - To help find the line number of the object within its
701 file.
702
Tim Petersf3f57472004-08-08 06:11:48 +0000703 Contained objects whose module does not match `module` are ignored.
704
705 If `module` is False, no attempt to find the module will be made.
706 This is obscure, of use mostly in tests: if `module` is False, or
707 is None but cannot be found automatically, then all objects are
708 considered to belong to the (non-existent) module, so all contained
709 objects will (recursively) be searched for doctests.
710
Tim Peters8485b562004-08-04 18:46:34 +0000711 The globals for each DocTest is formed by combining `globs`
712 and `extraglobs` (bindings in `extraglobs` override bindings
713 in `globs`). A new copy of the globals dictionary is created
714 for each DocTest. If `globs` is not specified, then it
715 defaults to the module's `__dict__`, if specified, or {}
716 otherwise. If `extraglobs` is not specified, then it defaults
717 to {}.
718
Tim Peters8485b562004-08-04 18:46:34 +0000719 """
720 # If name was not specified, then extract it from the object.
721 if name is None:
722 name = getattr(obj, '__name__', None)
723 if name is None:
724 raise ValueError("DocTestFinder.find: name must be given "
725 "when obj.__name__ doesn't exist: %r" %
726 (type(obj),))
727
728 # Find the module that contains the given object (if obj is
729 # a module, then module=obj.). Note: this may fail, in which
730 # case module will be None.
Tim Petersf3f57472004-08-08 06:11:48 +0000731 if module is False:
732 module = None
733 elif module is None:
Tim Peters8485b562004-08-04 18:46:34 +0000734 module = inspect.getmodule(obj)
735
736 # Read the module's source code. This is used by
737 # DocTestFinder._find_lineno to find the line number for a
738 # given object's docstring.
739 try:
740 file = inspect.getsourcefile(obj) or inspect.getfile(obj)
741 source_lines = linecache.getlines(file)
742 if not source_lines:
743 source_lines = None
744 except TypeError:
745 source_lines = None
746
747 # Initialize globals, and merge in extraglobs.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000748 if globs is None:
Tim Peters8485b562004-08-04 18:46:34 +0000749 if module is None:
750 globs = {}
751 else:
752 globs = module.__dict__.copy()
753 else:
754 globs = globs.copy()
755 if extraglobs is not None:
756 globs.update(extraglobs)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000757
Tim Peters8485b562004-08-04 18:46:34 +0000758 # Recursively expore `obj`, extracting DocTests.
759 tests = []
Tim Petersf3f57472004-08-08 06:11:48 +0000760 self._find(tests, obj, name, module, source_lines, globs, {})
Tim Peters8485b562004-08-04 18:46:34 +0000761 return tests
762
763 def _filter(self, obj, prefix, base):
764 """
765 Return true if the given object should not be examined.
766 """
Tim Petersf727c6c2004-08-08 01:48:59 +0000767 return (self._namefilter is not None and
768 self._namefilter(prefix, base))
Tim Peters8485b562004-08-04 18:46:34 +0000769
770 def _from_module(self, module, object):
771 """
772 Return true if the given object is defined in the given
773 module.
774 """
775 if module is None:
776 return True
777 elif inspect.isfunction(object):
778 return module.__dict__ is object.func_globals
779 elif inspect.isclass(object):
780 return module.__name__ == object.__module__
781 elif inspect.getmodule(object) is not None:
782 return module is inspect.getmodule(object)
783 elif hasattr(object, '__module__'):
784 return module.__name__ == object.__module__
785 elif isinstance(object, property):
786 return True # [XX] no way not be sure.
787 else:
788 raise ValueError("object must be a class or function")
789
Tim Petersf3f57472004-08-08 06:11:48 +0000790 def _find(self, tests, obj, name, module, source_lines, globs, seen):
Tim Peters8485b562004-08-04 18:46:34 +0000791 """
792 Find tests for the given object and any contained objects, and
793 add them to `tests`.
794 """
795 if self._verbose:
796 print 'Finding tests in %s' % name
797
798 # If we've already processed this object, then ignore it.
799 if id(obj) in seen:
800 return
801 seen[id(obj)] = 1
802
803 # Find a test for this object, and add it to the list of tests.
804 test = self._get_test(obj, name, module, globs, source_lines)
805 if test is not None:
806 tests.append(test)
807
808 # Look for tests in a module's contained objects.
809 if inspect.ismodule(obj) and self._recurse:
810 for valname, val in obj.__dict__.items():
811 # Check if this contained object should be ignored.
812 if self._filter(val, name, valname):
813 continue
814 valname = '%s.%s' % (name, valname)
815 # Recurse to functions & classes.
816 if ((inspect.isfunction(val) or inspect.isclass(val)) and
Tim Petersf3f57472004-08-08 06:11:48 +0000817 self._from_module(module, val)):
Tim Peters8485b562004-08-04 18:46:34 +0000818 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +0000819 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +0000820
821 # Look for tests in a module's __test__ dictionary.
822 if inspect.ismodule(obj) and self._recurse:
823 for valname, val in getattr(obj, '__test__', {}).items():
824 if not isinstance(valname, basestring):
825 raise ValueError("DocTestFinder.find: __test__ keys "
826 "must be strings: %r" %
827 (type(valname),))
828 if not (inspect.isfunction(val) or inspect.isclass(val) or
829 inspect.ismethod(val) or inspect.ismodule(val) or
830 isinstance(val, basestring)):
831 raise ValueError("DocTestFinder.find: __test__ values "
832 "must be strings, functions, methods, "
833 "classes, or modules: %r" %
834 (type(val),))
835 valname = '%s.%s' % (name, valname)
836 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +0000837 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +0000838
839 # Look for tests in a class's contained objects.
840 if inspect.isclass(obj) and self._recurse:
841 for valname, val in obj.__dict__.items():
842 # Check if this contained object should be ignored.
843 if self._filter(val, name, valname):
844 continue
845 # Special handling for staticmethod/classmethod.
846 if isinstance(val, staticmethod):
847 val = getattr(obj, valname)
848 if isinstance(val, classmethod):
849 val = getattr(obj, valname).im_func
850
851 # Recurse to methods, properties, and nested classes.
852 if ((inspect.isfunction(val) or inspect.isclass(val) or
Tim Petersf3f57472004-08-08 06:11:48 +0000853 isinstance(val, property)) and
854 self._from_module(module, val)):
Tim Peters8485b562004-08-04 18:46:34 +0000855 valname = '%s.%s' % (name, valname)
856 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +0000857 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +0000858
859 def _get_test(self, obj, name, module, globs, source_lines):
860 """
861 Return a DocTest for the given object, if it defines a docstring;
862 otherwise, return None.
863 """
864 # Extract the object's docstring. If it doesn't have one,
865 # then return None (no test for this object).
866 if isinstance(obj, basestring):
867 docstring = obj
868 else:
869 try:
870 if obj.__doc__ is None:
871 return None
872 docstring = str(obj.__doc__)
873 except (TypeError, AttributeError):
874 return None
875
876 # Don't bother if the docstring is empty.
877 if not docstring:
878 return None
879
880 # Find the docstring's location in the file.
881 lineno = self._find_lineno(obj, source_lines)
882
883 # Return a DocTest for this object.
884 if module is None:
885 filename = None
886 else:
887 filename = getattr(module, '__file__', module.__name__)
Tim Peters19397e52004-08-06 22:02:59 +0000888 return self._doctest_factory(docstring, globs, name, filename, lineno)
Tim Peters8485b562004-08-04 18:46:34 +0000889
890 def _find_lineno(self, obj, source_lines):
891 """
892 Return a line number of the given object's docstring. Note:
893 this method assumes that the object has a docstring.
894 """
895 lineno = None
896
897 # Find the line number for modules.
898 if inspect.ismodule(obj):
899 lineno = 0
900
901 # Find the line number for classes.
902 # Note: this could be fooled if a class is defined multiple
903 # times in a single file.
904 if inspect.isclass(obj):
905 if source_lines is None:
906 return None
907 pat = re.compile(r'^\s*class\s*%s\b' %
908 getattr(obj, '__name__', '-'))
909 for i, line in enumerate(source_lines):
910 if pat.match(line):
911 lineno = i
912 break
913
914 # Find the line number for functions & methods.
915 if inspect.ismethod(obj): obj = obj.im_func
916 if inspect.isfunction(obj): obj = obj.func_code
917 if inspect.istraceback(obj): obj = obj.tb_frame
918 if inspect.isframe(obj): obj = obj.f_code
919 if inspect.iscode(obj):
920 lineno = getattr(obj, 'co_firstlineno', None)-1
921
922 # Find the line number where the docstring starts. Assume
923 # that it's the first line that begins with a quote mark.
924 # Note: this could be fooled by a multiline function
925 # signature, where a continuation line begins with a quote
926 # mark.
927 if lineno is not None:
928 if source_lines is None:
929 return lineno+1
930 pat = re.compile('(^|.*:)\s*\w*("|\')')
931 for lineno in range(lineno, len(source_lines)):
932 if pat.match(source_lines[lineno]):
933 return lineno
934
935 # We couldn't find the line number.
936 return None
937
938######################################################################
Edward Loper7c748462004-08-09 02:06:06 +0000939## 5. DocTest Runner
Tim Peters8485b562004-08-04 18:46:34 +0000940######################################################################
941
Tim Peters8485b562004-08-04 18:46:34 +0000942class DocTestRunner:
943 """
944 A class used to run DocTest test cases, and accumulate statistics.
945 The `run` method is used to process a single DocTest case. It
946 returns a tuple `(f, t)`, where `t` is the number of test cases
947 tried, and `f` is the number of test cases that failed.
948
949 >>> tests = DocTestFinder().find(_TestClass)
950 >>> runner = DocTestRunner(verbose=False)
951 >>> for test in tests:
952 ... print runner.run(test)
953 (0, 2)
954 (0, 1)
955 (0, 2)
956 (0, 2)
957
958 The `summarize` method prints a summary of all the test cases that
959 have been run by the runner, and returns an aggregated `(f, t)`
960 tuple:
961
962 >>> runner.summarize(verbose=1)
963 4 items passed all tests:
964 2 tests in _TestClass
965 2 tests in _TestClass.__init__
966 2 tests in _TestClass.get
967 1 tests in _TestClass.square
968 7 tests in 4 items.
969 7 passed and 0 failed.
970 Test passed.
971 (0, 7)
972
973 The aggregated number of tried examples and failed examples is
974 also available via the `tries` and `failures` attributes:
975
976 >>> runner.tries
977 7
978 >>> runner.failures
979 0
980
981 The comparison between expected outputs and actual outputs is done
Edward Loper34fcb142004-08-09 02:45:41 +0000982 by an `OutputChecker`. This comparison may be customized with a
983 number of option flags; see the documentation for `testmod` for
984 more information. If the option flags are insufficient, then the
985 comparison may also be customized by passing a subclass of
986 `OutputChecker` to the constructor.
Tim Peters8485b562004-08-04 18:46:34 +0000987
988 The test runner's display output can be controlled in two ways.
989 First, an output function (`out) can be passed to
990 `TestRunner.run`; this function will be called with strings that
991 should be displayed. It defaults to `sys.stdout.write`. If
992 capturing the output is not sufficient, then the display output
993 can be also customized by subclassing DocTestRunner, and
994 overriding the methods `report_start`, `report_success`,
995 `report_unexpected_exception`, and `report_failure`.
996 """
997 # This divider string is used to separate failure messages, and to
998 # separate sections of the summary.
999 DIVIDER = "*" * 70
1000
Edward Loper34fcb142004-08-09 02:45:41 +00001001 def __init__(self, checker=None, verbose=None, optionflags=0):
Tim Peters8485b562004-08-04 18:46:34 +00001002 """
1003 Create a new test runner.
1004
Edward Loper34fcb142004-08-09 02:45:41 +00001005 Optional keyword arg `checker` is the `OutputChecker` that
1006 should be used to compare the expected outputs and actual
1007 outputs of doctest examples.
1008
Tim Peters8485b562004-08-04 18:46:34 +00001009 Optional keyword arg 'verbose' prints lots of stuff if true,
1010 only failures if false; by default, it's true iff '-v' is in
1011 sys.argv.
1012
1013 Optional argument `optionflags` can be used to control how the
1014 test runner compares expected output to actual output, and how
1015 it displays failures. See the documentation for `testmod` for
1016 more information.
1017 """
Edward Loper34fcb142004-08-09 02:45:41 +00001018 self._checker = checker or OutputChecker()
Tim Peters8a7d2d52001-01-16 07:10:57 +00001019 if verbose is None:
Tim Peters8485b562004-08-04 18:46:34 +00001020 verbose = '-v' in sys.argv
1021 self._verbose = verbose
Tim Peters6ebe61f2003-06-27 20:48:05 +00001022 self.optionflags = optionflags
1023
Tim Peters8485b562004-08-04 18:46:34 +00001024 # Keep track of the examples we've run.
1025 self.tries = 0
1026 self.failures = 0
1027 self._name2ft = {}
Tim Peters8a7d2d52001-01-16 07:10:57 +00001028
Tim Peters8485b562004-08-04 18:46:34 +00001029 # Create a fake output target for capturing doctest output.
1030 self._fakeout = _SpoofOut()
Tim Peters4fd9e2f2001-08-18 00:05:50 +00001031
Tim Peters8485b562004-08-04 18:46:34 +00001032 #/////////////////////////////////////////////////////////////////
Tim Peters8485b562004-08-04 18:46:34 +00001033 # Reporting methods
1034 #/////////////////////////////////////////////////////////////////
Tim Peters17111f32001-10-03 04:08:26 +00001035
Tim Peters8485b562004-08-04 18:46:34 +00001036 def report_start(self, out, test, example):
Tim Peters8a7d2d52001-01-16 07:10:57 +00001037 """
Tim Peters8485b562004-08-04 18:46:34 +00001038 Report that the test runner is about to process the given
1039 example. (Only displays a message if verbose=True)
1040 """
1041 if self._verbose:
1042 out(_tag_msg("Trying", example.source) +
1043 _tag_msg("Expecting", example.want or "nothing"))
Tim Peters8a7d2d52001-01-16 07:10:57 +00001044
Tim Peters8485b562004-08-04 18:46:34 +00001045 def report_success(self, out, test, example, got):
1046 """
1047 Report that the given example ran successfully. (Only
1048 displays a message if verbose=True)
1049 """
1050 if self._verbose:
1051 out("ok\n")
Tim Peters8a7d2d52001-01-16 07:10:57 +00001052
Tim Peters8485b562004-08-04 18:46:34 +00001053 def report_failure(self, out, test, example, got):
1054 """
1055 Report that the given example failed.
1056 """
1057 # Print an error message.
1058 out(self.__failure_header(test, example) +
Edward Loper34fcb142004-08-09 02:45:41 +00001059 self._checker.output_difference(example.want, got,
1060 self.optionflags))
Tim Peters7402f792001-10-02 03:53:41 +00001061
Tim Peters8485b562004-08-04 18:46:34 +00001062 def report_unexpected_exception(self, out, test, example, exc_info):
1063 """
1064 Report that the given example raised an unexpected exception.
1065 """
1066 # Get a traceback message.
1067 excout = StringIO()
1068 exc_type, exc_val, exc_tb = exc_info
1069 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout)
1070 exception_tb = excout.getvalue()
1071 # Print an error message.
1072 out(self.__failure_header(test, example) +
1073 _tag_msg("Exception raised", exception_tb))
Tim Peters7402f792001-10-02 03:53:41 +00001074
Tim Peters8485b562004-08-04 18:46:34 +00001075 def __failure_header(self, test, example):
1076 s = (self.DIVIDER + "\n" +
1077 _tag_msg("Failure in example", example.source))
1078 if test.filename is None:
1079 # [XX] I'm not putting +1 here, to give the same output
1080 # as the old version. But I think it *should* go here.
1081 return s + ("from line #%s of %s\n" %
1082 (example.lineno, test.name))
1083 elif test.lineno is None:
1084 return s + ("from line #%s of %s in %s\n" %
1085 (example.lineno+1, test.name, test.filename))
1086 else:
1087 lineno = test.lineno+example.lineno+1
1088 return s + ("from line #%s of %s (%s)\n" %
1089 (lineno, test.filename, test.name))
Tim Peters7402f792001-10-02 03:53:41 +00001090
Tim Peters8485b562004-08-04 18:46:34 +00001091 #/////////////////////////////////////////////////////////////////
1092 # DocTest Running
1093 #/////////////////////////////////////////////////////////////////
Tim Peters7402f792001-10-02 03:53:41 +00001094
Tim Peters8485b562004-08-04 18:46:34 +00001095 # A regular expression for handling `want` strings that contain
1096 # expected exceptions. It divides `want` into two pieces: the
1097 # pre-exception output (`out`) and the exception message (`exc`),
1098 # as generated by traceback.format_exception_only(). (I assume
1099 # that the exception_only message is the first non-indented line
1100 # starting with word characters after the "Traceback ...".)
1101 _EXCEPTION_RE = re.compile(('^(?P<out>.*)'
1102 '^(?P<hdr>Traceback \((?:%s|%s)\):)\s*$.*?'
1103 '^(?P<exc>\w+.*)') %
1104 ('most recent call last', 'innermost last'),
1105 re.MULTILINE | re.DOTALL)
Tim Peters7402f792001-10-02 03:53:41 +00001106
Tim Peters8485b562004-08-04 18:46:34 +00001107 _OPTION_DIRECTIVE_RE = re.compile('\s*doctest:\s*(?P<flags>[^#\n]*)')
Tim Peters7402f792001-10-02 03:53:41 +00001108
Tim Peters8485b562004-08-04 18:46:34 +00001109 def __handle_directive(self, example):
1110 """
1111 Check if the given example is actually a directive to doctest
1112 (to turn an optionflag on or off); and if it is, then handle
1113 the directive.
Tim Peters7402f792001-10-02 03:53:41 +00001114
Tim Peters8485b562004-08-04 18:46:34 +00001115 Return true iff the example is actually a directive (and so
1116 should not be executed).
Tim Peters4a9ac4a2001-10-02 22:47:08 +00001117
Tim Peters8a7d2d52001-01-16 07:10:57 +00001118 """
Tim Peters8485b562004-08-04 18:46:34 +00001119 m = self._OPTION_DIRECTIVE_RE.match(example.source)
1120 if m is None:
1121 return False
Tim Peters8a7d2d52001-01-16 07:10:57 +00001122
Tim Peters8485b562004-08-04 18:46:34 +00001123 for flag in m.group('flags').upper().split():
1124 if (flag[:1] not in '+-' or
1125 flag[1:] not in OPTIONFLAGS_BY_NAME):
1126 raise ValueError('Bad doctest option directive: '+flag)
1127 if flag[0] == '+':
1128 self.optionflags |= OPTIONFLAGS_BY_NAME[flag[1:]]
1129 else:
1130 self.optionflags &= ~OPTIONFLAGS_BY_NAME[flag[1:]]
1131 return True
Tim Peters8a7d2d52001-01-16 07:10:57 +00001132
Tim Peters8485b562004-08-04 18:46:34 +00001133 def __run(self, test, compileflags, out):
Tim Peters8a7d2d52001-01-16 07:10:57 +00001134 """
Tim Peters8485b562004-08-04 18:46:34 +00001135 Run the examples in `test`. Write the outcome of each example
1136 with one of the `DocTestRunner.report_*` methods, using the
1137 writer function `out`. `compileflags` is the set of compiler
1138 flags that should be used to execute examples. Return a tuple
1139 `(f, t)`, where `t` is the number of examples tried, and `f`
1140 is the number of examples that failed. The examples are run
1141 in the namespace `test.globs`.
1142 """
1143 # Keep track of the number of failures and tries.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001144 failures = tries = 0
Tim Peters8485b562004-08-04 18:46:34 +00001145
1146 # Save the option flags (since option directives can be used
1147 # to modify them).
1148 original_optionflags = self.optionflags
1149
1150 # Process each example.
1151 for example in test.examples:
1152 # Check if it's an option directive. If it is, then handle
1153 # it, and go on to the next example.
1154 if self.__handle_directive(example):
1155 continue
1156
1157 # Record that we started this example.
1158 tries += 1
1159 self.report_start(out, test, example)
1160
1161 # Run the example in the given context (globs), and record
1162 # any exception that gets raised. (But don't intercept
1163 # keyboard interrupts.)
1164 try:
Tim Peters208ca702004-08-09 04:12:36 +00001165 # Don't blink! This is where the user's code gets run.
Tim Petersbb431472004-08-09 03:51:46 +00001166 exec compile(example.source, "<string>", "single",
Tim Peters8485b562004-08-04 18:46:34 +00001167 compileflags, 1) in test.globs
1168 exception = None
1169 except KeyboardInterrupt:
1170 raise
1171 except:
1172 exception = sys.exc_info()
1173
Tim Peters208ca702004-08-09 04:12:36 +00001174 got = self._fakeout.getvalue() # the actual output
Tim Peters8485b562004-08-04 18:46:34 +00001175 self._fakeout.truncate(0)
1176
1177 # If the example executed without raising any exceptions,
1178 # then verify its output and report its outcome.
1179 if exception is None:
Edward Loper34fcb142004-08-09 02:45:41 +00001180 if self._checker.check_output(example.want, got,
1181 self.optionflags):
Tim Peters8485b562004-08-04 18:46:34 +00001182 self.report_success(out, test, example, got)
Tim Peters8a7d2d52001-01-16 07:10:57 +00001183 else:
Tim Peters8485b562004-08-04 18:46:34 +00001184 self.report_failure(out, test, example, got)
1185 failures += 1
1186
1187 # If the example raised an exception, then check if it was
1188 # expected.
1189 else:
1190 exc_info = sys.exc_info()
1191 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1]
1192
1193 # Search the `want` string for an exception. If we don't
1194 # find one, then report an unexpected exception.
1195 m = self._EXCEPTION_RE.match(example.want)
1196 if m is None:
1197 self.report_unexpected_exception(out, test, example,
1198 exc_info)
1199 failures += 1
1200 else:
1201 exc_hdr = m.group('hdr')+'\n' # Exception header
1202 # The test passes iff the pre-exception output and
1203 # the exception description match the values given
1204 # in `want`.
Edward Loper34fcb142004-08-09 02:45:41 +00001205 if (self._checker.check_output(m.group('out'), got,
1206 self.optionflags) and
1207 self._checker.check_output(m.group('exc'), exc_msg,
1208 self.optionflags)):
Tim Peters8485b562004-08-04 18:46:34 +00001209 # Is +exc_msg the right thing here??
1210 self.report_success(out, test, example,
1211 got+exc_hdr+exc_msg)
1212 else:
1213 self.report_failure(out, test, example,
1214 got+exc_hdr+exc_msg)
1215 failures += 1
1216
1217 # Restore the option flags (in case they were modified)
1218 self.optionflags = original_optionflags
1219
1220 # Record and return the number of failures and tries.
1221 self.__record_outcome(test, failures, tries)
Tim Peters8a7d2d52001-01-16 07:10:57 +00001222 return failures, tries
1223
Tim Peters8485b562004-08-04 18:46:34 +00001224 def __record_outcome(self, test, f, t):
1225 """
1226 Record the fact that the given DocTest (`test`) generated `f`
1227 failures out of `t` tried examples.
1228 """
1229 f2, t2 = self._name2ft.get(test.name, (0,0))
1230 self._name2ft[test.name] = (f+f2, t+t2)
1231 self.failures += f
1232 self.tries += t
1233
1234 def run(self, test, compileflags=None, out=None, clear_globs=True):
1235 """
1236 Run the examples in `test`, and display the results using the
1237 writer function `out`.
1238
1239 The examples are run in the namespace `test.globs`. If
1240 `clear_globs` is true (the default), then this namespace will
1241 be cleared after the test runs, to help with garbage
1242 collection. If you would like to examine the namespace after
1243 the test completes, then use `clear_globs=False`.
1244
1245 `compileflags` gives the set of flags that should be used by
1246 the Python compiler when running the examples. If not
1247 specified, then it will default to the set of future-import
1248 flags that apply to `globs`.
1249
1250 The output of each example is checked using
1251 `DocTestRunner.check_output`, and the results are formatted by
1252 the `DocTestRunner.report_*` methods.
1253 """
1254 if compileflags is None:
1255 compileflags = _extract_future_flags(test.globs)
Jim Fulton356fd192004-08-09 11:34:47 +00001256
Tim Peters8485b562004-08-04 18:46:34 +00001257 if out is None:
1258 out = sys.stdout.write
1259 saveout = sys.stdout
1260
Jim Fulton356fd192004-08-09 11:34:47 +00001261 # Note that don't save away the previous pdb.set_trace. Rather,
1262 # we safe pdb.set_trace on import (see import section above).
1263 # We then call and restore that original cersion. We do it this
1264 # way to make this feature testable. If we kept and called the
1265 # previous version, we'd end up restoring the original stdout,
1266 # which is not what we want.
1267 def set_trace():
1268 sys.stdout = saveout
1269 real_pdb_set_trace()
1270
Tim Peters8485b562004-08-04 18:46:34 +00001271 try:
1272 sys.stdout = self._fakeout
Jim Fulton356fd192004-08-09 11:34:47 +00001273 pdb.set_trace = set_trace
Tim Peters8485b562004-08-04 18:46:34 +00001274 return self.__run(test, compileflags, out)
1275 finally:
1276 sys.stdout = saveout
Jim Fulton356fd192004-08-09 11:34:47 +00001277 pdb.set_trace = real_pdb_set_trace
Tim Peters8485b562004-08-04 18:46:34 +00001278 if clear_globs:
1279 test.globs.clear()
1280
1281 #/////////////////////////////////////////////////////////////////
1282 # Summarization
1283 #/////////////////////////////////////////////////////////////////
Tim Peters8a7d2d52001-01-16 07:10:57 +00001284 def summarize(self, verbose=None):
1285 """
Tim Peters8485b562004-08-04 18:46:34 +00001286 Print a summary of all the test cases that have been run by
1287 this DocTestRunner, and return a tuple `(f, t)`, where `f` is
1288 the total number of failed examples, and `t` is the total
1289 number of tried examples.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001290
Tim Peters8485b562004-08-04 18:46:34 +00001291 The optional `verbose` argument controls how detailed the
1292 summary is. If the verbosity is not specified, then the
1293 DocTestRunner's verbosity is used.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001294 """
Tim Peters8a7d2d52001-01-16 07:10:57 +00001295 if verbose is None:
Tim Peters8485b562004-08-04 18:46:34 +00001296 verbose = self._verbose
Tim Peters8a7d2d52001-01-16 07:10:57 +00001297 notests = []
1298 passed = []
1299 failed = []
1300 totalt = totalf = 0
Tim Peters8485b562004-08-04 18:46:34 +00001301 for x in self._name2ft.items():
Tim Peters8a7d2d52001-01-16 07:10:57 +00001302 name, (f, t) = x
1303 assert f <= t
Tim Peters8485b562004-08-04 18:46:34 +00001304 totalt += t
1305 totalf += f
Tim Peters8a7d2d52001-01-16 07:10:57 +00001306 if t == 0:
1307 notests.append(name)
1308 elif f == 0:
1309 passed.append( (name, t) )
1310 else:
1311 failed.append(x)
1312 if verbose:
1313 if notests:
1314 print len(notests), "items had no tests:"
1315 notests.sort()
1316 for thing in notests:
1317 print " ", thing
1318 if passed:
1319 print len(passed), "items passed all tests:"
1320 passed.sort()
1321 for thing, count in passed:
1322 print " %3d tests in %s" % (count, thing)
1323 if failed:
Tim Peters8485b562004-08-04 18:46:34 +00001324 print self.DIVIDER
Tim Peters8a7d2d52001-01-16 07:10:57 +00001325 print len(failed), "items had failures:"
1326 failed.sort()
1327 for thing, (f, t) in failed:
1328 print " %3d of %3d in %s" % (f, t, thing)
1329 if verbose:
Tim Peters8485b562004-08-04 18:46:34 +00001330 print totalt, "tests in", len(self._name2ft), "items."
Tim Peters8a7d2d52001-01-16 07:10:57 +00001331 print totalt - totalf, "passed and", totalf, "failed."
1332 if totalf:
1333 print "***Test Failed***", totalf, "failures."
1334 elif verbose:
1335 print "Test passed."
1336 return totalf, totalt
1337
Edward Loper34fcb142004-08-09 02:45:41 +00001338class OutputChecker:
1339 """
1340 A class used to check the whether the actual output from a doctest
1341 example matches the expected output. `OutputChecker` defines two
1342 methods: `check_output`, which compares a given pair of outputs,
1343 and returns true if they match; and `output_difference`, which
1344 returns a string describing the differences between two outputs.
1345 """
1346 def check_output(self, want, got, optionflags):
1347 """
1348 Return True iff the actual output (`got`) matches the expected
1349 output (`want`). These strings are always considered to match
1350 if they are identical; but depending on what option flags the
1351 test runner is using, several non-exact match types are also
1352 possible. See the documentation for `TestRunner` for more
1353 information about option flags.
1354 """
1355 # Handle the common case first, for efficiency:
1356 # if they're string-identical, always return true.
1357 if got == want:
1358 return True
1359
1360 # The values True and False replaced 1 and 0 as the return
1361 # value for boolean comparisons in Python 2.3.
1362 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1):
1363 if (got,want) == ("True\n", "1\n"):
1364 return True
1365 if (got,want) == ("False\n", "0\n"):
1366 return True
1367
1368 # <BLANKLINE> can be used as a special sequence to signify a
1369 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used.
1370 if not (optionflags & DONT_ACCEPT_BLANKLINE):
1371 # Replace <BLANKLINE> in want with a blank line.
1372 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER),
1373 '', want)
1374 # If a line in got contains only spaces, then remove the
1375 # spaces.
1376 got = re.sub('(?m)^\s*?$', '', got)
1377 if got == want:
1378 return True
1379
1380 # This flag causes doctest to ignore any differences in the
1381 # contents of whitespace strings. Note that this can be used
1382 # in conjunction with the ELLISPIS flag.
1383 if (optionflags & NORMALIZE_WHITESPACE):
1384 got = ' '.join(got.split())
1385 want = ' '.join(want.split())
1386 if got == want:
1387 return True
1388
1389 # The ELLIPSIS flag says to let the sequence "..." in `want`
1390 # match any substring in `got`. We implement this by
1391 # transforming `want` into a regular expression.
1392 if (optionflags & ELLIPSIS):
1393 # Escape any special regexp characters
1394 want_re = re.escape(want)
1395 # Replace ellipsis markers ('...') with .*
1396 want_re = want_re.replace(re.escape(ELLIPSIS_MARKER), '.*')
1397 # Require that it matches the entire string; and set the
1398 # re.DOTALL flag (with '(?s)').
1399 want_re = '(?s)^%s$' % want_re
1400 # Check if the `want_re` regexp matches got.
1401 if re.match(want_re, got):
1402 return True
1403
1404 # We didn't find any match; return false.
1405 return False
1406
1407 def output_difference(self, want, got, optionflags):
1408 """
1409 Return a string describing the differences between the
1410 expected output (`want`) and the actual output (`got`).
1411 """
1412 # If <BLANKLINE>s are being used, then replace <BLANKLINE>
1413 # with blank lines in the expected output string.
1414 if not (optionflags & DONT_ACCEPT_BLANKLINE):
1415 want = re.sub('(?m)^%s$' % re.escape(BLANKLINE_MARKER), '', want)
1416
1417 # Check if we should use diff. Don't use diff if the actual
1418 # or expected outputs are too short, or if the expected output
1419 # contains an ellipsis marker.
1420 if ((optionflags & (UNIFIED_DIFF | CONTEXT_DIFF)) and
1421 want.count('\n') > 2 and got.count('\n') > 2 and
1422 not (optionflags & ELLIPSIS and '...' in want)):
1423 # Split want & got into lines.
1424 want_lines = [l+'\n' for l in want.split('\n')]
1425 got_lines = [l+'\n' for l in got.split('\n')]
1426 # Use difflib to find their differences.
1427 if optionflags & UNIFIED_DIFF:
1428 diff = difflib.unified_diff(want_lines, got_lines, n=2,
1429 fromfile='Expected', tofile='Got')
1430 kind = 'unified'
1431 elif optionflags & CONTEXT_DIFF:
1432 diff = difflib.context_diff(want_lines, got_lines, n=2,
1433 fromfile='Expected', tofile='Got')
1434 kind = 'context'
1435 else:
1436 assert 0, 'Bad diff option'
1437 # Remove trailing whitespace on diff output.
1438 diff = [line.rstrip() + '\n' for line in diff]
1439 return _tag_msg("Differences (" + kind + " diff)",
1440 ''.join(diff))
1441
1442 # If we're not using diff, then simply list the expected
1443 # output followed by the actual output.
1444 return (_tag_msg("Expected", want or "Nothing") +
1445 _tag_msg("Got", got))
1446
Tim Peters19397e52004-08-06 22:02:59 +00001447class DocTestFailure(Exception):
1448 """A DocTest example has failed in debugging mode.
1449
1450 The exception instance has variables:
1451
1452 - test: the DocTest object being run
1453
1454 - excample: the Example object that failed
1455
1456 - got: the actual output
1457 """
1458 def __init__(self, test, example, got):
1459 self.test = test
1460 self.example = example
1461 self.got = got
1462
1463 def __str__(self):
1464 return str(self.test)
1465
1466class UnexpectedException(Exception):
1467 """A DocTest example has encountered an unexpected exception
1468
1469 The exception instance has variables:
1470
1471 - test: the DocTest object being run
1472
1473 - excample: the Example object that failed
1474
1475 - exc_info: the exception info
1476 """
1477 def __init__(self, test, example, exc_info):
1478 self.test = test
1479 self.example = example
1480 self.exc_info = exc_info
1481
1482 def __str__(self):
1483 return str(self.test)
Tim Petersd1b78272004-08-07 06:03:09 +00001484
Tim Peters19397e52004-08-06 22:02:59 +00001485class DebugRunner(DocTestRunner):
1486 r"""Run doc tests but raise an exception as soon as there is a failure.
1487
1488 If an unexpected exception occurs, an UnexpectedException is raised.
1489 It contains the test, the example, and the original exception:
1490
1491 >>> runner = DebugRunner(verbose=False)
1492 >>> test = DocTest('>>> raise KeyError\n42', {}, 'foo', 'foo.py', 0)
1493 >>> try:
1494 ... runner.run(test)
1495 ... except UnexpectedException, failure:
1496 ... pass
1497
1498 >>> failure.test is test
1499 True
1500
1501 >>> failure.example.want
1502 '42\n'
1503
1504 >>> exc_info = failure.exc_info
1505 >>> raise exc_info[0], exc_info[1], exc_info[2]
1506 Traceback (most recent call last):
1507 ...
1508 KeyError
1509
1510 We wrap the original exception to give the calling application
1511 access to the test and example information.
1512
1513 If the output doesn't match, then a DocTestFailure is raised:
1514
1515 >>> test = DocTest('''
1516 ... >>> x = 1
1517 ... >>> x
1518 ... 2
1519 ... ''', {}, 'foo', 'foo.py', 0)
1520
1521 >>> try:
1522 ... runner.run(test)
1523 ... except DocTestFailure, failure:
1524 ... pass
1525
1526 DocTestFailure objects provide access to the test:
1527
1528 >>> failure.test is test
1529 True
1530
1531 As well as to the example:
1532
1533 >>> failure.example.want
1534 '2\n'
1535
1536 and the actual output:
1537
1538 >>> failure.got
1539 '1\n'
1540
1541 If a failure or error occurs, the globals are left intact:
1542
1543 >>> del test.globs['__builtins__']
1544 >>> test.globs
1545 {'x': 1}
1546
1547 >>> test = DocTest('''
1548 ... >>> x = 2
1549 ... >>> raise KeyError
1550 ... ''', {}, 'foo', 'foo.py', 0)
1551
1552 >>> runner.run(test)
1553 Traceback (most recent call last):
1554 ...
1555 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)>
Tim Petersd1b78272004-08-07 06:03:09 +00001556
Tim Peters19397e52004-08-06 22:02:59 +00001557 >>> del test.globs['__builtins__']
1558 >>> test.globs
1559 {'x': 2}
1560
1561 But the globals are cleared if there is no error:
1562
1563 >>> test = DocTest('''
1564 ... >>> x = 2
1565 ... ''', {}, 'foo', 'foo.py', 0)
1566
1567 >>> runner.run(test)
1568 (0, 1)
1569
1570 >>> test.globs
1571 {}
1572
1573 """
1574
1575 def run(self, test, compileflags=None, out=None, clear_globs=True):
1576 r = DocTestRunner.run(self, test, compileflags, out, False)
1577 if clear_globs:
1578 test.globs.clear()
1579 return r
1580
1581 def report_unexpected_exception(self, out, test, example, exc_info):
1582 raise UnexpectedException(test, example, exc_info)
1583
1584 def report_failure(self, out, test, example, got):
1585 raise DocTestFailure(test, example, got)
1586
Tim Peters8485b562004-08-04 18:46:34 +00001587######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001588## 6. Test Functions
Tim Peters8485b562004-08-04 18:46:34 +00001589######################################################################
1590# These should be backwards compatible.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001591
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001592def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None,
Tim Peters19397e52004-08-06 22:02:59 +00001593 report=True, optionflags=0, extraglobs=None,
1594 raise_on_error=False):
Tim Peters6ebe61f2003-06-27 20:48:05 +00001595 """m=None, name=None, globs=None, verbose=None, isprivate=None,
Tim Peters8485b562004-08-04 18:46:34 +00001596 report=True, optionflags=0, extraglobs=None
Tim Peters8a7d2d52001-01-16 07:10:57 +00001597
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001598 Test examples in docstrings in functions and classes reachable
1599 from module m (or the current module if m is not supplied), starting
Raymond Hettinger71adf7e2003-07-16 19:25:22 +00001600 with m.__doc__. Unless isprivate is specified, private names
1601 are not skipped.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001602
1603 Also test examples reachable from dict m.__test__ if it exists and is
1604 not None. m.__dict__ maps names to functions, classes and strings;
1605 function and class docstrings are tested even if the name is private;
1606 strings are tested directly, as if they were docstrings.
1607
1608 Return (#failures, #tests).
1609
1610 See doctest.__doc__ for an overview.
1611
1612 Optional keyword arg "name" gives the name of the module; by default
1613 use m.__name__.
1614
1615 Optional keyword arg "globs" gives a dict to be used as the globals
1616 when executing examples; by default, use m.__dict__. A copy of this
1617 dict is actually used for each docstring, so that each docstring's
1618 examples start with a clean slate.
1619
Tim Peters8485b562004-08-04 18:46:34 +00001620 Optional keyword arg "extraglobs" gives a dictionary that should be
1621 merged into the globals that are used to execute examples. By
1622 default, no extra globals are used. This is new in 2.4.
1623
Tim Peters8a7d2d52001-01-16 07:10:57 +00001624 Optional keyword arg "verbose" prints lots of stuff if true, prints
1625 only failures if false; by default, it's true iff "-v" is in sys.argv.
1626
Tim Peters8a7d2d52001-01-16 07:10:57 +00001627 Optional keyword arg "report" prints a summary at the end when true,
1628 else prints nothing at the end. In verbose mode, the summary is
1629 detailed, else very brief (in fact, empty if all tests passed).
1630
Tim Peters6ebe61f2003-06-27 20:48:05 +00001631 Optional keyword arg "optionflags" or's together module constants,
1632 and defaults to 0. This is new in 2.3. Possible values:
1633
1634 DONT_ACCEPT_TRUE_FOR_1
1635 By default, if an expected output block contains just "1",
1636 an actual output block containing just "True" is considered
1637 to be a match, and similarly for "0" versus "False". When
1638 DONT_ACCEPT_TRUE_FOR_1 is specified, neither substitution
1639 is allowed.
1640
Tim Peters8485b562004-08-04 18:46:34 +00001641 DONT_ACCEPT_BLANKLINE
1642 By default, if an expected output block contains a line
1643 containing only the string "<BLANKLINE>", then that line
1644 will match a blank line in the actual output. When
1645 DONT_ACCEPT_BLANKLINE is specified, this substitution is
1646 not allowed.
1647
1648 NORMALIZE_WHITESPACE
1649 When NORMALIZE_WHITESPACE is specified, all sequences of
1650 whitespace are treated as equal. I.e., any sequence of
1651 whitespace within the expected output will match any
1652 sequence of whitespace within the actual output.
1653
1654 ELLIPSIS
1655 When ELLIPSIS is specified, then an ellipsis marker
1656 ("...") in the expected output can match any substring in
1657 the actual output.
1658
1659 UNIFIED_DIFF
1660 When UNIFIED_DIFF is specified, failures that involve
1661 multi-line expected and actual outputs will be displayed
1662 using a unified diff.
1663
1664 CONTEXT_DIFF
1665 When CONTEXT_DIFF is specified, failures that involve
1666 multi-line expected and actual outputs will be displayed
1667 using a context diff.
Tim Peters19397e52004-08-06 22:02:59 +00001668
1669 Optional keyword arg "raise_on_error" raises an exception on the
1670 first unexpected exception or failure. This allows failures to be
1671 post-mortem debugged.
1672
Tim Petersf727c6c2004-08-08 01:48:59 +00001673 Deprecated in Python 2.4:
1674 Optional keyword arg "isprivate" specifies a function used to
1675 determine whether a name is private. The default function is
1676 treat all functions as public. Optionally, "isprivate" can be
1677 set to doctest.is_private to skip over functions marked as private
1678 using the underscore naming convention; see its docs for details.
Tim Peters8485b562004-08-04 18:46:34 +00001679 """
1680
1681 """ [XX] This is no longer true:
Tim Peters8a7d2d52001-01-16 07:10:57 +00001682 Advanced tomfoolery: testmod runs methods of a local instance of
1683 class doctest.Tester, then merges the results into (or creates)
1684 global Tester instance doctest.master. Methods of doctest.master
1685 can be called directly too, if you want to do something unusual.
1686 Passing report=0 to testmod is especially useful then, to delay
1687 displaying a summary. Invoke doctest.master.summarize(verbose)
1688 when you're done fiddling.
1689 """
Tim Petersf727c6c2004-08-08 01:48:59 +00001690 if isprivate is not None:
1691 warnings.warn("the isprivate argument is deprecated; "
1692 "examine DocTestFinder.find() lists instead",
1693 DeprecationWarning)
1694
Tim Peters8485b562004-08-04 18:46:34 +00001695 # If no module was given, then use __main__.
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001696 if m is None:
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001697 # DWA - m will still be None if this wasn't invoked from the command
1698 # line, in which case the following TypeError is about as good an error
1699 # as we should expect
1700 m = sys.modules.get('__main__')
1701
Tim Peters8485b562004-08-04 18:46:34 +00001702 # Check that we were actually given a module.
1703 if not inspect.ismodule(m):
Walter Dörwald70a6b492004-02-12 17:35:32 +00001704 raise TypeError("testmod: module required; %r" % (m,))
Tim Peters8485b562004-08-04 18:46:34 +00001705
1706 # If no name was given, then use the module's name.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001707 if name is None:
1708 name = m.__name__
Tim Peters8485b562004-08-04 18:46:34 +00001709
1710 # Find, parse, and run all tests in the given module.
Tim Petersf727c6c2004-08-08 01:48:59 +00001711 finder = DocTestFinder(_namefilter=isprivate)
Tim Peters19397e52004-08-06 22:02:59 +00001712
1713 if raise_on_error:
1714 runner = DebugRunner(verbose=verbose, optionflags=optionflags)
1715 else:
1716 runner = DocTestRunner(verbose=verbose, optionflags=optionflags)
1717
Tim Peters8485b562004-08-04 18:46:34 +00001718 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
1719 runner.run(test)
1720
Tim Peters8a7d2d52001-01-16 07:10:57 +00001721 if report:
Tim Peters8485b562004-08-04 18:46:34 +00001722 runner.summarize()
Tim Peters8a7d2d52001-01-16 07:10:57 +00001723
Tim Peters8485b562004-08-04 18:46:34 +00001724 return runner.failures, runner.tries
Tim Petersdb3756d2003-06-29 05:30:48 +00001725
Tim Peters8485b562004-08-04 18:46:34 +00001726def run_docstring_examples(f, globs, verbose=False, name="NoName",
1727 compileflags=None, optionflags=0):
1728 """
1729 Test examples in the given object's docstring (`f`), using `globs`
1730 as globals. Optional argument `name` is used in failure messages.
1731 If the optional argument `verbose` is true, then generate output
1732 even if there are no failures.
Tim Petersdb3756d2003-06-29 05:30:48 +00001733
Tim Peters8485b562004-08-04 18:46:34 +00001734 `compileflags` gives the set of flags that should be used by the
1735 Python compiler when running the examples. If not specified, then
1736 it will default to the set of future-import flags that apply to
1737 `globs`.
Tim Petersdb3756d2003-06-29 05:30:48 +00001738
Tim Peters8485b562004-08-04 18:46:34 +00001739 Optional keyword arg `optionflags` specifies options for the
1740 testing and output. See the documentation for `testmod` for more
1741 information.
1742 """
1743 # Find, parse, and run all tests in the given module.
1744 finder = DocTestFinder(verbose=verbose, recurse=False)
1745 runner = DocTestRunner(verbose=verbose, optionflags=optionflags)
1746 for test in finder.find(f, name, globs=globs):
1747 runner.run(test, compileflags=compileflags)
Tim Petersdb3756d2003-06-29 05:30:48 +00001748
Tim Peters8485b562004-08-04 18:46:34 +00001749######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001750## 7. Tester
Tim Peters8485b562004-08-04 18:46:34 +00001751######################################################################
1752# This is provided only for backwards compatibility. It's not
1753# actually used in any way.
Tim Petersdb3756d2003-06-29 05:30:48 +00001754
Tim Peters8485b562004-08-04 18:46:34 +00001755class Tester:
1756 def __init__(self, mod=None, globs=None, verbose=None,
1757 isprivate=None, optionflags=0):
Tim Peters3ddd60a2004-08-08 02:43:33 +00001758
1759 warnings.warn("class Tester is deprecated; "
1760 "use class doctest.DocTestRunner instead",
1761 DeprecationWarning, stacklevel=2)
Tim Peters8485b562004-08-04 18:46:34 +00001762 if mod is None and globs is None:
1763 raise TypeError("Tester.__init__: must specify mod or globs")
1764 if mod is not None and not _ismodule(mod):
1765 raise TypeError("Tester.__init__: mod must be a module; %r" %
1766 (mod,))
1767 if globs is None:
1768 globs = mod.__dict__
1769 self.globs = globs
Tim Petersdb3756d2003-06-29 05:30:48 +00001770
Tim Peters8485b562004-08-04 18:46:34 +00001771 self.verbose = verbose
1772 self.isprivate = isprivate
1773 self.optionflags = optionflags
Tim Petersf727c6c2004-08-08 01:48:59 +00001774 self.testfinder = DocTestFinder(_namefilter=isprivate)
Tim Peters8485b562004-08-04 18:46:34 +00001775 self.testrunner = DocTestRunner(verbose=verbose,
1776 optionflags=optionflags)
Tim Petersdb3756d2003-06-29 05:30:48 +00001777
Tim Peters8485b562004-08-04 18:46:34 +00001778 def runstring(self, s, name):
1779 test = DocTest(s, self.globs, name, None, None)
1780 if self.verbose:
1781 print "Running string", name
1782 (f,t) = self.testrunner.run(test)
1783 if self.verbose:
1784 print f, "of", t, "examples failed in string", name
1785 return (f,t)
Tim Petersdb3756d2003-06-29 05:30:48 +00001786
Tim Petersf3f57472004-08-08 06:11:48 +00001787 def rundoc(self, object, name=None, module=None):
Tim Peters8485b562004-08-04 18:46:34 +00001788 f = t = 0
1789 tests = self.testfinder.find(object, name, module=module,
Tim Petersf3f57472004-08-08 06:11:48 +00001790 globs=self.globs)
Tim Peters8485b562004-08-04 18:46:34 +00001791 for test in tests:
1792 (f2, t2) = self.testrunner.run(test)
1793 (f,t) = (f+f2, t+t2)
1794 return (f,t)
Tim Petersdb3756d2003-06-29 05:30:48 +00001795
Tim Peters8485b562004-08-04 18:46:34 +00001796 def rundict(self, d, name, module=None):
1797 import new
1798 m = new.module(name)
1799 m.__dict__.update(d)
Tim Petersf3f57472004-08-08 06:11:48 +00001800 if module is None:
1801 module = False
1802 return self.rundoc(m, name, module)
Tim Petersdb3756d2003-06-29 05:30:48 +00001803
Tim Peters8485b562004-08-04 18:46:34 +00001804 def run__test__(self, d, name):
1805 import new
1806 m = new.module(name)
1807 m.__test__ = d
1808 return self.rundoc(m, name, module)
Tim Petersdb3756d2003-06-29 05:30:48 +00001809
Tim Peters8485b562004-08-04 18:46:34 +00001810 def summarize(self, verbose=None):
1811 return self.testrunner.summarize(verbose)
Tim Petersdb3756d2003-06-29 05:30:48 +00001812
Tim Peters8485b562004-08-04 18:46:34 +00001813 def merge(self, other):
1814 d = self.testrunner._name2ft
1815 for name, (f, t) in other.testrunner._name2ft.items():
1816 if name in d:
1817 print "*** Tester.merge: '" + name + "' in both" \
1818 " testers; summing outcomes."
1819 f2, t2 = d[name]
1820 f = f + f2
1821 t = t + t2
1822 d[name] = f, t
Tim Petersdb3756d2003-06-29 05:30:48 +00001823
Tim Peters8485b562004-08-04 18:46:34 +00001824######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001825## 8. Unittest Support
Tim Peters8485b562004-08-04 18:46:34 +00001826######################################################################
Tim Petersdb3756d2003-06-29 05:30:48 +00001827
Tim Peters19397e52004-08-06 22:02:59 +00001828class DocTestCase(unittest.TestCase):
Tim Petersdb3756d2003-06-29 05:30:48 +00001829
Edward Loper34fcb142004-08-09 02:45:41 +00001830 def __init__(self, test, optionflags=0, setUp=None, tearDown=None,
1831 checker=None):
Jim Fultona643b652004-07-14 19:06:50 +00001832 unittest.TestCase.__init__(self)
Tim Peters19397e52004-08-06 22:02:59 +00001833 self._dt_optionflags = optionflags
Edward Loper34fcb142004-08-09 02:45:41 +00001834 self._dt_checker = checker
Tim Peters19397e52004-08-06 22:02:59 +00001835 self._dt_test = test
1836 self._dt_setUp = setUp
1837 self._dt_tearDown = tearDown
Tim Petersdb3756d2003-06-29 05:30:48 +00001838
Jim Fultona643b652004-07-14 19:06:50 +00001839 def setUp(self):
Tim Peters19397e52004-08-06 22:02:59 +00001840 if self._dt_setUp is not None:
1841 self._dt_setUp()
Jim Fultona643b652004-07-14 19:06:50 +00001842
1843 def tearDown(self):
Tim Peters19397e52004-08-06 22:02:59 +00001844 if self._dt_tearDown is not None:
1845 self._dt_tearDown()
Jim Fultona643b652004-07-14 19:06:50 +00001846
1847 def runTest(self):
Tim Peters19397e52004-08-06 22:02:59 +00001848 test = self._dt_test
Jim Fultona643b652004-07-14 19:06:50 +00001849 old = sys.stdout
1850 new = StringIO()
Edward Loper34fcb142004-08-09 02:45:41 +00001851 runner = DocTestRunner(optionflags=self._dt_optionflags,
1852 checker=self._dt_checker, verbose=False)
Tim Peters19397e52004-08-06 22:02:59 +00001853
Jim Fultona643b652004-07-14 19:06:50 +00001854 try:
Tim Peters19397e52004-08-06 22:02:59 +00001855 runner.DIVIDER = "-"*70
1856 failures, tries = runner.run(test, out=new.write)
Jim Fultona643b652004-07-14 19:06:50 +00001857 finally:
1858 sys.stdout = old
1859
1860 if failures:
Tim Peters19397e52004-08-06 22:02:59 +00001861 raise self.failureException(self.format_failure(new.getvalue()))
Tim Peters8485b562004-08-04 18:46:34 +00001862
Tim Peters19397e52004-08-06 22:02:59 +00001863 def format_failure(self, err):
1864 test = self._dt_test
1865 if test.lineno is None:
1866 lineno = 'unknown line number'
1867 else:
1868 lineno = 'line %s' % test.lineno
1869 lname = '.'.join(test.name.split('.')[-1:])
1870 return ('Failed doctest test for %s\n'
1871 ' File "%s", line %s, in %s\n\n%s'
1872 % (test.name, test.filename, lineno, lname, err)
1873 )
1874
1875 def debug(self):
1876 r"""Run the test case without results and without catching exceptions
1877
1878 The unit test framework includes a debug method on test cases
1879 and test suites to support post-mortem debugging. The test code
1880 is run in such a way that errors are not caught. This way a
1881 caller can catch the errors and initiate post-mortem debugging.
1882
1883 The DocTestCase provides a debug method that raises
1884 UnexpectedException errors if there is an unexepcted
1885 exception:
1886
1887 >>> test = DocTest('>>> raise KeyError\n42',
1888 ... {}, 'foo', 'foo.py', 0)
1889 >>> case = DocTestCase(test)
1890 >>> try:
1891 ... case.debug()
1892 ... except UnexpectedException, failure:
1893 ... pass
1894
1895 The UnexpectedException contains the test, the example, and
1896 the original exception:
1897
1898 >>> failure.test is test
1899 True
1900
1901 >>> failure.example.want
1902 '42\n'
1903
1904 >>> exc_info = failure.exc_info
1905 >>> raise exc_info[0], exc_info[1], exc_info[2]
1906 Traceback (most recent call last):
1907 ...
1908 KeyError
1909
1910 If the output doesn't match, then a DocTestFailure is raised:
1911
1912 >>> test = DocTest('''
1913 ... >>> x = 1
1914 ... >>> x
1915 ... 2
1916 ... ''', {}, 'foo', 'foo.py', 0)
1917 >>> case = DocTestCase(test)
1918
1919 >>> try:
1920 ... case.debug()
1921 ... except DocTestFailure, failure:
1922 ... pass
1923
1924 DocTestFailure objects provide access to the test:
1925
1926 >>> failure.test is test
1927 True
1928
1929 As well as to the example:
1930
1931 >>> failure.example.want
1932 '2\n'
1933
1934 and the actual output:
1935
1936 >>> failure.got
1937 '1\n'
1938
1939 """
1940
Edward Loper34fcb142004-08-09 02:45:41 +00001941 runner = DebugRunner(optionflags=self._dt_optionflags,
1942 checker=self._dt_checker, verbose=False)
Tim Peters19397e52004-08-06 22:02:59 +00001943 runner.run(self._dt_test, out=nooutput)
Jim Fultona643b652004-07-14 19:06:50 +00001944
1945 def id(self):
Tim Peters19397e52004-08-06 22:02:59 +00001946 return self._dt_test.name
Jim Fultona643b652004-07-14 19:06:50 +00001947
1948 def __repr__(self):
Tim Peters19397e52004-08-06 22:02:59 +00001949 name = self._dt_test.name.split('.')
Jim Fultona643b652004-07-14 19:06:50 +00001950 return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
1951
1952 __str__ = __repr__
1953
1954 def shortDescription(self):
Tim Peters19397e52004-08-06 22:02:59 +00001955 return "Doctest: " + self._dt_test.name
Jim Fultona643b652004-07-14 19:06:50 +00001956
Tim Peters19397e52004-08-06 22:02:59 +00001957def nooutput(*args):
1958 pass
Jim Fultona643b652004-07-14 19:06:50 +00001959
Tim Peters19397e52004-08-06 22:02:59 +00001960def DocTestSuite(module=None, globs=None, extraglobs=None,
1961 optionflags=0, test_finder=None,
Edward Loper34fcb142004-08-09 02:45:41 +00001962 setUp=lambda: None, tearDown=lambda: None,
1963 checker=None):
Tim Peters8485b562004-08-04 18:46:34 +00001964 """
Tim Peters19397e52004-08-06 22:02:59 +00001965 Convert doctest tests for a mudule to a unittest test suite.
Jim Fultona643b652004-07-14 19:06:50 +00001966
Tim Peters19397e52004-08-06 22:02:59 +00001967 This converts each documentation string in a module that
1968 contains doctest tests to a unittest test case. If any of the
1969 tests in a doc string fail, then the test case fails. An exception
1970 is raised showing the name of the file containing the test and a
Jim Fultona643b652004-07-14 19:06:50 +00001971 (sometimes approximate) line number.
1972
Tim Peters19397e52004-08-06 22:02:59 +00001973 The `module` argument provides the module to be tested. The argument
Jim Fultona643b652004-07-14 19:06:50 +00001974 can be either a module or a module name.
1975
1976 If no argument is given, the calling module is used.
Jim Fultona643b652004-07-14 19:06:50 +00001977 """
Jim Fultona643b652004-07-14 19:06:50 +00001978
Tim Peters8485b562004-08-04 18:46:34 +00001979 if test_finder is None:
1980 test_finder = DocTestFinder()
Tim Peters8485b562004-08-04 18:46:34 +00001981
Tim Peters19397e52004-08-06 22:02:59 +00001982 module = _normalize_module(module)
1983 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs)
1984 if globs is None:
1985 globs = module.__dict__
1986 if not tests: # [XX] why do we want to do this?
1987 raise ValueError(module, "has no tests")
Tim Petersdb3756d2003-06-29 05:30:48 +00001988
1989 tests.sort()
1990 suite = unittest.TestSuite()
Tim Peters8485b562004-08-04 18:46:34 +00001991 for test in tests:
Tim Peters19397e52004-08-06 22:02:59 +00001992 if len(test.examples) == 0:
1993 continue
Tim Peters8485b562004-08-04 18:46:34 +00001994 if not test.filename:
Tim Petersdb3756d2003-06-29 05:30:48 +00001995 filename = module.__file__
1996 if filename.endswith(".pyc"):
1997 filename = filename[:-1]
1998 elif filename.endswith(".pyo"):
1999 filename = filename[:-1]
Tim Peters8485b562004-08-04 18:46:34 +00002000 test.filename = filename
Edward Loper34fcb142004-08-09 02:45:41 +00002001 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown,
2002 checker))
Tim Peters19397e52004-08-06 22:02:59 +00002003
2004 return suite
2005
2006class DocFileCase(DocTestCase):
2007
2008 def id(self):
2009 return '_'.join(self._dt_test.name.split('.'))
2010
2011 def __repr__(self):
2012 return self._dt_test.filename
2013 __str__ = __repr__
2014
2015 def format_failure(self, err):
2016 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s'
2017 % (self._dt_test.name, self._dt_test.filename, err)
2018 )
2019
2020def DocFileTest(path, package=None, globs=None,
2021 setUp=None, tearDown=None,
2022 optionflags=0):
2023 package = _normalize_module(package)
2024 name = path.split('/')[-1]
2025 dir = os.path.split(package.__file__)[0]
2026 path = os.path.join(dir, *(path.split('/')))
2027 doc = open(path).read()
2028
2029 if globs is None:
2030 globs = {}
2031
2032 test = DocTest(doc, globs, name, path, 0)
2033
2034 return DocFileCase(test, optionflags, setUp, tearDown)
2035
2036def DocFileSuite(*paths, **kw):
2037 """Creates a suite of doctest files.
2038
2039 One or more text file paths are given as strings. These should
2040 use "/" characters to separate path segments. Paths are relative
2041 to the directory of the calling module, or relative to the package
2042 passed as a keyword argument.
2043
2044 A number of options may be provided as keyword arguments:
2045
2046 package
2047 The name of a Python package. Text-file paths will be
2048 interpreted relative to the directory containing this package.
2049 The package may be supplied as a package object or as a dotted
2050 package name.
2051
2052 setUp
2053 The name of a set-up function. This is called before running the
2054 tests in each file.
2055
2056 tearDown
2057 The name of a tear-down function. This is called after running the
2058 tests in each file.
2059
2060 globs
2061 A dictionary containing initial global variables for the tests.
2062 """
2063 suite = unittest.TestSuite()
2064
2065 # We do this here so that _normalize_module is called at the right
2066 # level. If it were called in DocFileTest, then this function
2067 # would be the caller and we might guess the package incorrectly.
2068 kw['package'] = _normalize_module(kw.get('package'))
2069
2070 for path in paths:
2071 suite.addTest(DocFileTest(path, **kw))
Jim Fultona643b652004-07-14 19:06:50 +00002072
Tim Petersdb3756d2003-06-29 05:30:48 +00002073 return suite
2074
Tim Peters8485b562004-08-04 18:46:34 +00002075######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00002076## 9. Debugging Support
Tim Peters8485b562004-08-04 18:46:34 +00002077######################################################################
Jim Fultona643b652004-07-14 19:06:50 +00002078
Tim Peters19397e52004-08-06 22:02:59 +00002079def script_from_examples(s):
2080 r"""Extract script from text with examples.
2081
2082 Converts text with examples to a Python script. Example input is
2083 converted to regular code. Example output and all other words
2084 are converted to comments:
2085
2086 >>> text = '''
2087 ... Here are examples of simple math.
2088 ...
2089 ... Python has super accurate integer addition
2090 ...
2091 ... >>> 2 + 2
2092 ... 5
2093 ...
2094 ... And very friendly error messages:
2095 ...
2096 ... >>> 1/0
2097 ... To Infinity
2098 ... And
2099 ... Beyond
2100 ...
2101 ... You can use logic if you want:
2102 ...
2103 ... >>> if 0:
2104 ... ... blah
2105 ... ... blah
2106 ... ...
2107 ...
2108 ... Ho hum
2109 ... '''
2110
2111 >>> print script_from_examples(text)
2112 # Here are examples of simple math.
2113 #
2114 # Python has super accurate integer addition
2115 #
2116 2 + 2
2117 # Expected:
2118 # 5
2119 #
2120 # And very friendly error messages:
2121 #
2122 1/0
2123 # Expected:
2124 # To Infinity
2125 # And
2126 # Beyond
2127 #
2128 # You can use logic if you want:
2129 #
2130 if 0:
2131 blah
2132 blah
2133 <BLANKLINE>
2134 #
2135 # Ho hum
2136 """
2137
2138 return Parser('<string>', s).get_program()
2139
Tim Peters8485b562004-08-04 18:46:34 +00002140def _want_comment(example):
2141 """
Tim Peters19397e52004-08-06 22:02:59 +00002142 Return a comment containing the expected output for the given example.
Tim Peters8485b562004-08-04 18:46:34 +00002143 """
Jim Fultona643b652004-07-14 19:06:50 +00002144 # Return the expected output, if any
Tim Peters8485b562004-08-04 18:46:34 +00002145 want = example.want
2146 if want:
Tim Peters19397e52004-08-06 22:02:59 +00002147 if want[-1] == '\n':
2148 want = want[:-1]
Tim Peters8485b562004-08-04 18:46:34 +00002149 want = "\n# ".join(want.split("\n"))
2150 want = "\n# Expected:\n# %s" % want
2151 return want
Tim Petersdb3756d2003-06-29 05:30:48 +00002152
2153def testsource(module, name):
Tim Peters19397e52004-08-06 22:02:59 +00002154 """Extract the test sources from a doctest docstring as a script.
Tim Petersdb3756d2003-06-29 05:30:48 +00002155
2156 Provide the module (or dotted name of the module) containing the
Jim Fultona643b652004-07-14 19:06:50 +00002157 test to be debugged and the name (within the module) of the object
2158 with the doc string with tests to be debugged.
Tim Petersdb3756d2003-06-29 05:30:48 +00002159 """
Tim Peters8485b562004-08-04 18:46:34 +00002160 module = _normalize_module(module)
2161 tests = DocTestFinder().find(module)
2162 test = [t for t in tests if t.name == name]
Tim Petersdb3756d2003-06-29 05:30:48 +00002163 if not test:
2164 raise ValueError(name, "not found in tests")
2165 test = test[0]
Tim Peters19397e52004-08-06 22:02:59 +00002166 testsrc = script_from_examples(test.docstring)
Jim Fultona643b652004-07-14 19:06:50 +00002167 return testsrc
Tim Petersdb3756d2003-06-29 05:30:48 +00002168
Jim Fultona643b652004-07-14 19:06:50 +00002169def debug_src(src, pm=False, globs=None):
Tim Peters19397e52004-08-06 22:02:59 +00002170 """Debug a single doctest docstring, in argument `src`'"""
2171 testsrc = script_from_examples(src)
Tim Peters8485b562004-08-04 18:46:34 +00002172 debug_script(testsrc, pm, globs)
Tim Petersdb3756d2003-06-29 05:30:48 +00002173
Jim Fultona643b652004-07-14 19:06:50 +00002174def debug_script(src, pm=False, globs=None):
Tim Peters19397e52004-08-06 22:02:59 +00002175 "Debug a test script. `src` is the script, as a string."
Tim Petersdb3756d2003-06-29 05:30:48 +00002176 import pdb
Tim Petersdb3756d2003-06-29 05:30:48 +00002177
Tim Petersdb3756d2003-06-29 05:30:48 +00002178 srcfilename = tempfile.mktemp("doctestdebug.py")
Tim Peters8485b562004-08-04 18:46:34 +00002179 f = open(srcfilename, 'w')
2180 f.write(src)
2181 f.close()
2182
Jim Fultona643b652004-07-14 19:06:50 +00002183 if globs:
2184 globs = globs.copy()
2185 else:
2186 globs = {}
Tim Petersdb3756d2003-06-29 05:30:48 +00002187
Tim Peters8485b562004-08-04 18:46:34 +00002188 if pm:
2189 try:
2190 execfile(srcfilename, globs, globs)
2191 except:
2192 print sys.exc_info()[1]
2193 pdb.post_mortem(sys.exc_info()[2])
2194 else:
2195 # Note that %r is vital here. '%s' instead can, e.g., cause
2196 # backslashes to get treated as metacharacters on Windows.
2197 pdb.run("execfile(%r)" % srcfilename, globs, globs)
Tim Petersdb3756d2003-06-29 05:30:48 +00002198
Jim Fultona643b652004-07-14 19:06:50 +00002199def debug(module, name, pm=False):
Tim Peters19397e52004-08-06 22:02:59 +00002200 """Debug a single doctest docstring.
Jim Fultona643b652004-07-14 19:06:50 +00002201
2202 Provide the module (or dotted name of the module) containing the
2203 test to be debugged and the name (within the module) of the object
Tim Peters19397e52004-08-06 22:02:59 +00002204 with the docstring with tests to be debugged.
Jim Fultona643b652004-07-14 19:06:50 +00002205 """
Tim Peters8485b562004-08-04 18:46:34 +00002206 module = _normalize_module(module)
Jim Fultona643b652004-07-14 19:06:50 +00002207 testsrc = testsource(module, name)
2208 debug_script(testsrc, pm, module.__dict__)
2209
Tim Peters8485b562004-08-04 18:46:34 +00002210######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00002211## 10. Example Usage
Tim Peters8485b562004-08-04 18:46:34 +00002212######################################################################
Tim Peters8a7d2d52001-01-16 07:10:57 +00002213class _TestClass:
2214 """
2215 A pointless class, for sanity-checking of docstring testing.
2216
2217 Methods:
2218 square()
2219 get()
2220
2221 >>> _TestClass(13).get() + _TestClass(-12).get()
2222 1
2223 >>> hex(_TestClass(13).square().get())
2224 '0xa9'
2225 """
2226
2227 def __init__(self, val):
2228 """val -> _TestClass object with associated value val.
2229
2230 >>> t = _TestClass(123)
2231 >>> print t.get()
2232 123
2233 """
2234
2235 self.val = val
2236
2237 def square(self):
2238 """square() -> square TestClass's associated value
2239
2240 >>> _TestClass(13).square().get()
2241 169
2242 """
2243
2244 self.val = self.val ** 2
2245 return self
2246
2247 def get(self):
2248 """get() -> return TestClass's associated value.
2249
2250 >>> x = _TestClass(-42)
2251 >>> print x.get()
2252 -42
2253 """
2254
2255 return self.val
2256
2257__test__ = {"_TestClass": _TestClass,
2258 "string": r"""
2259 Example of a string object, searched as-is.
2260 >>> x = 1; y = 2
2261 >>> x + y, x * y
2262 (3, 2)
Tim Peters6ebe61f2003-06-27 20:48:05 +00002263 """,
2264 "bool-int equivalence": r"""
2265 In 2.2, boolean expressions displayed
2266 0 or 1. By default, we still accept
2267 them. This can be disabled by passing
2268 DONT_ACCEPT_TRUE_FOR_1 to the new
2269 optionflags argument.
2270 >>> 4 == 4
2271 1
2272 >>> 4 == 4
2273 True
2274 >>> 4 > 4
2275 0
2276 >>> 4 > 4
2277 False
2278 """,
Tim Peters8485b562004-08-04 18:46:34 +00002279 "blank lines": r"""
2280 Blank lines can be marked with <BLANKLINE>:
2281 >>> print 'foo\n\nbar\n'
2282 foo
2283 <BLANKLINE>
2284 bar
2285 <BLANKLINE>
2286 """,
2287 }
2288# "ellipsis": r"""
2289# If the ellipsis flag is used, then '...' can be used to
2290# elide substrings in the desired output:
2291# >>> print range(1000)
2292# [0, 1, 2, ..., 999]
2293# """,
2294# "whitespace normalization": r"""
2295# If the whitespace normalization flag is used, then
2296# differences in whitespace are ignored.
2297# >>> print range(30)
2298# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
2299# 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
2300# 27, 28, 29]
2301# """,
2302# }
2303
2304def test1(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002305>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning,
2306... "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002307>>> from doctest import Tester
2308>>> t = Tester(globs={'x': 42}, verbose=0)
2309>>> t.runstring(r'''
2310... >>> x = x * 2
2311... >>> print x
2312... 42
2313... ''', 'XYZ')
2314**********************************************************************
2315Failure in example: print x
2316from line #2 of XYZ
2317Expected: 42
2318Got: 84
2319(1, 2)
2320>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2')
2321(0, 2)
2322>>> t.summarize()
2323**********************************************************************
23241 items had failures:
2325 1 of 2 in XYZ
2326***Test Failed*** 1 failures.
2327(1, 4)
2328>>> t.summarize(verbose=1)
23291 items passed all tests:
2330 2 tests in example2
2331**********************************************************************
23321 items had failures:
2333 1 of 2 in XYZ
23344 tests in 2 items.
23353 passed and 1 failed.
2336***Test Failed*** 1 failures.
2337(1, 4)
2338"""
2339
2340def test2(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002341 >>> warnings.filterwarnings("ignore", "class Tester",
2342 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002343 >>> t = Tester(globs={}, verbose=1)
2344 >>> test = r'''
2345 ... # just an example
2346 ... >>> x = 1 + 2
2347 ... >>> x
2348 ... 3
2349 ... '''
2350 >>> t.runstring(test, "Example")
2351 Running string Example
2352 Trying: x = 1 + 2
2353 Expecting: nothing
2354 ok
2355 Trying: x
2356 Expecting: 3
2357 ok
2358 0 of 2 examples failed in string Example
2359 (0, 2)
2360"""
2361def test3(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002362 >>> warnings.filterwarnings("ignore", "class Tester",
2363 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002364 >>> t = Tester(globs={}, verbose=0)
2365 >>> def _f():
2366 ... '''Trivial docstring example.
2367 ... >>> assert 2 == 2
2368 ... '''
2369 ... return 32
2370 ...
2371 >>> t.rundoc(_f) # expect 0 failures in 1 example
2372 (0, 1)
2373"""
2374def test4(): """
2375 >>> import new
2376 >>> m1 = new.module('_m1')
2377 >>> m2 = new.module('_m2')
2378 >>> test_data = \"""
2379 ... def _f():
2380 ... '''>>> assert 1 == 1
2381 ... '''
2382 ... def g():
2383 ... '''>>> assert 2 != 1
2384 ... '''
2385 ... class H:
2386 ... '''>>> assert 2 > 1
2387 ... '''
2388 ... def bar(self):
2389 ... '''>>> assert 1 < 2
2390 ... '''
2391 ... \"""
2392 >>> exec test_data in m1.__dict__
2393 >>> exec test_data in m2.__dict__
2394 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H})
2395
2396 Tests that objects outside m1 are excluded:
2397
Tim Peters3ddd60a2004-08-08 02:43:33 +00002398 >>> warnings.filterwarnings("ignore", "class Tester",
2399 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002400 >>> t = Tester(globs={}, verbose=0)
Tim Petersf727c6c2004-08-08 01:48:59 +00002401 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped
Tim Peters8485b562004-08-04 18:46:34 +00002402 (0, 4)
2403
Tim Petersf727c6c2004-08-08 01:48:59 +00002404 Once more, not excluding stuff outside m1:
Tim Peters8485b562004-08-04 18:46:34 +00002405
2406 >>> t = Tester(globs={}, verbose=0)
2407 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped.
2408 (0, 8)
2409
2410 The exclusion of objects from outside the designated module is
2411 meant to be invoked automagically by testmod.
2412
Tim Petersf727c6c2004-08-08 01:48:59 +00002413 >>> testmod(m1, verbose=False)
2414 (0, 4)
Tim Peters8485b562004-08-04 18:46:34 +00002415"""
Tim Peters8a7d2d52001-01-16 07:10:57 +00002416
2417def _test():
Tim Peters8485b562004-08-04 18:46:34 +00002418 #import doctest
2419 #doctest.testmod(doctest, verbose=False,
2420 # optionflags=ELLIPSIS | NORMALIZE_WHITESPACE |
2421 # UNIFIED_DIFF)
2422 #print '~'*70
2423 r = unittest.TextTestRunner()
2424 r.run(DocTestSuite())
Tim Peters8a7d2d52001-01-16 07:10:57 +00002425
2426if __name__ == "__main__":
2427 _test()