blob: dcd64072e1a65d9b448c885bbfa61c4ef62d786b [file] [log] [blame]
Tim Peters4fd9e2f2001-08-18 00:05:50 +00001# Module doctest.
Tim Peters8485b562004-08-04 18:46:34 +00002# Released to the public domain 16-Jan-2001, by Tim Peters (tim@python.org).
Tim Peters19397e52004-08-06 22:02:59 +00003# Major enhancements and refactoring by:
Tim Peters8485b562004-08-04 18:46:34 +00004# Jim Fulton
5# Edward Loper
Tim Peters8a7d2d52001-01-16 07:10:57 +00006
7# Provided as-is; use at your own risk; no warranty; no promises; enjoy!
8
Martin v. Löwis92816de2004-05-31 19:01:00 +00009r"""Module doctest -- a framework for running examples in docstrings.
Tim Peters8a7d2d52001-01-16 07:10:57 +000010
11NORMAL USAGE
12
Tim Peters80e53142004-08-09 04:34:45 +000013In simplest use, end each module M to be tested with:
Tim Peters8a7d2d52001-01-16 07:10:57 +000014
15def _test():
Tim Peters80e53142004-08-09 04:34:45 +000016 import doctest
17 return doctest.testmod()
Tim Peters8a7d2d52001-01-16 07:10:57 +000018
19if __name__ == "__main__":
20 _test()
21
22Then running the module as a script will cause the examples in the
23docstrings to get executed and verified:
24
25python M.py
26
27This won't display anything unless an example fails, in which case the
28failing example(s) and the cause(s) of the failure(s) are printed to stdout
29(why not stderr? because stderr is a lame hack <0.2 wink>), and the final
30line of output is "Test failed.".
31
32Run it with the -v switch instead:
33
34python M.py -v
35
36and a detailed report of all examples tried is printed to stdout, along
37with assorted summaries at the end.
38
Tim Peters80e53142004-08-09 04:34:45 +000039You can force verbose mode by passing "verbose=True" to testmod, or prohibit
40it by passing "verbose=False". In either of those cases, sys.argv is not
Tim Peters8a7d2d52001-01-16 07:10:57 +000041examined by testmod.
42
43In any case, testmod returns a 2-tuple of ints (f, t), where f is the
44number of docstring examples that failed and t is the total number of
45docstring examples attempted.
46
Tim Peters80e53142004-08-09 04:34:45 +000047There are a variety of other ways to run doctests, including integration
48with the unittest framework, and support for running non-Python text
49files containing doctests. There are also many ways to override parts
50of doctest's default behaviors. See the Library Reference Manual for
51details.
52
Tim Peters8a7d2d52001-01-16 07:10:57 +000053
54WHICH DOCSTRINGS ARE EXAMINED?
55
56+ M.__doc__.
57
58+ f.__doc__ for all functions f in M.__dict__.values(), except those
Raymond Hettinger71adf7e2003-07-16 19:25:22 +000059 defined in other modules.
Tim Peters8a7d2d52001-01-16 07:10:57 +000060
Raymond Hettinger71adf7e2003-07-16 19:25:22 +000061+ C.__doc__ for all classes C in M.__dict__.values(), except those
62 defined in other modules.
Tim Peters8a7d2d52001-01-16 07:10:57 +000063
64+ If M.__test__ exists and "is true", it must be a dict, and
65 each entry maps a (string) name to a function object, class object, or
66 string. Function and class object docstrings found from M.__test__
Tim Peters80e53142004-08-09 04:34:45 +000067 are searched, and strings are searched directly as if they were docstrings.
68 In output, a key K in M.__test__ appears with name
Tim Peters8a7d2d52001-01-16 07:10:57 +000069 <name of M>.__test__.K
70
71Any classes found are recursively searched similarly, to test docstrings in
Tim Peters80e53142004-08-09 04:34:45 +000072their contained methods and nested classes.
Tim Peters8a7d2d52001-01-16 07:10:57 +000073
Tim Peters8a7d2d52001-01-16 07:10:57 +000074
Tim Peters8a7d2d52001-01-16 07:10:57 +000075WHAT'S THE EXECUTION CONTEXT?
76
77By default, each time testmod finds a docstring to test, it uses a *copy*
78of M's globals (so that running tests on a module doesn't change the
79module's real globals, and so that one test in M can't leave behind crumbs
80that accidentally allow another test to work). This means examples can
81freely use any names defined at top-level in M. It also means that sloppy
82imports (see above) can cause examples in external docstrings to use
83globals inappropriate for them.
84
85You can force use of your own dict as the execution context by passing
86"globs=your_dict" to testmod instead. Presumably this would be a copy of
87M.__dict__ merged with the globals from other imported modules.
88
89
Tim Peters8a7d2d52001-01-16 07:10:57 +000090WHAT ABOUT EXCEPTIONS?
91
92No problem, as long as the only output generated by the example is the
93traceback itself. For example:
94
Tim Peters60e23f42001-02-14 00:43:21 +000095 >>> [1, 2, 3].remove(42)
Tim Petersea4f9312001-02-13 20:54:42 +000096 Traceback (most recent call last):
Tim Peters8a7d2d52001-01-16 07:10:57 +000097 File "<stdin>", line 1, in ?
Tim Peters60e23f42001-02-14 00:43:21 +000098 ValueError: list.remove(x): x not in list
Tim Peters8a7d2d52001-01-16 07:10:57 +000099 >>>
100
Tim Peters80e53142004-08-09 04:34:45 +0000101Note that only the exception type and value are compared.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000102
103
Tim Peters80e53142004-08-09 04:34:45 +0000104SO WHAT DOES A DOCTEST EXAMPLE LOOK LIKE ALREADY!?
Tim Peters8a7d2d52001-01-16 07:10:57 +0000105
106Oh ya. It's easy! In most cases a copy-and-paste of an interactive
107console session works fine -- just make sure the leading whitespace is
108rigidly consistent (you can mix tabs and spaces if you're too lazy to do it
109right, but doctest is not in the business of guessing what you think a tab
110means).
111
112 >>> # comments are ignored
113 >>> x = 12
114 >>> x
115 12
116 >>> if x == 13:
117 ... print "yes"
118 ... else:
119 ... print "no"
120 ... print "NO"
121 ... print "NO!!!"
122 ...
123 no
124 NO
125 NO!!!
126 >>>
127
128Any expected output must immediately follow the final ">>>" or "..." line
129containing the code, and the expected output (if any) extends to the next
130">>>" or all-whitespace line. That's it.
131
132Bummers:
133
Tim Peters8a7d2d52001-01-16 07:10:57 +0000134+ Output to stdout is captured, but not output to stderr (exception
135 tracebacks are captured via a different means).
136
Martin v. Löwis92816de2004-05-31 19:01:00 +0000137+ If you continue a line via backslashing in an interactive session,
138 or for any other reason use a backslash, you should use a raw
139 docstring, which will preserve your backslahses exactly as you type
140 them:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000141
Tim Peters4e0e1b62004-07-07 20:54:48 +0000142 >>> def f(x):
Martin v. Löwis92816de2004-05-31 19:01:00 +0000143 ... r'''Backslashes in a raw docstring: m\n'''
144 >>> print f.__doc__
145 Backslashes in a raw docstring: m\n
Tim Peters8a7d2d52001-01-16 07:10:57 +0000146
Martin v. Löwis92816de2004-05-31 19:01:00 +0000147 Otherwise, the backslash will be interpreted as part of the string.
148 E.g., the "\n" above would be interpreted as a newline character.
149 Alternatively, you can double each backslash in the doctest version
150 (and not use a raw string):
151
Tim Peters4e0e1b62004-07-07 20:54:48 +0000152 >>> def f(x):
Martin v. Löwis92816de2004-05-31 19:01:00 +0000153 ... '''Backslashes in a raw docstring: m\\n'''
154 >>> print f.__doc__
155 Backslashes in a raw docstring: m\n
Tim Peters4e0e1b62004-07-07 20:54:48 +0000156
Tim Peters8a7d2d52001-01-16 07:10:57 +0000157The starting column doesn't matter:
158
159>>> assert "Easy!"
160 >>> import math
161 >>> math.floor(1.9)
162 1.0
163
164and as many leading whitespace characters are stripped from the expected
165output as appeared in the initial ">>>" line that triggered it.
166
167If you execute this very file, the examples above will be found and
Tim Peters80e53142004-08-09 04:34:45 +0000168executed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000169"""
Edward Loper8e4a34b2004-08-12 02:34:27 +0000170__docformat__ = 'reStructuredText en'
Tim Peters8a7d2d52001-01-16 07:10:57 +0000171
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000172__all__ = [
Edward Loperb7503ff2004-08-19 19:19:03 +0000173 # 0, Option Flags
174 'register_optionflag',
175 'DONT_ACCEPT_TRUE_FOR_1',
176 'DONT_ACCEPT_BLANKLINE',
177 'NORMALIZE_WHITESPACE',
178 'ELLIPSIS',
179 'UNIFIED_DIFF',
180 'CONTEXT_DIFF',
181 # 1. Utility Functions
Tim Peters8485b562004-08-04 18:46:34 +0000182 'is_private',
Edward Loperb7503ff2004-08-19 19:19:03 +0000183 # 2. Example & DocTest
Tim Peters8485b562004-08-04 18:46:34 +0000184 'Example',
185 'DocTest',
Edward Loperb7503ff2004-08-19 19:19:03 +0000186 # 3. Doctest Parser
187 'DocTestParser',
188 # 4. Doctest Finder
Tim Peters8485b562004-08-04 18:46:34 +0000189 'DocTestFinder',
Edward Loperb7503ff2004-08-19 19:19:03 +0000190 # 5. Doctest Runner
Tim Peters8485b562004-08-04 18:46:34 +0000191 'DocTestRunner',
Edward Loperb7503ff2004-08-19 19:19:03 +0000192 'OutputChecker',
193 'DocTestFailure',
194 'UnexpectedException',
195 'DebugRunner',
196 # 6. Test Functions
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000197 'testmod',
198 'run_docstring_examples',
Edward Loperb7503ff2004-08-19 19:19:03 +0000199 # 7. Tester
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000200 'Tester',
Edward Loperb7503ff2004-08-19 19:19:03 +0000201 # 8. Unittest Support
Tim Peters19397e52004-08-06 22:02:59 +0000202 'DocTestCase',
Edward Loperb7503ff2004-08-19 19:19:03 +0000203 'nooutput',
Tim Petersdb3756d2003-06-29 05:30:48 +0000204 'DocTestSuite',
Edward Loperb7503ff2004-08-19 19:19:03 +0000205 'DocFileCase',
206 'DocFileTest',
207 'DocFileSuite',
208 # 9. Debugging Support
209 'script_from_examples',
Tim Petersdb3756d2003-06-29 05:30:48 +0000210 'testsource',
Edward Loperb7503ff2004-08-19 19:19:03 +0000211 'debug_src',
212 'debug_script',
Tim Petersdb3756d2003-06-29 05:30:48 +0000213 'debug',
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000214]
Tim Peters8a7d2d52001-01-16 07:10:57 +0000215
Tim Peters4fd9e2f2001-08-18 00:05:50 +0000216import __future__
Tim Peters8a7d2d52001-01-16 07:10:57 +0000217
Tim Peters19397e52004-08-06 22:02:59 +0000218import sys, traceback, inspect, linecache, os, re, types
Jim Fulton356fd192004-08-09 11:34:47 +0000219import unittest, difflib, pdb, tempfile
Tim Petersf727c6c2004-08-08 01:48:59 +0000220import warnings
Tim Peters8485b562004-08-04 18:46:34 +0000221from StringIO import StringIO
Tim Peters7402f792001-10-02 03:53:41 +0000222
Jim Fulton356fd192004-08-09 11:34:47 +0000223real_pdb_set_trace = pdb.set_trace
224
Tim Peters19397e52004-08-06 22:02:59 +0000225# There are 4 basic classes:
226# - Example: a <source, want> pair, plus an intra-docstring line number.
227# - DocTest: a collection of examples, parsed from a docstring, plus
228# info about where the docstring came from (name, filename, lineno).
229# - DocTestFinder: extracts DocTests from a given object's docstring and
230# its contained objects' docstrings.
231# - DocTestRunner: runs DocTest cases, and accumulates statistics.
232#
233# So the basic picture is:
234#
235# list of:
236# +------+ +---------+ +-------+
237# |object| --DocTestFinder-> | DocTest | --DocTestRunner-> |results|
238# +------+ +---------+ +-------+
239# | Example |
240# | ... |
241# | Example |
242# +---------+
243
Edward Loperac20f572004-08-12 02:02:24 +0000244# Option constants.
245OPTIONFLAGS_BY_NAME = {}
246def register_optionflag(name):
247 flag = 1 << len(OPTIONFLAGS_BY_NAME)
248 OPTIONFLAGS_BY_NAME[name] = flag
249 return flag
250
251DONT_ACCEPT_TRUE_FOR_1 = register_optionflag('DONT_ACCEPT_TRUE_FOR_1')
252DONT_ACCEPT_BLANKLINE = register_optionflag('DONT_ACCEPT_BLANKLINE')
253NORMALIZE_WHITESPACE = register_optionflag('NORMALIZE_WHITESPACE')
254ELLIPSIS = register_optionflag('ELLIPSIS')
255UNIFIED_DIFF = register_optionflag('UNIFIED_DIFF')
256CONTEXT_DIFF = register_optionflag('CONTEXT_DIFF')
257
258# Special string markers for use in `want` strings:
259BLANKLINE_MARKER = '<BLANKLINE>'
260ELLIPSIS_MARKER = '...'
261
Tim Peters8485b562004-08-04 18:46:34 +0000262######################################################################
263## Table of Contents
264######################################################################
Edward Loper7c748462004-08-09 02:06:06 +0000265# 1. Utility Functions
266# 2. Example & DocTest -- store test cases
267# 3. DocTest Parser -- extracts examples from strings
268# 4. DocTest Finder -- extracts test cases from objects
269# 5. DocTest Runner -- runs test cases
270# 6. Test Functions -- convenient wrappers for testing
271# 7. Tester Class -- for backwards compatibility
272# 8. Unittest Support
273# 9. Debugging Support
274# 10. Example Usage
Tim Peters8a7d2d52001-01-16 07:10:57 +0000275
Tim Peters8485b562004-08-04 18:46:34 +0000276######################################################################
277## 1. Utility Functions
278######################################################################
Tim Peters8a7d2d52001-01-16 07:10:57 +0000279
280def is_private(prefix, base):
281 """prefix, base -> true iff name prefix + "." + base is "private".
282
283 Prefix may be an empty string, and base does not contain a period.
284 Prefix is ignored (although functions you write conforming to this
285 protocol may make use of it).
286 Return true iff base begins with an (at least one) underscore, but
287 does not both begin and end with (at least) two underscores.
288
Tim Petersbafb1fe2004-08-08 01:52:57 +0000289 >>> warnings.filterwarnings("ignore", "is_private", DeprecationWarning,
290 ... "doctest", 0)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000291 >>> is_private("a.b", "my_func")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000292 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000293 >>> is_private("____", "_my_func")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000294 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000295 >>> is_private("someclass", "__init__")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000296 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000297 >>> is_private("sometypo", "__init_")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000298 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000299 >>> is_private("x.y.z", "_")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000300 True
Tim Peters8a7d2d52001-01-16 07:10:57 +0000301 >>> is_private("_x.y.z", "__")
Guido van Rossum77f6a652002-04-03 22:41:51 +0000302 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000303 >>> is_private("", "") # senseless but consistent
Guido van Rossum77f6a652002-04-03 22:41:51 +0000304 False
Tim Peters8a7d2d52001-01-16 07:10:57 +0000305 """
Tim Petersbafb1fe2004-08-08 01:52:57 +0000306 warnings.warn("is_private is deprecated; it wasn't useful; "
307 "examine DocTestFinder.find() lists instead",
Tim Peters3ddd60a2004-08-08 02:43:33 +0000308 DeprecationWarning, stacklevel=2)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000309 return base[:1] == "_" and not base[:2] == "__" == base[-2:]
310
Tim Peters8485b562004-08-04 18:46:34 +0000311def _extract_future_flags(globs):
312 """
313 Return the compiler-flags associated with the future features that
314 have been imported into the given namespace (globs).
315 """
316 flags = 0
317 for fname in __future__.all_feature_names:
318 feature = globs.get(fname, None)
319 if feature is getattr(__future__, fname):
320 flags |= feature.compiler_flag
321 return flags
Tim Peters7402f792001-10-02 03:53:41 +0000322
Tim Peters8485b562004-08-04 18:46:34 +0000323def _normalize_module(module, depth=2):
324 """
325 Return the module specified by `module`. In particular:
326 - If `module` is a module, then return module.
327 - If `module` is a string, then import and return the
328 module with that name.
329 - If `module` is None, then return the calling module.
330 The calling module is assumed to be the module of
331 the stack frame at the given depth in the call stack.
332 """
333 if inspect.ismodule(module):
334 return module
335 elif isinstance(module, (str, unicode)):
336 return __import__(module, globals(), locals(), ["*"])
337 elif module is None:
338 return sys.modules[sys._getframe(depth).f_globals['__name__']]
339 else:
340 raise TypeError("Expected a module, string, or None")
Tim Peters7402f792001-10-02 03:53:41 +0000341
Edward Lopera1ef6112004-08-09 16:14:41 +0000342def _tag_msg(tag, msg, indent=' '):
Tim Peters8485b562004-08-04 18:46:34 +0000343 """
344 Return a string that displays a tag-and-message pair nicely,
345 keeping the tag and its message on the same line when that
Edward Lopera1ef6112004-08-09 16:14:41 +0000346 makes sense. If the message is displayed on separate lines,
347 then `indent` is added to the beginning of each line.
Tim Peters8485b562004-08-04 18:46:34 +0000348 """
Tim Peters8485b562004-08-04 18:46:34 +0000349 # If the message doesn't end in a newline, then add one.
350 if msg[-1:] != '\n':
351 msg += '\n'
352 # If the message is short enough, and contains no internal
353 # newlines, then display it on the same line as the tag.
354 # Otherwise, display the tag on its own line.
355 if (len(tag) + len(msg) < 75 and
356 msg.find('\n', 0, len(msg)-1) == -1):
357 return '%s: %s' % (tag, msg)
358 else:
Edward Lopera1ef6112004-08-09 16:14:41 +0000359 msg = '\n'.join([indent+l for l in msg[:-1].split('\n')])
360 return '%s:\n%s\n' % (tag, msg)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000361
Edward Loper8e4a34b2004-08-12 02:34:27 +0000362def _exception_traceback(exc_info):
363 """
364 Return a string containing a traceback message for the given
365 exc_info tuple (as returned by sys.exc_info()).
366 """
367 # Get a traceback message.
368 excout = StringIO()
369 exc_type, exc_val, exc_tb = exc_info
370 traceback.print_exception(exc_type, exc_val, exc_tb, file=excout)
371 return excout.getvalue()
372
Tim Peters8485b562004-08-04 18:46:34 +0000373# Override some StringIO methods.
374class _SpoofOut(StringIO):
375 def getvalue(self):
376 result = StringIO.getvalue(self)
377 # If anything at all was written, make sure there's a trailing
378 # newline. There's no way for the expected output to indicate
379 # that a trailing newline is missing.
380 if result and not result.endswith("\n"):
381 result += "\n"
382 # Prevent softspace from screwing up the next test case, in
383 # case they used print with a trailing comma in an example.
384 if hasattr(self, "softspace"):
385 del self.softspace
386 return result
Tim Peters8a7d2d52001-01-16 07:10:57 +0000387
Tim Peters8485b562004-08-04 18:46:34 +0000388 def truncate(self, size=None):
389 StringIO.truncate(self, size)
390 if hasattr(self, "softspace"):
391 del self.softspace
Tim Peters8a7d2d52001-01-16 07:10:57 +0000392
Tim Peters26b3ebb2004-08-19 08:10:08 +0000393# Worst-case linear-time ellipsis matching.
394def ellipsis_match(want, got):
Tim Petersdc5de3b2004-08-19 14:06:20 +0000395 """
396 Essentially the only subtle case:
397 >>> ellipsis_match('aa...aa', 'aaa')
398 False
399 """
Tim Peters26b3ebb2004-08-19 08:10:08 +0000400 if ELLIPSIS_MARKER not in want:
401 return want == got
402 # Remove \n from ...\n, else the newline will be required,
403 # and (for example) ... on a line by itself can't match
404 # nothing gracefully.
405 want = want.replace(ELLIPSIS_MARKER + '\n', ELLIPSIS_MARKER)
Tim Petersdc5de3b2004-08-19 14:06:20 +0000406
Tim Peters26b3ebb2004-08-19 08:10:08 +0000407 # Find "the real" strings.
408 ws = want.split(ELLIPSIS_MARKER)
409 assert len(ws) >= 2
Tim Peters26b3ebb2004-08-19 08:10:08 +0000410
Tim Petersdc5de3b2004-08-19 14:06:20 +0000411 # Deal with exact matches possibly needed at one or both ends.
412 startpos, endpos = 0, len(got)
413 w = ws[0]
414 if w: # starts with exact match
415 if got.startswith(w):
416 startpos = len(w)
417 del ws[0]
418 else:
419 return False
420 w = ws[-1]
421 if w: # ends with exact match
422 if got.endswith(w):
423 endpos -= len(w)
424 del ws[-1]
425 else:
426 return False
427
428 if startpos > endpos:
429 # Exact end matches required more characters than we have, as in
430 # ellipsis_match('aa...aa', 'aaa')
431 return False
432
433 # For the rest, we only need to find the leftmost non-overlapping
434 # match for each piece. If there's no overall match that way alone,
435 # there's no overall match period.
Tim Peters26b3ebb2004-08-19 08:10:08 +0000436 for w in ws:
437 # w may be '' at times, if there are consecutive ellipses, or
438 # due to an ellipsis at the start or end of `want`. That's OK.
Tim Petersdc5de3b2004-08-19 14:06:20 +0000439 # Search for an empty string succeeds, and doesn't change startpos.
440 startpos = got.find(w, startpos, endpos)
441 if startpos < 0:
Tim Peters26b3ebb2004-08-19 08:10:08 +0000442 return False
Tim Petersdc5de3b2004-08-19 14:06:20 +0000443 startpos += len(w)
Tim Peters26b3ebb2004-08-19 08:10:08 +0000444
Tim Petersdc5de3b2004-08-19 14:06:20 +0000445 return True
Tim Peters26b3ebb2004-08-19 08:10:08 +0000446
Tim Peters8485b562004-08-04 18:46:34 +0000447######################################################################
448## 2. Example & DocTest
449######################################################################
450## - An "example" is a <source, want> pair, where "source" is a
451## fragment of source code, and "want" is the expected output for
452## "source." The Example class also includes information about
453## where the example was extracted from.
454##
Edward Lopera1ef6112004-08-09 16:14:41 +0000455## - A "doctest" is a collection of examples, typically extracted from
456## a string (such as an object's docstring). The DocTest class also
457## includes information about where the string was extracted from.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000458
Tim Peters8485b562004-08-04 18:46:34 +0000459class Example:
460 """
461 A single doctest example, consisting of source code and expected
Edward Lopera1ef6112004-08-09 16:14:41 +0000462 output. `Example` defines the following attributes:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000463
Edward Loper74bca7a2004-08-12 02:27:44 +0000464 - source: A single Python statement, always ending with a newline.
Tim Petersbb431472004-08-09 03:51:46 +0000465 The constructor adds a newline if needed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000466
Edward Loper74bca7a2004-08-12 02:27:44 +0000467 - want: The expected output from running the source code (either
Tim Petersbb431472004-08-09 03:51:46 +0000468 from stdout, or a traceback in case of exception). `want` ends
469 with a newline unless it's empty, in which case it's an empty
470 string. The constructor adds a newline if needed.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000471
Edward Loper74bca7a2004-08-12 02:27:44 +0000472 - lineno: The line number within the DocTest string containing
Tim Peters8485b562004-08-04 18:46:34 +0000473 this Example where the Example begins. This line number is
474 zero-based, with respect to the beginning of the DocTest.
Edward Loper74bca7a2004-08-12 02:27:44 +0000475
476 - indent: The example's indentation in the DocTest string.
477 I.e., the number of space characters that preceed the
478 example's first prompt.
479
480 - options: A dictionary mapping from option flags to True or
481 False, which is used to override default options for this
482 example. Any option flags not contained in this dictionary
483 are left at their default value (as specified by the
484 DocTestRunner's optionflags). By default, no options are set.
Tim Peters8485b562004-08-04 18:46:34 +0000485 """
Edward Loper74bca7a2004-08-12 02:27:44 +0000486 def __init__(self, source, want, lineno, indent=0, options=None):
Tim Petersbb431472004-08-09 03:51:46 +0000487 # Normalize inputs.
488 if not source.endswith('\n'):
489 source += '\n'
490 if want and not want.endswith('\n'):
491 want += '\n'
Tim Peters8485b562004-08-04 18:46:34 +0000492 # Store properties.
493 self.source = source
494 self.want = want
495 self.lineno = lineno
Edward Loper74bca7a2004-08-12 02:27:44 +0000496 self.indent = indent
497 if options is None: options = {}
498 self.options = options
Tim Peters8a7d2d52001-01-16 07:10:57 +0000499
Tim Peters8485b562004-08-04 18:46:34 +0000500class DocTest:
501 """
502 A collection of doctest examples that should be run in a single
Edward Lopera1ef6112004-08-09 16:14:41 +0000503 namespace. Each `DocTest` defines the following attributes:
Tim Peters8a7d2d52001-01-16 07:10:57 +0000504
Tim Peters8485b562004-08-04 18:46:34 +0000505 - examples: the list of examples.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000506
Tim Peters8485b562004-08-04 18:46:34 +0000507 - globs: The namespace (aka globals) that the examples should
508 be run in.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000509
Tim Peters8485b562004-08-04 18:46:34 +0000510 - name: A name identifying the DocTest (typically, the name of
511 the object whose docstring this DocTest was extracted from).
Tim Peters8a7d2d52001-01-16 07:10:57 +0000512
Tim Peters8485b562004-08-04 18:46:34 +0000513 - filename: The name of the file that this DocTest was extracted
Edward Lopera1ef6112004-08-09 16:14:41 +0000514 from, or `None` if the filename is unknown.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000515
Tim Peters8485b562004-08-04 18:46:34 +0000516 - lineno: The line number within filename where this DocTest
Edward Lopera1ef6112004-08-09 16:14:41 +0000517 begins, or `None` if the line number is unavailable. This
518 line number is zero-based, with respect to the beginning of
519 the file.
520
521 - docstring: The string that the examples were extracted from,
522 or `None` if the string is unavailable.
Tim Peters8485b562004-08-04 18:46:34 +0000523 """
Edward Lopera1ef6112004-08-09 16:14:41 +0000524 def __init__(self, examples, globs, name, filename, lineno, docstring):
Tim Peters8485b562004-08-04 18:46:34 +0000525 """
Edward Lopera1ef6112004-08-09 16:14:41 +0000526 Create a new DocTest containing the given examples. The
527 DocTest's globals are initialized with a copy of `globs`.
Tim Peters8485b562004-08-04 18:46:34 +0000528 """
Edward Lopera1ef6112004-08-09 16:14:41 +0000529 assert not isinstance(examples, basestring), \
530 "DocTest no longer accepts str; use DocTestParser instead"
531 self.examples = examples
532 self.docstring = docstring
Tim Peters8485b562004-08-04 18:46:34 +0000533 self.globs = globs.copy()
Tim Peters8485b562004-08-04 18:46:34 +0000534 self.name = name
535 self.filename = filename
536 self.lineno = lineno
Tim Peters8485b562004-08-04 18:46:34 +0000537
538 def __repr__(self):
539 if len(self.examples) == 0:
540 examples = 'no examples'
541 elif len(self.examples) == 1:
542 examples = '1 example'
543 else:
544 examples = '%d examples' % len(self.examples)
545 return ('<DocTest %s from %s:%s (%s)>' %
546 (self.name, self.filename, self.lineno, examples))
547
548
549 # This lets us sort tests by name:
550 def __cmp__(self, other):
551 if not isinstance(other, DocTest):
552 return -1
553 return cmp((self.name, self.filename, self.lineno, id(self)),
554 (other.name, other.filename, other.lineno, id(other)))
555
556######################################################################
Edward Loperb7503ff2004-08-19 19:19:03 +0000557## 3. DocTestParser
Edward Loper7c748462004-08-09 02:06:06 +0000558######################################################################
559
Edward Lopera1ef6112004-08-09 16:14:41 +0000560class DocTestParser:
Edward Loper7c748462004-08-09 02:06:06 +0000561 """
Edward Lopera1ef6112004-08-09 16:14:41 +0000562 A class used to parse strings containing doctest examples.
Edward Loper7c748462004-08-09 02:06:06 +0000563 """
Edward Loper8e4a34b2004-08-12 02:34:27 +0000564 # This regular expression is used to find doctest examples in a
565 # string. It defines three groups: `source` is the source code
566 # (including leading indentation and prompts); `indent` is the
567 # indentation of the first (PS1) line of the source code; and
568 # `want` is the expected output (including leading indentation).
Edward Loper7c748462004-08-09 02:06:06 +0000569 _EXAMPLE_RE = re.compile(r'''
Tim Petersd40a92b2004-08-09 03:28:45 +0000570 # Source consists of a PS1 line followed by zero or more PS2 lines.
571 (?P<source>
572 (?:^(?P<indent> [ ]*) >>> .*) # PS1 line
573 (?:\n [ ]* \.\.\. .*)*) # PS2 lines
574 \n?
575 # Want consists of any non-blank lines that do not start with PS1.
576 (?P<want> (?:(?![ ]*$) # Not a blank line
577 (?![ ]*>>>) # Not a line starting with PS1
578 .*$\n? # But any other line
579 )*)
580 ''', re.MULTILINE | re.VERBOSE)
Edward Loper8e4a34b2004-08-12 02:34:27 +0000581
Tim Peters7ea48dd2004-08-13 01:52:59 +0000582 # A callable returning a true value iff its argument is a blank line
583 # or contains a single comment.
Edward Loper8e4a34b2004-08-12 02:34:27 +0000584 _IS_BLANK_OR_COMMENT = re.compile(r'^[ ]*(#.*)?$').match
Edward Loper7c748462004-08-09 02:06:06 +0000585
Edward Lopera1ef6112004-08-09 16:14:41 +0000586 def get_doctest(self, string, globs, name, filename, lineno):
Edward Loper7c748462004-08-09 02:06:06 +0000587 """
Edward Lopera1ef6112004-08-09 16:14:41 +0000588 Extract all doctest examples from the given string, and
589 collect them into a `DocTest` object.
590
591 `globs`, `name`, `filename`, and `lineno` are attributes for
592 the new `DocTest` object. See the documentation for `DocTest`
593 for more information.
594 """
595 return DocTest(self.get_examples(string, name), globs,
596 name, filename, lineno, string)
597
598 def get_examples(self, string, name='<string>'):
599 """
600 Extract all doctest examples from the given string, and return
601 them as a list of `Example` objects. Line numbers are
602 0-based, because it's most common in doctests that nothing
603 interesting appears on the same line as opening triple-quote,
604 and so the first interesting line is called \"line 1\" then.
605
606 The optional argument `name` is a name identifying this
607 string, and is only used for error messages.
Edward Loper7c748462004-08-09 02:06:06 +0000608
609 >>> text = '''
610 ... >>> x, y = 2, 3 # no output expected
611 ... >>> if 1:
612 ... ... print x
613 ... ... print y
614 ... 2
615 ... 3
616 ...
617 ... Some text.
618 ... >>> x+y
619 ... 5
620 ... '''
Edward Lopera1ef6112004-08-09 16:14:41 +0000621 >>> for x in DocTestParser().get_examples(text):
Edward Loper78b58f32004-08-09 02:56:02 +0000622 ... print (x.source, x.want, x.lineno)
Tim Petersbb431472004-08-09 03:51:46 +0000623 ('x, y = 2, 3 # no output expected\\n', '', 1)
Edward Loper7c748462004-08-09 02:06:06 +0000624 ('if 1:\\n print x\\n print y\\n', '2\\n3\\n', 2)
Tim Petersbb431472004-08-09 03:51:46 +0000625 ('x+y\\n', '5\\n', 9)
Edward Loper7c748462004-08-09 02:06:06 +0000626 """
627 examples = []
628 charno, lineno = 0, 0
629 # Find all doctest examples in the string:
Edward Lopera1ef6112004-08-09 16:14:41 +0000630 for m in self._EXAMPLE_RE.finditer(string.expandtabs()):
Edward Loper7c748462004-08-09 02:06:06 +0000631 # Update lineno (lines before this example)
Edward Lopera1ef6112004-08-09 16:14:41 +0000632 lineno += string.count('\n', charno, m.start())
Edward Loper7c748462004-08-09 02:06:06 +0000633 # Extract source/want from the regexp match.
Edward Lopera1ef6112004-08-09 16:14:41 +0000634 (source, want) = self._parse_example(m, name, lineno)
Edward Loper74bca7a2004-08-12 02:27:44 +0000635 # Extract extra options from the source.
636 options = self._find_options(source, name, lineno)
Edward Loper74bca7a2004-08-12 02:27:44 +0000637 # Create an Example, and add it to the list.
Edward Loperb51b2342004-08-17 16:37:12 +0000638 if not self._IS_BLANK_OR_COMMENT(source):
639 examples.append( Example(source, want, lineno,
640 len(m.group('indent')), options) )
Edward Loper7c748462004-08-09 02:06:06 +0000641 # Update lineno (lines inside this example)
Edward Lopera1ef6112004-08-09 16:14:41 +0000642 lineno += string.count('\n', m.start(), m.end())
Edward Loper7c748462004-08-09 02:06:06 +0000643 # Update charno.
644 charno = m.end()
645 return examples
646
Edward Lopera1ef6112004-08-09 16:14:41 +0000647 def get_program(self, string, name="<string>"):
Edward Loper7c748462004-08-09 02:06:06 +0000648 """
Edward Lopera1ef6112004-08-09 16:14:41 +0000649 Return an executable program from the given string, as a string.
Edward Loper7c748462004-08-09 02:06:06 +0000650
651 The format of this isn't rigidly defined. In general, doctest
652 examples become the executable statements in the result, and
653 their expected outputs become comments, preceded by an \"#Expected:\"
654 comment. Everything else (text, comments, everything not part of
655 a doctest test) is also placed in comments.
656
Edward Lopera1ef6112004-08-09 16:14:41 +0000657 The optional argument `name` is a name identifying this
658 string, and is only used for error messages.
659
Edward Loper7c748462004-08-09 02:06:06 +0000660 >>> text = '''
661 ... >>> x, y = 2, 3 # no output expected
662 ... >>> if 1:
663 ... ... print x
664 ... ... print y
665 ... 2
666 ... 3
667 ...
668 ... Some text.
669 ... >>> x+y
670 ... 5
671 ... '''
Edward Lopera1ef6112004-08-09 16:14:41 +0000672 >>> print DocTestParser().get_program(text)
Edward Loper7c748462004-08-09 02:06:06 +0000673 x, y = 2, 3 # no output expected
674 if 1:
675 print x
676 print y
677 # Expected:
Edward Lopera5db6002004-08-12 02:41:30 +0000678 ## 2
679 ## 3
Edward Loper7c748462004-08-09 02:06:06 +0000680 #
Edward Lopera5db6002004-08-12 02:41:30 +0000681 # Some text.
Edward Loper7c748462004-08-09 02:06:06 +0000682 x+y
683 # Expected:
Edward Lopera5db6002004-08-12 02:41:30 +0000684 ## 5
Edward Loper7c748462004-08-09 02:06:06 +0000685 """
Edward Lopera5db6002004-08-12 02:41:30 +0000686 string = string.expandtabs()
687 # If all lines begin with the same indentation, then strip it.
688 min_indent = self._min_indent(string)
689 if min_indent > 0:
690 string = '\n'.join([l[min_indent:] for l in string.split('\n')])
691
Edward Loper7c748462004-08-09 02:06:06 +0000692 output = []
693 charnum, lineno = 0, 0
694 # Find all doctest examples in the string:
Edward Lopera1ef6112004-08-09 16:14:41 +0000695 for m in self._EXAMPLE_RE.finditer(string.expandtabs()):
Edward Loper7c748462004-08-09 02:06:06 +0000696 # Add any text before this example, as a comment.
697 if m.start() > charnum:
Edward Lopera1ef6112004-08-09 16:14:41 +0000698 lines = string[charnum:m.start()-1].split('\n')
Edward Loper7c748462004-08-09 02:06:06 +0000699 output.extend([self._comment_line(l) for l in lines])
700 lineno += len(lines)
701
702 # Extract source/want from the regexp match.
Edward Loper74bca7a2004-08-12 02:27:44 +0000703 (source, want) = self._parse_example(m, name, lineno)
Edward Loper7c748462004-08-09 02:06:06 +0000704 # Display the source
705 output.append(source)
706 # Display the expected output, if any
707 if want:
708 output.append('# Expected:')
Edward Lopera5db6002004-08-12 02:41:30 +0000709 output.extend(['## '+l for l in want.split('\n')])
Edward Loper7c748462004-08-09 02:06:06 +0000710
711 # Update the line number & char number.
Edward Lopera1ef6112004-08-09 16:14:41 +0000712 lineno += string.count('\n', m.start(), m.end())
Edward Loper7c748462004-08-09 02:06:06 +0000713 charnum = m.end()
714 # Add any remaining text, as comments.
715 output.extend([self._comment_line(l)
Edward Lopera1ef6112004-08-09 16:14:41 +0000716 for l in string[charnum:].split('\n')])
Edward Loper7c748462004-08-09 02:06:06 +0000717 # Trim junk on both ends.
718 while output and output[-1] == '#':
719 output.pop()
720 while output and output[0] == '#':
721 output.pop(0)
722 # Combine the output, and return it.
723 return '\n'.join(output)
724
Edward Loper74bca7a2004-08-12 02:27:44 +0000725 def _parse_example(self, m, name, lineno):
726 """
727 Given a regular expression match from `_EXAMPLE_RE` (`m`),
728 return a pair `(source, want)`, where `source` is the matched
729 example's source code (with prompts and indentation stripped);
730 and `want` is the example's expected output (with indentation
731 stripped).
732
733 `name` is the string's name, and `lineno` is the line number
734 where the example starts; both are used for error messages.
735 """
Edward Loper7c748462004-08-09 02:06:06 +0000736 # Get the example's indentation level.
737 indent = len(m.group('indent'))
738
739 # Divide source into lines; check that they're properly
740 # indented; and then strip their indentation & prompts.
741 source_lines = m.group('source').split('\n')
Edward Lopera1ef6112004-08-09 16:14:41 +0000742 self._check_prompt_blank(source_lines, indent, name, lineno)
743 self._check_prefix(source_lines[1:], ' '*indent+'.', name, lineno)
Edward Loper7c748462004-08-09 02:06:06 +0000744 source = '\n'.join([sl[indent+4:] for sl in source_lines])
Edward Loper7c748462004-08-09 02:06:06 +0000745
746 # Divide want into lines; check that it's properly
747 # indented; and then strip the indentation.
748 want_lines = m.group('want').rstrip().split('\n')
Edward Lopera1ef6112004-08-09 16:14:41 +0000749 self._check_prefix(want_lines, ' '*indent, name,
Edward Loper7c748462004-08-09 02:06:06 +0000750 lineno+len(source_lines))
751 want = '\n'.join([wl[indent:] for wl in want_lines])
Edward Loper7c748462004-08-09 02:06:06 +0000752
753 return source, want
754
Edward Loper74bca7a2004-08-12 02:27:44 +0000755 # This regular expression looks for option directives in the
756 # source code of an example. Option directives are comments
757 # starting with "doctest:". Warning: this may give false
758 # positives for string-literals that contain the string
759 # "#doctest:". Eliminating these false positives would require
760 # actually parsing the string; but we limit them by ignoring any
761 # line containing "#doctest:" that is *followed* by a quote mark.
762 _OPTION_DIRECTIVE_RE = re.compile(r'#\s*doctest:\s*([^\n\'"]*)$',
763 re.MULTILINE)
764
765 def _find_options(self, source, name, lineno):
766 """
767 Return a dictionary containing option overrides extracted from
768 option directives in the given source string.
769
770 `name` is the string's name, and `lineno` is the line number
771 where the example starts; both are used for error messages.
772 """
773 options = {}
774 # (note: with the current regexp, this will match at most once:)
775 for m in self._OPTION_DIRECTIVE_RE.finditer(source):
776 option_strings = m.group(1).replace(',', ' ').split()
777 for option in option_strings:
778 if (option[0] not in '+-' or
779 option[1:] not in OPTIONFLAGS_BY_NAME):
780 raise ValueError('line %r of the doctest for %s '
781 'has an invalid option: %r' %
782 (lineno+1, name, option))
783 flag = OPTIONFLAGS_BY_NAME[option[1:]]
784 options[flag] = (option[0] == '+')
785 if options and self._IS_BLANK_OR_COMMENT(source):
786 raise ValueError('line %r of the doctest for %s has an option '
787 'directive on a line with no example: %r' %
788 (lineno, name, source))
789 return options
790
Edward Lopera5db6002004-08-12 02:41:30 +0000791 # This regular expression finds the indentation of every non-blank
792 # line in a string.
793 _INDENT_RE = re.compile('^([ ]+)(?=\S)', re.MULTILINE)
794
795 def _min_indent(self, s):
796 "Return the minimum indentation of any non-blank line in `s`"
797 return min([len(indent) for indent in self._INDENT_RE.findall(s)])
798
Edward Loper7c748462004-08-09 02:06:06 +0000799 def _comment_line(self, line):
Edward Loper74bca7a2004-08-12 02:27:44 +0000800 "Return a commented form of the given line"
Edward Loper7c748462004-08-09 02:06:06 +0000801 line = line.rstrip()
Tim Petersdd0e4752004-08-09 03:31:56 +0000802 if line:
Edward Lopera5db6002004-08-12 02:41:30 +0000803 return '# '+line
Tim Petersdd0e4752004-08-09 03:31:56 +0000804 else:
805 return '#'
Edward Loper7c748462004-08-09 02:06:06 +0000806
Edward Lopera1ef6112004-08-09 16:14:41 +0000807 def _check_prompt_blank(self, lines, indent, name, lineno):
Edward Loper74bca7a2004-08-12 02:27:44 +0000808 """
809 Given the lines of a source string (including prompts and
810 leading indentation), check to make sure that every prompt is
811 followed by a space character. If any line is not followed by
812 a space character, then raise ValueError.
813 """
Edward Loper7c748462004-08-09 02:06:06 +0000814 for i, line in enumerate(lines):
815 if len(line) >= indent+4 and line[indent+3] != ' ':
816 raise ValueError('line %r of the docstring for %s '
817 'lacks blank after %s: %r' %
Edward Lopera1ef6112004-08-09 16:14:41 +0000818 (lineno+i+1, name,
Edward Loper7c748462004-08-09 02:06:06 +0000819 line[indent:indent+3], line))
820
Edward Lopera1ef6112004-08-09 16:14:41 +0000821 def _check_prefix(self, lines, prefix, name, lineno):
Edward Loper74bca7a2004-08-12 02:27:44 +0000822 """
823 Check that every line in the given list starts with the given
824 prefix; if any line does not, then raise a ValueError.
825 """
Edward Loper7c748462004-08-09 02:06:06 +0000826 for i, line in enumerate(lines):
827 if line and not line.startswith(prefix):
828 raise ValueError('line %r of the docstring for %s has '
829 'inconsistent leading whitespace: %r' %
Edward Lopera1ef6112004-08-09 16:14:41 +0000830 (lineno+i+1, name, line))
Edward Loper7c748462004-08-09 02:06:06 +0000831
832
833######################################################################
834## 4. DocTest Finder
Tim Peters8485b562004-08-04 18:46:34 +0000835######################################################################
836
837class DocTestFinder:
838 """
839 A class used to extract the DocTests that are relevant to a given
840 object, from its docstring and the docstrings of its contained
841 objects. Doctests can currently be extracted from the following
842 object types: modules, functions, classes, methods, staticmethods,
843 classmethods, and properties.
Tim Peters8485b562004-08-04 18:46:34 +0000844 """
845
Edward Lopera1ef6112004-08-09 16:14:41 +0000846 def __init__(self, verbose=False, parser=DocTestParser(),
Tim Petersf727c6c2004-08-08 01:48:59 +0000847 recurse=True, _namefilter=None):
Tim Peters8485b562004-08-04 18:46:34 +0000848 """
849 Create a new doctest finder.
850
Edward Lopera1ef6112004-08-09 16:14:41 +0000851 The optional argument `parser` specifies a class or
Tim Peters19397e52004-08-06 22:02:59 +0000852 function that should be used to create new DocTest objects (or
Tim Peters161c9632004-08-08 03:38:33 +0000853 objects that implement the same interface as DocTest). The
Tim Peters19397e52004-08-06 22:02:59 +0000854 signature for this factory function should match the signature
855 of the DocTest constructor.
856
Tim Peters8485b562004-08-04 18:46:34 +0000857 If the optional argument `recurse` is false, then `find` will
858 only examine the given object, and not any contained objects.
859 """
Edward Lopera1ef6112004-08-09 16:14:41 +0000860 self._parser = parser
Tim Peters8485b562004-08-04 18:46:34 +0000861 self._verbose = verbose
Tim Peters8485b562004-08-04 18:46:34 +0000862 self._recurse = recurse
Tim Petersf727c6c2004-08-08 01:48:59 +0000863 # _namefilter is undocumented, and exists only for temporary backward-
864 # compatibility support of testmod's deprecated isprivate mess.
865 self._namefilter = _namefilter
Tim Peters8485b562004-08-04 18:46:34 +0000866
867 def find(self, obj, name=None, module=None, globs=None,
Tim Petersf3f57472004-08-08 06:11:48 +0000868 extraglobs=None):
Tim Peters8485b562004-08-04 18:46:34 +0000869 """
870 Return a list of the DocTests that are defined by the given
871 object's docstring, or by any of its contained objects'
872 docstrings.
873
874 The optional parameter `module` is the module that contains
Tim Petersf3f57472004-08-08 06:11:48 +0000875 the given object. If the module is not specified or is None, then
876 the test finder will attempt to automatically determine the
Tim Peters8485b562004-08-04 18:46:34 +0000877 correct module. The object's module is used:
878
879 - As a default namespace, if `globs` is not specified.
880 - To prevent the DocTestFinder from extracting DocTests
Tim Petersf3f57472004-08-08 06:11:48 +0000881 from objects that are imported from other modules.
Tim Peters8485b562004-08-04 18:46:34 +0000882 - To find the name of the file containing the object.
883 - To help find the line number of the object within its
884 file.
885
Tim Petersf3f57472004-08-08 06:11:48 +0000886 Contained objects whose module does not match `module` are ignored.
887
888 If `module` is False, no attempt to find the module will be made.
889 This is obscure, of use mostly in tests: if `module` is False, or
890 is None but cannot be found automatically, then all objects are
891 considered to belong to the (non-existent) module, so all contained
892 objects will (recursively) be searched for doctests.
893
Tim Peters8485b562004-08-04 18:46:34 +0000894 The globals for each DocTest is formed by combining `globs`
895 and `extraglobs` (bindings in `extraglobs` override bindings
896 in `globs`). A new copy of the globals dictionary is created
897 for each DocTest. If `globs` is not specified, then it
898 defaults to the module's `__dict__`, if specified, or {}
899 otherwise. If `extraglobs` is not specified, then it defaults
900 to {}.
901
Tim Peters8485b562004-08-04 18:46:34 +0000902 """
903 # If name was not specified, then extract it from the object.
904 if name is None:
905 name = getattr(obj, '__name__', None)
906 if name is None:
907 raise ValueError("DocTestFinder.find: name must be given "
908 "when obj.__name__ doesn't exist: %r" %
909 (type(obj),))
910
911 # Find the module that contains the given object (if obj is
912 # a module, then module=obj.). Note: this may fail, in which
913 # case module will be None.
Tim Petersf3f57472004-08-08 06:11:48 +0000914 if module is False:
915 module = None
916 elif module is None:
Tim Peters8485b562004-08-04 18:46:34 +0000917 module = inspect.getmodule(obj)
918
919 # Read the module's source code. This is used by
920 # DocTestFinder._find_lineno to find the line number for a
921 # given object's docstring.
922 try:
923 file = inspect.getsourcefile(obj) or inspect.getfile(obj)
924 source_lines = linecache.getlines(file)
925 if not source_lines:
926 source_lines = None
927 except TypeError:
928 source_lines = None
929
930 # Initialize globals, and merge in extraglobs.
Tim Peters8a7d2d52001-01-16 07:10:57 +0000931 if globs is None:
Tim Peters8485b562004-08-04 18:46:34 +0000932 if module is None:
933 globs = {}
934 else:
935 globs = module.__dict__.copy()
936 else:
937 globs = globs.copy()
938 if extraglobs is not None:
939 globs.update(extraglobs)
Tim Peters8a7d2d52001-01-16 07:10:57 +0000940
Tim Peters8485b562004-08-04 18:46:34 +0000941 # Recursively expore `obj`, extracting DocTests.
942 tests = []
Tim Petersf3f57472004-08-08 06:11:48 +0000943 self._find(tests, obj, name, module, source_lines, globs, {})
Tim Peters8485b562004-08-04 18:46:34 +0000944 return tests
945
946 def _filter(self, obj, prefix, base):
947 """
948 Return true if the given object should not be examined.
949 """
Tim Petersf727c6c2004-08-08 01:48:59 +0000950 return (self._namefilter is not None and
951 self._namefilter(prefix, base))
Tim Peters8485b562004-08-04 18:46:34 +0000952
953 def _from_module(self, module, object):
954 """
955 Return true if the given object is defined in the given
956 module.
957 """
958 if module is None:
959 return True
960 elif inspect.isfunction(object):
961 return module.__dict__ is object.func_globals
962 elif inspect.isclass(object):
963 return module.__name__ == object.__module__
964 elif inspect.getmodule(object) is not None:
965 return module is inspect.getmodule(object)
966 elif hasattr(object, '__module__'):
967 return module.__name__ == object.__module__
968 elif isinstance(object, property):
969 return True # [XX] no way not be sure.
970 else:
971 raise ValueError("object must be a class or function")
972
Tim Petersf3f57472004-08-08 06:11:48 +0000973 def _find(self, tests, obj, name, module, source_lines, globs, seen):
Tim Peters8485b562004-08-04 18:46:34 +0000974 """
975 Find tests for the given object and any contained objects, and
976 add them to `tests`.
977 """
978 if self._verbose:
979 print 'Finding tests in %s' % name
980
981 # If we've already processed this object, then ignore it.
982 if id(obj) in seen:
983 return
984 seen[id(obj)] = 1
985
986 # Find a test for this object, and add it to the list of tests.
987 test = self._get_test(obj, name, module, globs, source_lines)
988 if test is not None:
989 tests.append(test)
990
991 # Look for tests in a module's contained objects.
992 if inspect.ismodule(obj) and self._recurse:
993 for valname, val in obj.__dict__.items():
994 # Check if this contained object should be ignored.
995 if self._filter(val, name, valname):
996 continue
997 valname = '%s.%s' % (name, valname)
998 # Recurse to functions & classes.
999 if ((inspect.isfunction(val) or inspect.isclass(val)) and
Tim Petersf3f57472004-08-08 06:11:48 +00001000 self._from_module(module, val)):
Tim Peters8485b562004-08-04 18:46:34 +00001001 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +00001002 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +00001003
1004 # Look for tests in a module's __test__ dictionary.
1005 if inspect.ismodule(obj) and self._recurse:
1006 for valname, val in getattr(obj, '__test__', {}).items():
1007 if not isinstance(valname, basestring):
1008 raise ValueError("DocTestFinder.find: __test__ keys "
1009 "must be strings: %r" %
1010 (type(valname),))
1011 if not (inspect.isfunction(val) or inspect.isclass(val) or
1012 inspect.ismethod(val) or inspect.ismodule(val) or
1013 isinstance(val, basestring)):
1014 raise ValueError("DocTestFinder.find: __test__ values "
1015 "must be strings, functions, methods, "
1016 "classes, or modules: %r" %
1017 (type(val),))
1018 valname = '%s.%s' % (name, valname)
1019 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +00001020 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +00001021
1022 # Look for tests in a class's contained objects.
1023 if inspect.isclass(obj) and self._recurse:
1024 for valname, val in obj.__dict__.items():
1025 # Check if this contained object should be ignored.
1026 if self._filter(val, name, valname):
1027 continue
1028 # Special handling for staticmethod/classmethod.
1029 if isinstance(val, staticmethod):
1030 val = getattr(obj, valname)
1031 if isinstance(val, classmethod):
1032 val = getattr(obj, valname).im_func
1033
1034 # Recurse to methods, properties, and nested classes.
1035 if ((inspect.isfunction(val) or inspect.isclass(val) or
Tim Petersf3f57472004-08-08 06:11:48 +00001036 isinstance(val, property)) and
1037 self._from_module(module, val)):
Tim Peters8485b562004-08-04 18:46:34 +00001038 valname = '%s.%s' % (name, valname)
1039 self._find(tests, val, valname, module, source_lines,
Tim Petersf3f57472004-08-08 06:11:48 +00001040 globs, seen)
Tim Peters8485b562004-08-04 18:46:34 +00001041
1042 def _get_test(self, obj, name, module, globs, source_lines):
1043 """
1044 Return a DocTest for the given object, if it defines a docstring;
1045 otherwise, return None.
1046 """
1047 # Extract the object's docstring. If it doesn't have one,
1048 # then return None (no test for this object).
1049 if isinstance(obj, basestring):
1050 docstring = obj
1051 else:
1052 try:
1053 if obj.__doc__ is None:
1054 return None
1055 docstring = str(obj.__doc__)
1056 except (TypeError, AttributeError):
1057 return None
1058
1059 # Don't bother if the docstring is empty.
1060 if not docstring:
1061 return None
1062
1063 # Find the docstring's location in the file.
1064 lineno = self._find_lineno(obj, source_lines)
1065
1066 # Return a DocTest for this object.
1067 if module is None:
1068 filename = None
1069 else:
1070 filename = getattr(module, '__file__', module.__name__)
Edward Lopera1ef6112004-08-09 16:14:41 +00001071 return self._parser.get_doctest(docstring, globs, name,
1072 filename, lineno)
Tim Peters8485b562004-08-04 18:46:34 +00001073
1074 def _find_lineno(self, obj, source_lines):
1075 """
1076 Return a line number of the given object's docstring. Note:
1077 this method assumes that the object has a docstring.
1078 """
1079 lineno = None
1080
1081 # Find the line number for modules.
1082 if inspect.ismodule(obj):
1083 lineno = 0
1084
1085 # Find the line number for classes.
1086 # Note: this could be fooled if a class is defined multiple
1087 # times in a single file.
1088 if inspect.isclass(obj):
1089 if source_lines is None:
1090 return None
1091 pat = re.compile(r'^\s*class\s*%s\b' %
1092 getattr(obj, '__name__', '-'))
1093 for i, line in enumerate(source_lines):
1094 if pat.match(line):
1095 lineno = i
1096 break
1097
1098 # Find the line number for functions & methods.
1099 if inspect.ismethod(obj): obj = obj.im_func
1100 if inspect.isfunction(obj): obj = obj.func_code
1101 if inspect.istraceback(obj): obj = obj.tb_frame
1102 if inspect.isframe(obj): obj = obj.f_code
1103 if inspect.iscode(obj):
1104 lineno = getattr(obj, 'co_firstlineno', None)-1
1105
1106 # Find the line number where the docstring starts. Assume
1107 # that it's the first line that begins with a quote mark.
1108 # Note: this could be fooled by a multiline function
1109 # signature, where a continuation line begins with a quote
1110 # mark.
1111 if lineno is not None:
1112 if source_lines is None:
1113 return lineno+1
1114 pat = re.compile('(^|.*:)\s*\w*("|\')')
1115 for lineno in range(lineno, len(source_lines)):
1116 if pat.match(source_lines[lineno]):
1117 return lineno
1118
1119 # We couldn't find the line number.
1120 return None
1121
1122######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001123## 5. DocTest Runner
Tim Peters8485b562004-08-04 18:46:34 +00001124######################################################################
1125
Tim Peters8485b562004-08-04 18:46:34 +00001126class DocTestRunner:
1127 """
1128 A class used to run DocTest test cases, and accumulate statistics.
1129 The `run` method is used to process a single DocTest case. It
1130 returns a tuple `(f, t)`, where `t` is the number of test cases
1131 tried, and `f` is the number of test cases that failed.
1132
1133 >>> tests = DocTestFinder().find(_TestClass)
1134 >>> runner = DocTestRunner(verbose=False)
1135 >>> for test in tests:
1136 ... print runner.run(test)
1137 (0, 2)
1138 (0, 1)
1139 (0, 2)
1140 (0, 2)
1141
1142 The `summarize` method prints a summary of all the test cases that
1143 have been run by the runner, and returns an aggregated `(f, t)`
1144 tuple:
1145
1146 >>> runner.summarize(verbose=1)
1147 4 items passed all tests:
1148 2 tests in _TestClass
1149 2 tests in _TestClass.__init__
1150 2 tests in _TestClass.get
1151 1 tests in _TestClass.square
1152 7 tests in 4 items.
1153 7 passed and 0 failed.
1154 Test passed.
1155 (0, 7)
1156
1157 The aggregated number of tried examples and failed examples is
1158 also available via the `tries` and `failures` attributes:
1159
1160 >>> runner.tries
1161 7
1162 >>> runner.failures
1163 0
1164
1165 The comparison between expected outputs and actual outputs is done
Edward Loper34fcb142004-08-09 02:45:41 +00001166 by an `OutputChecker`. This comparison may be customized with a
1167 number of option flags; see the documentation for `testmod` for
1168 more information. If the option flags are insufficient, then the
1169 comparison may also be customized by passing a subclass of
1170 `OutputChecker` to the constructor.
Tim Peters8485b562004-08-04 18:46:34 +00001171
1172 The test runner's display output can be controlled in two ways.
1173 First, an output function (`out) can be passed to
1174 `TestRunner.run`; this function will be called with strings that
1175 should be displayed. It defaults to `sys.stdout.write`. If
1176 capturing the output is not sufficient, then the display output
1177 can be also customized by subclassing DocTestRunner, and
1178 overriding the methods `report_start`, `report_success`,
1179 `report_unexpected_exception`, and `report_failure`.
1180 """
1181 # This divider string is used to separate failure messages, and to
1182 # separate sections of the summary.
1183 DIVIDER = "*" * 70
1184
Edward Loper34fcb142004-08-09 02:45:41 +00001185 def __init__(self, checker=None, verbose=None, optionflags=0):
Tim Peters8485b562004-08-04 18:46:34 +00001186 """
1187 Create a new test runner.
1188
Edward Loper34fcb142004-08-09 02:45:41 +00001189 Optional keyword arg `checker` is the `OutputChecker` that
1190 should be used to compare the expected outputs and actual
1191 outputs of doctest examples.
1192
Tim Peters8485b562004-08-04 18:46:34 +00001193 Optional keyword arg 'verbose' prints lots of stuff if true,
1194 only failures if false; by default, it's true iff '-v' is in
1195 sys.argv.
1196
1197 Optional argument `optionflags` can be used to control how the
1198 test runner compares expected output to actual output, and how
1199 it displays failures. See the documentation for `testmod` for
1200 more information.
1201 """
Edward Loper34fcb142004-08-09 02:45:41 +00001202 self._checker = checker or OutputChecker()
Tim Peters8a7d2d52001-01-16 07:10:57 +00001203 if verbose is None:
Tim Peters8485b562004-08-04 18:46:34 +00001204 verbose = '-v' in sys.argv
1205 self._verbose = verbose
Tim Peters6ebe61f2003-06-27 20:48:05 +00001206 self.optionflags = optionflags
1207
Tim Peters8485b562004-08-04 18:46:34 +00001208 # Keep track of the examples we've run.
1209 self.tries = 0
1210 self.failures = 0
1211 self._name2ft = {}
Tim Peters8a7d2d52001-01-16 07:10:57 +00001212
Tim Peters8485b562004-08-04 18:46:34 +00001213 # Create a fake output target for capturing doctest output.
1214 self._fakeout = _SpoofOut()
Tim Peters4fd9e2f2001-08-18 00:05:50 +00001215
Tim Peters8485b562004-08-04 18:46:34 +00001216 #/////////////////////////////////////////////////////////////////
Tim Peters8485b562004-08-04 18:46:34 +00001217 # Reporting methods
1218 #/////////////////////////////////////////////////////////////////
Tim Peters17111f32001-10-03 04:08:26 +00001219
Tim Peters8485b562004-08-04 18:46:34 +00001220 def report_start(self, out, test, example):
Tim Peters8a7d2d52001-01-16 07:10:57 +00001221 """
Tim Peters8485b562004-08-04 18:46:34 +00001222 Report that the test runner is about to process the given
1223 example. (Only displays a message if verbose=True)
1224 """
1225 if self._verbose:
1226 out(_tag_msg("Trying", example.source) +
1227 _tag_msg("Expecting", example.want or "nothing"))
Tim Peters8a7d2d52001-01-16 07:10:57 +00001228
Tim Peters8485b562004-08-04 18:46:34 +00001229 def report_success(self, out, test, example, got):
1230 """
1231 Report that the given example ran successfully. (Only
1232 displays a message if verbose=True)
1233 """
1234 if self._verbose:
1235 out("ok\n")
Tim Peters8a7d2d52001-01-16 07:10:57 +00001236
Tim Peters8485b562004-08-04 18:46:34 +00001237 def report_failure(self, out, test, example, got):
1238 """
1239 Report that the given example failed.
1240 """
1241 # Print an error message.
Edward Loper8e4a34b2004-08-12 02:34:27 +00001242 out(self._failure_header(test, example) +
Edward Loper34fcb142004-08-09 02:45:41 +00001243 self._checker.output_difference(example.want, got,
1244 self.optionflags))
Tim Peters7402f792001-10-02 03:53:41 +00001245
Tim Peters8485b562004-08-04 18:46:34 +00001246 def report_unexpected_exception(self, out, test, example, exc_info):
1247 """
1248 Report that the given example raised an unexpected exception.
1249 """
Edward Loper8e4a34b2004-08-12 02:34:27 +00001250 out(self._failure_header(test, example) +
1251 _tag_msg("Exception raised", _exception_traceback(exc_info)))
Tim Peters7402f792001-10-02 03:53:41 +00001252
Edward Loper8e4a34b2004-08-12 02:34:27 +00001253 def _failure_header(self, test, example):
Tim Peters8485b562004-08-04 18:46:34 +00001254 s = (self.DIVIDER + "\n" +
1255 _tag_msg("Failure in example", example.source))
1256 if test.filename is None:
1257 # [XX] I'm not putting +1 here, to give the same output
1258 # as the old version. But I think it *should* go here.
1259 return s + ("from line #%s of %s\n" %
1260 (example.lineno, test.name))
1261 elif test.lineno is None:
1262 return s + ("from line #%s of %s in %s\n" %
1263 (example.lineno+1, test.name, test.filename))
1264 else:
1265 lineno = test.lineno+example.lineno+1
1266 return s + ("from line #%s of %s (%s)\n" %
1267 (lineno, test.filename, test.name))
Tim Peters7402f792001-10-02 03:53:41 +00001268
Tim Peters8485b562004-08-04 18:46:34 +00001269 #/////////////////////////////////////////////////////////////////
1270 # DocTest Running
1271 #/////////////////////////////////////////////////////////////////
Tim Peters7402f792001-10-02 03:53:41 +00001272
Tim Peters8485b562004-08-04 18:46:34 +00001273 # A regular expression for handling `want` strings that contain
Tim Peters41a65ea2004-08-13 03:55:05 +00001274 # expected exceptions. It divides `want` into three pieces:
1275 # - the pre-exception output (`want`)
1276 # - the traceback header line (`hdr`)
1277 # - the exception message (`msg`), as generated by
1278 # traceback.format_exception_only()
1279 # `msg` may have multiple lines. We assume/require that the
1280 # exception message is the first non-indented line starting with a word
1281 # character following the traceback header line.
1282 _EXCEPTION_RE = re.compile(r"""
1283 (?P<want> .*?) # suck up everything until traceback header
1284 # Grab the traceback header. Different versions of Python have
1285 # said different things on the first traceback line.
1286 ^(?P<hdr> Traceback\ \(
1287 (?: most\ recent\ call\ last
1288 | innermost\ last
1289 ) \) :
1290 )
1291 \s* $ # toss trailing whitespace on traceback header
1292 .*? # don't blink: absorb stuff until a line *starts* with \w
1293 ^ (?P<msg> \w+ .*)
1294 """, re.VERBOSE | re.MULTILINE | re.DOTALL)
Tim Peters7402f792001-10-02 03:53:41 +00001295
Tim Peters8485b562004-08-04 18:46:34 +00001296 def __run(self, test, compileflags, out):
Tim Peters8a7d2d52001-01-16 07:10:57 +00001297 """
Tim Peters8485b562004-08-04 18:46:34 +00001298 Run the examples in `test`. Write the outcome of each example
1299 with one of the `DocTestRunner.report_*` methods, using the
1300 writer function `out`. `compileflags` is the set of compiler
1301 flags that should be used to execute examples. Return a tuple
1302 `(f, t)`, where `t` is the number of examples tried, and `f`
1303 is the number of examples that failed. The examples are run
1304 in the namespace `test.globs`.
1305 """
1306 # Keep track of the number of failures and tries.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001307 failures = tries = 0
Tim Peters8485b562004-08-04 18:46:34 +00001308
1309 # Save the option flags (since option directives can be used
1310 # to modify them).
1311 original_optionflags = self.optionflags
1312
1313 # Process each example.
1314 for example in test.examples:
Edward Loper74bca7a2004-08-12 02:27:44 +00001315 # Merge in the example's options.
1316 self.optionflags = original_optionflags
1317 if example.options:
1318 for (optionflag, val) in example.options.items():
1319 if val:
1320 self.optionflags |= optionflag
1321 else:
1322 self.optionflags &= ~optionflag
Tim Peters8485b562004-08-04 18:46:34 +00001323
1324 # Record that we started this example.
1325 tries += 1
1326 self.report_start(out, test, example)
1327
1328 # Run the example in the given context (globs), and record
1329 # any exception that gets raised. (But don't intercept
1330 # keyboard interrupts.)
1331 try:
Tim Peters208ca702004-08-09 04:12:36 +00001332 # Don't blink! This is where the user's code gets run.
Tim Petersbb431472004-08-09 03:51:46 +00001333 exec compile(example.source, "<string>", "single",
Tim Peters8485b562004-08-04 18:46:34 +00001334 compileflags, 1) in test.globs
1335 exception = None
1336 except KeyboardInterrupt:
1337 raise
1338 except:
1339 exception = sys.exc_info()
1340
Tim Peters208ca702004-08-09 04:12:36 +00001341 got = self._fakeout.getvalue() # the actual output
Tim Peters8485b562004-08-04 18:46:34 +00001342 self._fakeout.truncate(0)
1343
1344 # If the example executed without raising any exceptions,
1345 # then verify its output and report its outcome.
1346 if exception is None:
Edward Loper34fcb142004-08-09 02:45:41 +00001347 if self._checker.check_output(example.want, got,
1348 self.optionflags):
Tim Peters8485b562004-08-04 18:46:34 +00001349 self.report_success(out, test, example, got)
Tim Peters8a7d2d52001-01-16 07:10:57 +00001350 else:
Tim Peters8485b562004-08-04 18:46:34 +00001351 self.report_failure(out, test, example, got)
1352 failures += 1
1353
1354 # If the example raised an exception, then check if it was
1355 # expected.
1356 else:
1357 exc_info = sys.exc_info()
1358 exc_msg = traceback.format_exception_only(*exc_info[:2])[-1]
1359
1360 # Search the `want` string for an exception. If we don't
1361 # find one, then report an unexpected exception.
1362 m = self._EXCEPTION_RE.match(example.want)
1363 if m is None:
1364 self.report_unexpected_exception(out, test, example,
1365 exc_info)
1366 failures += 1
1367 else:
Tim Peters41a65ea2004-08-13 03:55:05 +00001368 e_want, e_msg = m.group('want', 'msg')
Tim Peters8485b562004-08-04 18:46:34 +00001369 # The test passes iff the pre-exception output and
1370 # the exception description match the values given
1371 # in `want`.
Tim Peters41a65ea2004-08-13 03:55:05 +00001372 if (self._checker.check_output(e_want, got,
Edward Loper34fcb142004-08-09 02:45:41 +00001373 self.optionflags) and
Tim Peters41a65ea2004-08-13 03:55:05 +00001374 self._checker.check_output(e_msg, exc_msg,
Edward Loper34fcb142004-08-09 02:45:41 +00001375 self.optionflags)):
Tim Peters8485b562004-08-04 18:46:34 +00001376 self.report_success(out, test, example,
Tim Peters41a65ea2004-08-13 03:55:05 +00001377 got + _exception_traceback(exc_info))
Tim Peters8485b562004-08-04 18:46:34 +00001378 else:
1379 self.report_failure(out, test, example,
Tim Peters41a65ea2004-08-13 03:55:05 +00001380 got + _exception_traceback(exc_info))
Tim Peters8485b562004-08-04 18:46:34 +00001381 failures += 1
1382
1383 # Restore the option flags (in case they were modified)
1384 self.optionflags = original_optionflags
1385
1386 # Record and return the number of failures and tries.
1387 self.__record_outcome(test, failures, tries)
Tim Peters8a7d2d52001-01-16 07:10:57 +00001388 return failures, tries
1389
Tim Peters8485b562004-08-04 18:46:34 +00001390 def __record_outcome(self, test, f, t):
1391 """
1392 Record the fact that the given DocTest (`test`) generated `f`
1393 failures out of `t` tried examples.
1394 """
1395 f2, t2 = self._name2ft.get(test.name, (0,0))
1396 self._name2ft[test.name] = (f+f2, t+t2)
1397 self.failures += f
1398 self.tries += t
1399
1400 def run(self, test, compileflags=None, out=None, clear_globs=True):
1401 """
1402 Run the examples in `test`, and display the results using the
1403 writer function `out`.
1404
1405 The examples are run in the namespace `test.globs`. If
1406 `clear_globs` is true (the default), then this namespace will
1407 be cleared after the test runs, to help with garbage
1408 collection. If you would like to examine the namespace after
1409 the test completes, then use `clear_globs=False`.
1410
1411 `compileflags` gives the set of flags that should be used by
1412 the Python compiler when running the examples. If not
1413 specified, then it will default to the set of future-import
1414 flags that apply to `globs`.
1415
1416 The output of each example is checked using
1417 `DocTestRunner.check_output`, and the results are formatted by
1418 the `DocTestRunner.report_*` methods.
1419 """
1420 if compileflags is None:
1421 compileflags = _extract_future_flags(test.globs)
Jim Fulton356fd192004-08-09 11:34:47 +00001422
Tim Peters6c542b72004-08-09 16:43:36 +00001423 save_stdout = sys.stdout
Tim Peters8485b562004-08-04 18:46:34 +00001424 if out is None:
Tim Peters6c542b72004-08-09 16:43:36 +00001425 out = save_stdout.write
1426 sys.stdout = self._fakeout
Tim Peters8485b562004-08-04 18:46:34 +00001427
Tim Peters6c542b72004-08-09 16:43:36 +00001428 # Patch pdb.set_trace to restore sys.stdout, so that interactive
1429 # debugging output is visible (not still redirected to self._fakeout).
1430 # Note that we run "the real" pdb.set_trace (captured at doctest
1431 # import time) in our replacement. Because the current run() may
1432 # run another doctest (and so on), the current pdb.set_trace may be
1433 # our set_trace function, which changes sys.stdout. If we called
1434 # a chain of those, we wouldn't be left with the save_stdout
1435 # *this* run() invocation wants.
Jim Fulton356fd192004-08-09 11:34:47 +00001436 def set_trace():
Tim Peters6c542b72004-08-09 16:43:36 +00001437 sys.stdout = save_stdout
Jim Fulton356fd192004-08-09 11:34:47 +00001438 real_pdb_set_trace()
1439
Tim Peters6c542b72004-08-09 16:43:36 +00001440 save_set_trace = pdb.set_trace
1441 pdb.set_trace = set_trace
Tim Peters8485b562004-08-04 18:46:34 +00001442 try:
Tim Peters8485b562004-08-04 18:46:34 +00001443 return self.__run(test, compileflags, out)
1444 finally:
Tim Peters6c542b72004-08-09 16:43:36 +00001445 sys.stdout = save_stdout
1446 pdb.set_trace = save_set_trace
Tim Peters8485b562004-08-04 18:46:34 +00001447 if clear_globs:
1448 test.globs.clear()
1449
1450 #/////////////////////////////////////////////////////////////////
1451 # Summarization
1452 #/////////////////////////////////////////////////////////////////
Tim Peters8a7d2d52001-01-16 07:10:57 +00001453 def summarize(self, verbose=None):
1454 """
Tim Peters8485b562004-08-04 18:46:34 +00001455 Print a summary of all the test cases that have been run by
1456 this DocTestRunner, and return a tuple `(f, t)`, where `f` is
1457 the total number of failed examples, and `t` is the total
1458 number of tried examples.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001459
Tim Peters8485b562004-08-04 18:46:34 +00001460 The optional `verbose` argument controls how detailed the
1461 summary is. If the verbosity is not specified, then the
1462 DocTestRunner's verbosity is used.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001463 """
Tim Peters8a7d2d52001-01-16 07:10:57 +00001464 if verbose is None:
Tim Peters8485b562004-08-04 18:46:34 +00001465 verbose = self._verbose
Tim Peters8a7d2d52001-01-16 07:10:57 +00001466 notests = []
1467 passed = []
1468 failed = []
1469 totalt = totalf = 0
Tim Peters8485b562004-08-04 18:46:34 +00001470 for x in self._name2ft.items():
Tim Peters8a7d2d52001-01-16 07:10:57 +00001471 name, (f, t) = x
1472 assert f <= t
Tim Peters8485b562004-08-04 18:46:34 +00001473 totalt += t
1474 totalf += f
Tim Peters8a7d2d52001-01-16 07:10:57 +00001475 if t == 0:
1476 notests.append(name)
1477 elif f == 0:
1478 passed.append( (name, t) )
1479 else:
1480 failed.append(x)
1481 if verbose:
1482 if notests:
1483 print len(notests), "items had no tests:"
1484 notests.sort()
1485 for thing in notests:
1486 print " ", thing
1487 if passed:
1488 print len(passed), "items passed all tests:"
1489 passed.sort()
1490 for thing, count in passed:
1491 print " %3d tests in %s" % (count, thing)
1492 if failed:
Tim Peters8485b562004-08-04 18:46:34 +00001493 print self.DIVIDER
Tim Peters8a7d2d52001-01-16 07:10:57 +00001494 print len(failed), "items had failures:"
1495 failed.sort()
1496 for thing, (f, t) in failed:
1497 print " %3d of %3d in %s" % (f, t, thing)
1498 if verbose:
Tim Peters8485b562004-08-04 18:46:34 +00001499 print totalt, "tests in", len(self._name2ft), "items."
Tim Peters8a7d2d52001-01-16 07:10:57 +00001500 print totalt - totalf, "passed and", totalf, "failed."
1501 if totalf:
1502 print "***Test Failed***", totalf, "failures."
1503 elif verbose:
1504 print "Test passed."
1505 return totalf, totalt
1506
Edward Loper34fcb142004-08-09 02:45:41 +00001507class OutputChecker:
1508 """
1509 A class used to check the whether the actual output from a doctest
1510 example matches the expected output. `OutputChecker` defines two
1511 methods: `check_output`, which compares a given pair of outputs,
1512 and returns true if they match; and `output_difference`, which
1513 returns a string describing the differences between two outputs.
1514 """
1515 def check_output(self, want, got, optionflags):
1516 """
Edward Loper74bca7a2004-08-12 02:27:44 +00001517 Return True iff the actual output from an example (`got`)
1518 matches the expected output (`want`). These strings are
1519 always considered to match if they are identical; but
1520 depending on what option flags the test runner is using,
1521 several non-exact match types are also possible. See the
1522 documentation for `TestRunner` for more information about
1523 option flags.
Edward Loper34fcb142004-08-09 02:45:41 +00001524 """
1525 # Handle the common case first, for efficiency:
1526 # if they're string-identical, always return true.
1527 if got == want:
1528 return True
1529
1530 # The values True and False replaced 1 and 0 as the return
1531 # value for boolean comparisons in Python 2.3.
1532 if not (optionflags & DONT_ACCEPT_TRUE_FOR_1):
1533 if (got,want) == ("True\n", "1\n"):
1534 return True
1535 if (got,want) == ("False\n", "0\n"):
1536 return True
1537
1538 # <BLANKLINE> can be used as a special sequence to signify a
1539 # blank line, unless the DONT_ACCEPT_BLANKLINE flag is used.
1540 if not (optionflags & DONT_ACCEPT_BLANKLINE):
1541 # Replace <BLANKLINE> in want with a blank line.
1542 want = re.sub('(?m)^%s\s*?$' % re.escape(BLANKLINE_MARKER),
1543 '', want)
1544 # If a line in got contains only spaces, then remove the
1545 # spaces.
1546 got = re.sub('(?m)^\s*?$', '', got)
1547 if got == want:
1548 return True
1549
1550 # This flag causes doctest to ignore any differences in the
1551 # contents of whitespace strings. Note that this can be used
1552 # in conjunction with the ELLISPIS flag.
Tim Peters1cf3aa62004-08-19 06:49:33 +00001553 if optionflags & NORMALIZE_WHITESPACE:
Edward Loper34fcb142004-08-09 02:45:41 +00001554 got = ' '.join(got.split())
1555 want = ' '.join(want.split())
1556 if got == want:
1557 return True
1558
1559 # The ELLIPSIS flag says to let the sequence "..." in `want`
Tim Peters26b3ebb2004-08-19 08:10:08 +00001560 # match any substring in `got`.
Tim Peters1cf3aa62004-08-19 06:49:33 +00001561 if optionflags & ELLIPSIS:
Tim Peters26b3ebb2004-08-19 08:10:08 +00001562 if ellipsis_match(want, got):
Edward Loper34fcb142004-08-09 02:45:41 +00001563 return True
1564
1565 # We didn't find any match; return false.
1566 return False
1567
1568 def output_difference(self, want, got, optionflags):
1569 """
1570 Return a string describing the differences between the
Edward Loper74bca7a2004-08-12 02:27:44 +00001571 expected output for an example (`want`) and the actual output
1572 (`got`). `optionflags` is the set of option flags used to
1573 compare `want` and `got`. `indent` is the indentation of the
1574 original example.
Edward Loper34fcb142004-08-09 02:45:41 +00001575 """
Edward Loper68ba9a62004-08-12 02:43:49 +00001576 # If <BLANKLINE>s are being used, then replace blank lines
1577 # with <BLANKLINE> in the actual output string.
Edward Loper34fcb142004-08-09 02:45:41 +00001578 if not (optionflags & DONT_ACCEPT_BLANKLINE):
Edward Loper68ba9a62004-08-12 02:43:49 +00001579 got = re.sub('(?m)^[ ]*(?=\n)', BLANKLINE_MARKER, got)
Edward Loper34fcb142004-08-09 02:45:41 +00001580
1581 # Check if we should use diff. Don't use diff if the actual
1582 # or expected outputs are too short, or if the expected output
1583 # contains an ellipsis marker.
1584 if ((optionflags & (UNIFIED_DIFF | CONTEXT_DIFF)) and
1585 want.count('\n') > 2 and got.count('\n') > 2 and
1586 not (optionflags & ELLIPSIS and '...' in want)):
1587 # Split want & got into lines.
1588 want_lines = [l+'\n' for l in want.split('\n')]
1589 got_lines = [l+'\n' for l in got.split('\n')]
1590 # Use difflib to find their differences.
1591 if optionflags & UNIFIED_DIFF:
1592 diff = difflib.unified_diff(want_lines, got_lines, n=2,
1593 fromfile='Expected', tofile='Got')
1594 kind = 'unified'
1595 elif optionflags & CONTEXT_DIFF:
1596 diff = difflib.context_diff(want_lines, got_lines, n=2,
1597 fromfile='Expected', tofile='Got')
1598 kind = 'context'
1599 else:
1600 assert 0, 'Bad diff option'
1601 # Remove trailing whitespace on diff output.
1602 diff = [line.rstrip() + '\n' for line in diff]
1603 return _tag_msg("Differences (" + kind + " diff)",
1604 ''.join(diff))
1605
1606 # If we're not using diff, then simply list the expected
1607 # output followed by the actual output.
1608 return (_tag_msg("Expected", want or "Nothing") +
1609 _tag_msg("Got", got))
1610
Tim Peters19397e52004-08-06 22:02:59 +00001611class DocTestFailure(Exception):
1612 """A DocTest example has failed in debugging mode.
1613
1614 The exception instance has variables:
1615
1616 - test: the DocTest object being run
1617
1618 - excample: the Example object that failed
1619
1620 - got: the actual output
1621 """
1622 def __init__(self, test, example, got):
1623 self.test = test
1624 self.example = example
1625 self.got = got
1626
1627 def __str__(self):
1628 return str(self.test)
1629
1630class UnexpectedException(Exception):
1631 """A DocTest example has encountered an unexpected exception
1632
1633 The exception instance has variables:
1634
1635 - test: the DocTest object being run
1636
1637 - excample: the Example object that failed
1638
1639 - exc_info: the exception info
1640 """
1641 def __init__(self, test, example, exc_info):
1642 self.test = test
1643 self.example = example
1644 self.exc_info = exc_info
1645
1646 def __str__(self):
1647 return str(self.test)
Tim Petersd1b78272004-08-07 06:03:09 +00001648
Tim Peters19397e52004-08-06 22:02:59 +00001649class DebugRunner(DocTestRunner):
1650 r"""Run doc tests but raise an exception as soon as there is a failure.
1651
1652 If an unexpected exception occurs, an UnexpectedException is raised.
1653 It contains the test, the example, and the original exception:
1654
1655 >>> runner = DebugRunner(verbose=False)
Edward Lopera1ef6112004-08-09 16:14:41 +00001656 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42',
1657 ... {}, 'foo', 'foo.py', 0)
Tim Peters19397e52004-08-06 22:02:59 +00001658 >>> try:
1659 ... runner.run(test)
1660 ... except UnexpectedException, failure:
1661 ... pass
1662
1663 >>> failure.test is test
1664 True
1665
1666 >>> failure.example.want
1667 '42\n'
1668
1669 >>> exc_info = failure.exc_info
1670 >>> raise exc_info[0], exc_info[1], exc_info[2]
1671 Traceback (most recent call last):
1672 ...
1673 KeyError
1674
1675 We wrap the original exception to give the calling application
1676 access to the test and example information.
1677
1678 If the output doesn't match, then a DocTestFailure is raised:
1679
Edward Lopera1ef6112004-08-09 16:14:41 +00001680 >>> test = DocTestParser().get_doctest('''
Tim Peters19397e52004-08-06 22:02:59 +00001681 ... >>> x = 1
1682 ... >>> x
1683 ... 2
1684 ... ''', {}, 'foo', 'foo.py', 0)
1685
1686 >>> try:
1687 ... runner.run(test)
1688 ... except DocTestFailure, failure:
1689 ... pass
1690
1691 DocTestFailure objects provide access to the test:
1692
1693 >>> failure.test is test
1694 True
1695
1696 As well as to the example:
1697
1698 >>> failure.example.want
1699 '2\n'
1700
1701 and the actual output:
1702
1703 >>> failure.got
1704 '1\n'
1705
1706 If a failure or error occurs, the globals are left intact:
1707
1708 >>> del test.globs['__builtins__']
1709 >>> test.globs
1710 {'x': 1}
1711
Edward Lopera1ef6112004-08-09 16:14:41 +00001712 >>> test = DocTestParser().get_doctest('''
Tim Peters19397e52004-08-06 22:02:59 +00001713 ... >>> x = 2
1714 ... >>> raise KeyError
1715 ... ''', {}, 'foo', 'foo.py', 0)
1716
1717 >>> runner.run(test)
1718 Traceback (most recent call last):
1719 ...
1720 UnexpectedException: <DocTest foo from foo.py:0 (2 examples)>
Tim Petersd1b78272004-08-07 06:03:09 +00001721
Tim Peters19397e52004-08-06 22:02:59 +00001722 >>> del test.globs['__builtins__']
1723 >>> test.globs
1724 {'x': 2}
1725
1726 But the globals are cleared if there is no error:
1727
Edward Lopera1ef6112004-08-09 16:14:41 +00001728 >>> test = DocTestParser().get_doctest('''
Tim Peters19397e52004-08-06 22:02:59 +00001729 ... >>> x = 2
1730 ... ''', {}, 'foo', 'foo.py', 0)
1731
1732 >>> runner.run(test)
1733 (0, 1)
1734
1735 >>> test.globs
1736 {}
1737
1738 """
1739
1740 def run(self, test, compileflags=None, out=None, clear_globs=True):
1741 r = DocTestRunner.run(self, test, compileflags, out, False)
1742 if clear_globs:
1743 test.globs.clear()
1744 return r
1745
1746 def report_unexpected_exception(self, out, test, example, exc_info):
1747 raise UnexpectedException(test, example, exc_info)
1748
1749 def report_failure(self, out, test, example, got):
1750 raise DocTestFailure(test, example, got)
1751
Tim Peters8485b562004-08-04 18:46:34 +00001752######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001753## 6. Test Functions
Tim Peters8485b562004-08-04 18:46:34 +00001754######################################################################
1755# These should be backwards compatible.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001756
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001757def testmod(m=None, name=None, globs=None, verbose=None, isprivate=None,
Tim Peters19397e52004-08-06 22:02:59 +00001758 report=True, optionflags=0, extraglobs=None,
1759 raise_on_error=False):
Tim Peters6ebe61f2003-06-27 20:48:05 +00001760 """m=None, name=None, globs=None, verbose=None, isprivate=None,
Tim Peters8485b562004-08-04 18:46:34 +00001761 report=True, optionflags=0, extraglobs=None
Tim Peters8a7d2d52001-01-16 07:10:57 +00001762
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001763 Test examples in docstrings in functions and classes reachable
1764 from module m (or the current module if m is not supplied), starting
Raymond Hettinger71adf7e2003-07-16 19:25:22 +00001765 with m.__doc__. Unless isprivate is specified, private names
1766 are not skipped.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001767
1768 Also test examples reachable from dict m.__test__ if it exists and is
Tim Petersc2388a22004-08-10 01:41:28 +00001769 not None. m.__test__ maps names to functions, classes and strings;
Tim Peters8a7d2d52001-01-16 07:10:57 +00001770 function and class docstrings are tested even if the name is private;
1771 strings are tested directly, as if they were docstrings.
1772
1773 Return (#failures, #tests).
1774
1775 See doctest.__doc__ for an overview.
1776
1777 Optional keyword arg "name" gives the name of the module; by default
1778 use m.__name__.
1779
1780 Optional keyword arg "globs" gives a dict to be used as the globals
1781 when executing examples; by default, use m.__dict__. A copy of this
1782 dict is actually used for each docstring, so that each docstring's
1783 examples start with a clean slate.
1784
Tim Peters8485b562004-08-04 18:46:34 +00001785 Optional keyword arg "extraglobs" gives a dictionary that should be
1786 merged into the globals that are used to execute examples. By
1787 default, no extra globals are used. This is new in 2.4.
1788
Tim Peters8a7d2d52001-01-16 07:10:57 +00001789 Optional keyword arg "verbose" prints lots of stuff if true, prints
1790 only failures if false; by default, it's true iff "-v" is in sys.argv.
1791
Tim Peters8a7d2d52001-01-16 07:10:57 +00001792 Optional keyword arg "report" prints a summary at the end when true,
1793 else prints nothing at the end. In verbose mode, the summary is
1794 detailed, else very brief (in fact, empty if all tests passed).
1795
Tim Peters6ebe61f2003-06-27 20:48:05 +00001796 Optional keyword arg "optionflags" or's together module constants,
1797 and defaults to 0. This is new in 2.3. Possible values:
1798
1799 DONT_ACCEPT_TRUE_FOR_1
1800 By default, if an expected output block contains just "1",
1801 an actual output block containing just "True" is considered
1802 to be a match, and similarly for "0" versus "False". When
1803 DONT_ACCEPT_TRUE_FOR_1 is specified, neither substitution
1804 is allowed.
1805
Tim Peters8485b562004-08-04 18:46:34 +00001806 DONT_ACCEPT_BLANKLINE
1807 By default, if an expected output block contains a line
1808 containing only the string "<BLANKLINE>", then that line
1809 will match a blank line in the actual output. When
1810 DONT_ACCEPT_BLANKLINE is specified, this substitution is
1811 not allowed.
1812
1813 NORMALIZE_WHITESPACE
1814 When NORMALIZE_WHITESPACE is specified, all sequences of
1815 whitespace are treated as equal. I.e., any sequence of
1816 whitespace within the expected output will match any
1817 sequence of whitespace within the actual output.
1818
1819 ELLIPSIS
1820 When ELLIPSIS is specified, then an ellipsis marker
1821 ("...") in the expected output can match any substring in
1822 the actual output.
1823
1824 UNIFIED_DIFF
1825 When UNIFIED_DIFF is specified, failures that involve
1826 multi-line expected and actual outputs will be displayed
1827 using a unified diff.
1828
1829 CONTEXT_DIFF
1830 When CONTEXT_DIFF is specified, failures that involve
1831 multi-line expected and actual outputs will be displayed
1832 using a context diff.
Tim Peters19397e52004-08-06 22:02:59 +00001833
1834 Optional keyword arg "raise_on_error" raises an exception on the
1835 first unexpected exception or failure. This allows failures to be
1836 post-mortem debugged.
1837
Tim Petersf727c6c2004-08-08 01:48:59 +00001838 Deprecated in Python 2.4:
1839 Optional keyword arg "isprivate" specifies a function used to
1840 determine whether a name is private. The default function is
1841 treat all functions as public. Optionally, "isprivate" can be
1842 set to doctest.is_private to skip over functions marked as private
1843 using the underscore naming convention; see its docs for details.
Tim Peters8485b562004-08-04 18:46:34 +00001844 """
1845
1846 """ [XX] This is no longer true:
Tim Peters8a7d2d52001-01-16 07:10:57 +00001847 Advanced tomfoolery: testmod runs methods of a local instance of
1848 class doctest.Tester, then merges the results into (or creates)
1849 global Tester instance doctest.master. Methods of doctest.master
1850 can be called directly too, if you want to do something unusual.
1851 Passing report=0 to testmod is especially useful then, to delay
1852 displaying a summary. Invoke doctest.master.summarize(verbose)
1853 when you're done fiddling.
1854 """
Tim Petersf727c6c2004-08-08 01:48:59 +00001855 if isprivate is not None:
1856 warnings.warn("the isprivate argument is deprecated; "
1857 "examine DocTestFinder.find() lists instead",
1858 DeprecationWarning)
1859
Tim Peters8485b562004-08-04 18:46:34 +00001860 # If no module was given, then use __main__.
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001861 if m is None:
Martin v. Löwis4581cfa2002-11-22 08:23:09 +00001862 # DWA - m will still be None if this wasn't invoked from the command
1863 # line, in which case the following TypeError is about as good an error
1864 # as we should expect
1865 m = sys.modules.get('__main__')
1866
Tim Peters8485b562004-08-04 18:46:34 +00001867 # Check that we were actually given a module.
1868 if not inspect.ismodule(m):
Walter Dörwald70a6b492004-02-12 17:35:32 +00001869 raise TypeError("testmod: module required; %r" % (m,))
Tim Peters8485b562004-08-04 18:46:34 +00001870
1871 # If no name was given, then use the module's name.
Tim Peters8a7d2d52001-01-16 07:10:57 +00001872 if name is None:
1873 name = m.__name__
Tim Peters8485b562004-08-04 18:46:34 +00001874
1875 # Find, parse, and run all tests in the given module.
Tim Petersf727c6c2004-08-08 01:48:59 +00001876 finder = DocTestFinder(_namefilter=isprivate)
Tim Peters19397e52004-08-06 22:02:59 +00001877
1878 if raise_on_error:
1879 runner = DebugRunner(verbose=verbose, optionflags=optionflags)
1880 else:
1881 runner = DocTestRunner(verbose=verbose, optionflags=optionflags)
1882
Tim Peters8485b562004-08-04 18:46:34 +00001883 for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
1884 runner.run(test)
1885
Tim Peters8a7d2d52001-01-16 07:10:57 +00001886 if report:
Tim Peters8485b562004-08-04 18:46:34 +00001887 runner.summarize()
Tim Peters8a7d2d52001-01-16 07:10:57 +00001888
Tim Peters8485b562004-08-04 18:46:34 +00001889 return runner.failures, runner.tries
Tim Petersdb3756d2003-06-29 05:30:48 +00001890
Tim Peters8485b562004-08-04 18:46:34 +00001891def run_docstring_examples(f, globs, verbose=False, name="NoName",
1892 compileflags=None, optionflags=0):
1893 """
1894 Test examples in the given object's docstring (`f`), using `globs`
1895 as globals. Optional argument `name` is used in failure messages.
1896 If the optional argument `verbose` is true, then generate output
1897 even if there are no failures.
Tim Petersdb3756d2003-06-29 05:30:48 +00001898
Tim Peters8485b562004-08-04 18:46:34 +00001899 `compileflags` gives the set of flags that should be used by the
1900 Python compiler when running the examples. If not specified, then
1901 it will default to the set of future-import flags that apply to
1902 `globs`.
Tim Petersdb3756d2003-06-29 05:30:48 +00001903
Tim Peters8485b562004-08-04 18:46:34 +00001904 Optional keyword arg `optionflags` specifies options for the
1905 testing and output. See the documentation for `testmod` for more
1906 information.
1907 """
1908 # Find, parse, and run all tests in the given module.
1909 finder = DocTestFinder(verbose=verbose, recurse=False)
1910 runner = DocTestRunner(verbose=verbose, optionflags=optionflags)
1911 for test in finder.find(f, name, globs=globs):
1912 runner.run(test, compileflags=compileflags)
Tim Petersdb3756d2003-06-29 05:30:48 +00001913
Tim Peters8485b562004-08-04 18:46:34 +00001914######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001915## 7. Tester
Tim Peters8485b562004-08-04 18:46:34 +00001916######################################################################
1917# This is provided only for backwards compatibility. It's not
1918# actually used in any way.
Tim Petersdb3756d2003-06-29 05:30:48 +00001919
Tim Peters8485b562004-08-04 18:46:34 +00001920class Tester:
1921 def __init__(self, mod=None, globs=None, verbose=None,
1922 isprivate=None, optionflags=0):
Tim Peters3ddd60a2004-08-08 02:43:33 +00001923
1924 warnings.warn("class Tester is deprecated; "
1925 "use class doctest.DocTestRunner instead",
1926 DeprecationWarning, stacklevel=2)
Tim Peters8485b562004-08-04 18:46:34 +00001927 if mod is None and globs is None:
1928 raise TypeError("Tester.__init__: must specify mod or globs")
1929 if mod is not None and not _ismodule(mod):
1930 raise TypeError("Tester.__init__: mod must be a module; %r" %
1931 (mod,))
1932 if globs is None:
1933 globs = mod.__dict__
1934 self.globs = globs
Tim Petersdb3756d2003-06-29 05:30:48 +00001935
Tim Peters8485b562004-08-04 18:46:34 +00001936 self.verbose = verbose
1937 self.isprivate = isprivate
1938 self.optionflags = optionflags
Tim Petersf727c6c2004-08-08 01:48:59 +00001939 self.testfinder = DocTestFinder(_namefilter=isprivate)
Tim Peters8485b562004-08-04 18:46:34 +00001940 self.testrunner = DocTestRunner(verbose=verbose,
1941 optionflags=optionflags)
Tim Petersdb3756d2003-06-29 05:30:48 +00001942
Tim Peters8485b562004-08-04 18:46:34 +00001943 def runstring(self, s, name):
Edward Lopera1ef6112004-08-09 16:14:41 +00001944 test = DocTestParser().get_doctest(s, self.globs, name, None, None)
Tim Peters8485b562004-08-04 18:46:34 +00001945 if self.verbose:
1946 print "Running string", name
1947 (f,t) = self.testrunner.run(test)
1948 if self.verbose:
1949 print f, "of", t, "examples failed in string", name
1950 return (f,t)
Tim Petersdb3756d2003-06-29 05:30:48 +00001951
Tim Petersf3f57472004-08-08 06:11:48 +00001952 def rundoc(self, object, name=None, module=None):
Tim Peters8485b562004-08-04 18:46:34 +00001953 f = t = 0
1954 tests = self.testfinder.find(object, name, module=module,
Tim Petersf3f57472004-08-08 06:11:48 +00001955 globs=self.globs)
Tim Peters8485b562004-08-04 18:46:34 +00001956 for test in tests:
1957 (f2, t2) = self.testrunner.run(test)
1958 (f,t) = (f+f2, t+t2)
1959 return (f,t)
Tim Petersdb3756d2003-06-29 05:30:48 +00001960
Tim Peters8485b562004-08-04 18:46:34 +00001961 def rundict(self, d, name, module=None):
1962 import new
1963 m = new.module(name)
1964 m.__dict__.update(d)
Tim Petersf3f57472004-08-08 06:11:48 +00001965 if module is None:
1966 module = False
1967 return self.rundoc(m, name, module)
Tim Petersdb3756d2003-06-29 05:30:48 +00001968
Tim Peters8485b562004-08-04 18:46:34 +00001969 def run__test__(self, d, name):
1970 import new
1971 m = new.module(name)
1972 m.__test__ = d
1973 return self.rundoc(m, name, module)
Tim Petersdb3756d2003-06-29 05:30:48 +00001974
Tim Peters8485b562004-08-04 18:46:34 +00001975 def summarize(self, verbose=None):
1976 return self.testrunner.summarize(verbose)
Tim Petersdb3756d2003-06-29 05:30:48 +00001977
Tim Peters8485b562004-08-04 18:46:34 +00001978 def merge(self, other):
1979 d = self.testrunner._name2ft
1980 for name, (f, t) in other.testrunner._name2ft.items():
1981 if name in d:
1982 print "*** Tester.merge: '" + name + "' in both" \
1983 " testers; summing outcomes."
1984 f2, t2 = d[name]
1985 f = f + f2
1986 t = t + t2
1987 d[name] = f, t
Tim Petersdb3756d2003-06-29 05:30:48 +00001988
Tim Peters8485b562004-08-04 18:46:34 +00001989######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00001990## 8. Unittest Support
Tim Peters8485b562004-08-04 18:46:34 +00001991######################################################################
Tim Petersdb3756d2003-06-29 05:30:48 +00001992
Tim Peters19397e52004-08-06 22:02:59 +00001993class DocTestCase(unittest.TestCase):
Tim Petersdb3756d2003-06-29 05:30:48 +00001994
Edward Loper34fcb142004-08-09 02:45:41 +00001995 def __init__(self, test, optionflags=0, setUp=None, tearDown=None,
1996 checker=None):
Jim Fultona643b652004-07-14 19:06:50 +00001997 unittest.TestCase.__init__(self)
Tim Peters19397e52004-08-06 22:02:59 +00001998 self._dt_optionflags = optionflags
Edward Loper34fcb142004-08-09 02:45:41 +00001999 self._dt_checker = checker
Tim Peters19397e52004-08-06 22:02:59 +00002000 self._dt_test = test
2001 self._dt_setUp = setUp
2002 self._dt_tearDown = tearDown
Tim Petersdb3756d2003-06-29 05:30:48 +00002003
Jim Fultona643b652004-07-14 19:06:50 +00002004 def setUp(self):
Tim Peters19397e52004-08-06 22:02:59 +00002005 if self._dt_setUp is not None:
2006 self._dt_setUp()
Jim Fultona643b652004-07-14 19:06:50 +00002007
2008 def tearDown(self):
Tim Peters19397e52004-08-06 22:02:59 +00002009 if self._dt_tearDown is not None:
2010 self._dt_tearDown()
Jim Fultona643b652004-07-14 19:06:50 +00002011
2012 def runTest(self):
Tim Peters19397e52004-08-06 22:02:59 +00002013 test = self._dt_test
Jim Fultona643b652004-07-14 19:06:50 +00002014 old = sys.stdout
2015 new = StringIO()
Edward Loper34fcb142004-08-09 02:45:41 +00002016 runner = DocTestRunner(optionflags=self._dt_optionflags,
2017 checker=self._dt_checker, verbose=False)
Tim Peters19397e52004-08-06 22:02:59 +00002018
Jim Fultona643b652004-07-14 19:06:50 +00002019 try:
Tim Peters19397e52004-08-06 22:02:59 +00002020 runner.DIVIDER = "-"*70
2021 failures, tries = runner.run(test, out=new.write)
Jim Fultona643b652004-07-14 19:06:50 +00002022 finally:
2023 sys.stdout = old
2024
2025 if failures:
Tim Peters19397e52004-08-06 22:02:59 +00002026 raise self.failureException(self.format_failure(new.getvalue()))
Tim Peters8485b562004-08-04 18:46:34 +00002027
Tim Peters19397e52004-08-06 22:02:59 +00002028 def format_failure(self, err):
2029 test = self._dt_test
2030 if test.lineno is None:
2031 lineno = 'unknown line number'
2032 else:
2033 lineno = 'line %s' % test.lineno
2034 lname = '.'.join(test.name.split('.')[-1:])
2035 return ('Failed doctest test for %s\n'
2036 ' File "%s", line %s, in %s\n\n%s'
2037 % (test.name, test.filename, lineno, lname, err)
2038 )
2039
2040 def debug(self):
2041 r"""Run the test case without results and without catching exceptions
2042
2043 The unit test framework includes a debug method on test cases
2044 and test suites to support post-mortem debugging. The test code
2045 is run in such a way that errors are not caught. This way a
2046 caller can catch the errors and initiate post-mortem debugging.
2047
2048 The DocTestCase provides a debug method that raises
2049 UnexpectedException errors if there is an unexepcted
2050 exception:
2051
Edward Lopera1ef6112004-08-09 16:14:41 +00002052 >>> test = DocTestParser().get_doctest('>>> raise KeyError\n42',
Tim Peters19397e52004-08-06 22:02:59 +00002053 ... {}, 'foo', 'foo.py', 0)
2054 >>> case = DocTestCase(test)
2055 >>> try:
2056 ... case.debug()
2057 ... except UnexpectedException, failure:
2058 ... pass
2059
2060 The UnexpectedException contains the test, the example, and
2061 the original exception:
2062
2063 >>> failure.test is test
2064 True
2065
2066 >>> failure.example.want
2067 '42\n'
2068
2069 >>> exc_info = failure.exc_info
2070 >>> raise exc_info[0], exc_info[1], exc_info[2]
2071 Traceback (most recent call last):
2072 ...
2073 KeyError
2074
2075 If the output doesn't match, then a DocTestFailure is raised:
2076
Edward Lopera1ef6112004-08-09 16:14:41 +00002077 >>> test = DocTestParser().get_doctest('''
Tim Peters19397e52004-08-06 22:02:59 +00002078 ... >>> x = 1
2079 ... >>> x
2080 ... 2
2081 ... ''', {}, 'foo', 'foo.py', 0)
2082 >>> case = DocTestCase(test)
2083
2084 >>> try:
2085 ... case.debug()
2086 ... except DocTestFailure, failure:
2087 ... pass
2088
2089 DocTestFailure objects provide access to the test:
2090
2091 >>> failure.test is test
2092 True
2093
2094 As well as to the example:
2095
2096 >>> failure.example.want
2097 '2\n'
2098
2099 and the actual output:
2100
2101 >>> failure.got
2102 '1\n'
2103
2104 """
2105
Edward Loper34fcb142004-08-09 02:45:41 +00002106 runner = DebugRunner(optionflags=self._dt_optionflags,
2107 checker=self._dt_checker, verbose=False)
Tim Peters19397e52004-08-06 22:02:59 +00002108 runner.run(self._dt_test, out=nooutput)
Jim Fultona643b652004-07-14 19:06:50 +00002109
2110 def id(self):
Tim Peters19397e52004-08-06 22:02:59 +00002111 return self._dt_test.name
Jim Fultona643b652004-07-14 19:06:50 +00002112
2113 def __repr__(self):
Tim Peters19397e52004-08-06 22:02:59 +00002114 name = self._dt_test.name.split('.')
Jim Fultona643b652004-07-14 19:06:50 +00002115 return "%s (%s)" % (name[-1], '.'.join(name[:-1]))
2116
2117 __str__ = __repr__
2118
2119 def shortDescription(self):
Tim Peters19397e52004-08-06 22:02:59 +00002120 return "Doctest: " + self._dt_test.name
Jim Fultona643b652004-07-14 19:06:50 +00002121
Tim Peters19397e52004-08-06 22:02:59 +00002122def nooutput(*args):
2123 pass
Jim Fultona643b652004-07-14 19:06:50 +00002124
Tim Peters19397e52004-08-06 22:02:59 +00002125def DocTestSuite(module=None, globs=None, extraglobs=None,
2126 optionflags=0, test_finder=None,
Edward Loper34fcb142004-08-09 02:45:41 +00002127 setUp=lambda: None, tearDown=lambda: None,
2128 checker=None):
Tim Peters8485b562004-08-04 18:46:34 +00002129 """
Tim Peters19397e52004-08-06 22:02:59 +00002130 Convert doctest tests for a mudule to a unittest test suite.
Jim Fultona643b652004-07-14 19:06:50 +00002131
Tim Peters19397e52004-08-06 22:02:59 +00002132 This converts each documentation string in a module that
2133 contains doctest tests to a unittest test case. If any of the
2134 tests in a doc string fail, then the test case fails. An exception
2135 is raised showing the name of the file containing the test and a
Jim Fultona643b652004-07-14 19:06:50 +00002136 (sometimes approximate) line number.
2137
Tim Peters19397e52004-08-06 22:02:59 +00002138 The `module` argument provides the module to be tested. The argument
Jim Fultona643b652004-07-14 19:06:50 +00002139 can be either a module or a module name.
2140
2141 If no argument is given, the calling module is used.
Jim Fultona643b652004-07-14 19:06:50 +00002142 """
Jim Fultona643b652004-07-14 19:06:50 +00002143
Tim Peters8485b562004-08-04 18:46:34 +00002144 if test_finder is None:
2145 test_finder = DocTestFinder()
Tim Peters8485b562004-08-04 18:46:34 +00002146
Tim Peters19397e52004-08-06 22:02:59 +00002147 module = _normalize_module(module)
2148 tests = test_finder.find(module, globs=globs, extraglobs=extraglobs)
2149 if globs is None:
2150 globs = module.__dict__
2151 if not tests: # [XX] why do we want to do this?
2152 raise ValueError(module, "has no tests")
Tim Petersdb3756d2003-06-29 05:30:48 +00002153
2154 tests.sort()
2155 suite = unittest.TestSuite()
Tim Peters8485b562004-08-04 18:46:34 +00002156 for test in tests:
Tim Peters19397e52004-08-06 22:02:59 +00002157 if len(test.examples) == 0:
2158 continue
Tim Peters8485b562004-08-04 18:46:34 +00002159 if not test.filename:
Tim Petersdb3756d2003-06-29 05:30:48 +00002160 filename = module.__file__
2161 if filename.endswith(".pyc"):
2162 filename = filename[:-1]
2163 elif filename.endswith(".pyo"):
2164 filename = filename[:-1]
Tim Peters8485b562004-08-04 18:46:34 +00002165 test.filename = filename
Edward Loper34fcb142004-08-09 02:45:41 +00002166 suite.addTest(DocTestCase(test, optionflags, setUp, tearDown,
2167 checker))
Tim Peters19397e52004-08-06 22:02:59 +00002168
2169 return suite
2170
2171class DocFileCase(DocTestCase):
2172
2173 def id(self):
2174 return '_'.join(self._dt_test.name.split('.'))
2175
2176 def __repr__(self):
2177 return self._dt_test.filename
2178 __str__ = __repr__
2179
2180 def format_failure(self, err):
2181 return ('Failed doctest test for %s\n File "%s", line 0\n\n%s'
2182 % (self._dt_test.name, self._dt_test.filename, err)
2183 )
2184
2185def DocFileTest(path, package=None, globs=None,
2186 setUp=None, tearDown=None,
2187 optionflags=0):
2188 package = _normalize_module(package)
2189 name = path.split('/')[-1]
2190 dir = os.path.split(package.__file__)[0]
2191 path = os.path.join(dir, *(path.split('/')))
2192 doc = open(path).read()
2193
2194 if globs is None:
2195 globs = {}
2196
Edward Lopera1ef6112004-08-09 16:14:41 +00002197 test = DocTestParser().get_doctest(doc, globs, name, path, 0)
Tim Peters19397e52004-08-06 22:02:59 +00002198
2199 return DocFileCase(test, optionflags, setUp, tearDown)
2200
2201def DocFileSuite(*paths, **kw):
2202 """Creates a suite of doctest files.
2203
2204 One or more text file paths are given as strings. These should
2205 use "/" characters to separate path segments. Paths are relative
2206 to the directory of the calling module, or relative to the package
2207 passed as a keyword argument.
2208
2209 A number of options may be provided as keyword arguments:
2210
2211 package
2212 The name of a Python package. Text-file paths will be
2213 interpreted relative to the directory containing this package.
2214 The package may be supplied as a package object or as a dotted
2215 package name.
2216
2217 setUp
2218 The name of a set-up function. This is called before running the
2219 tests in each file.
2220
2221 tearDown
2222 The name of a tear-down function. This is called after running the
2223 tests in each file.
2224
2225 globs
2226 A dictionary containing initial global variables for the tests.
2227 """
2228 suite = unittest.TestSuite()
2229
2230 # We do this here so that _normalize_module is called at the right
2231 # level. If it were called in DocFileTest, then this function
2232 # would be the caller and we might guess the package incorrectly.
2233 kw['package'] = _normalize_module(kw.get('package'))
2234
2235 for path in paths:
2236 suite.addTest(DocFileTest(path, **kw))
Jim Fultona643b652004-07-14 19:06:50 +00002237
Tim Petersdb3756d2003-06-29 05:30:48 +00002238 return suite
2239
Tim Peters8485b562004-08-04 18:46:34 +00002240######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00002241## 9. Debugging Support
Tim Peters8485b562004-08-04 18:46:34 +00002242######################################################################
Jim Fultona643b652004-07-14 19:06:50 +00002243
Tim Peters19397e52004-08-06 22:02:59 +00002244def script_from_examples(s):
2245 r"""Extract script from text with examples.
2246
2247 Converts text with examples to a Python script. Example input is
2248 converted to regular code. Example output and all other words
2249 are converted to comments:
2250
2251 >>> text = '''
2252 ... Here are examples of simple math.
2253 ...
2254 ... Python has super accurate integer addition
2255 ...
2256 ... >>> 2 + 2
2257 ... 5
2258 ...
2259 ... And very friendly error messages:
2260 ...
2261 ... >>> 1/0
2262 ... To Infinity
2263 ... And
2264 ... Beyond
2265 ...
2266 ... You can use logic if you want:
2267 ...
2268 ... >>> if 0:
2269 ... ... blah
2270 ... ... blah
2271 ... ...
2272 ...
2273 ... Ho hum
2274 ... '''
2275
2276 >>> print script_from_examples(text)
Edward Lopera5db6002004-08-12 02:41:30 +00002277 # Here are examples of simple math.
Tim Peters19397e52004-08-06 22:02:59 +00002278 #
Edward Lopera5db6002004-08-12 02:41:30 +00002279 # Python has super accurate integer addition
Tim Peters19397e52004-08-06 22:02:59 +00002280 #
2281 2 + 2
2282 # Expected:
Edward Lopera5db6002004-08-12 02:41:30 +00002283 ## 5
Tim Peters19397e52004-08-06 22:02:59 +00002284 #
Edward Lopera5db6002004-08-12 02:41:30 +00002285 # And very friendly error messages:
Tim Peters19397e52004-08-06 22:02:59 +00002286 #
2287 1/0
2288 # Expected:
Edward Lopera5db6002004-08-12 02:41:30 +00002289 ## To Infinity
2290 ## And
2291 ## Beyond
Tim Peters19397e52004-08-06 22:02:59 +00002292 #
Edward Lopera5db6002004-08-12 02:41:30 +00002293 # You can use logic if you want:
Tim Peters19397e52004-08-06 22:02:59 +00002294 #
2295 if 0:
2296 blah
2297 blah
2298 <BLANKLINE>
2299 #
Edward Lopera5db6002004-08-12 02:41:30 +00002300 # Ho hum
Tim Peters19397e52004-08-06 22:02:59 +00002301 """
2302
Edward Lopera1ef6112004-08-09 16:14:41 +00002303 return DocTestParser().get_program(s)
Tim Peters19397e52004-08-06 22:02:59 +00002304
Tim Peters8485b562004-08-04 18:46:34 +00002305def _want_comment(example):
2306 """
Tim Peters19397e52004-08-06 22:02:59 +00002307 Return a comment containing the expected output for the given example.
Tim Peters8485b562004-08-04 18:46:34 +00002308 """
Jim Fultona643b652004-07-14 19:06:50 +00002309 # Return the expected output, if any
Tim Peters8485b562004-08-04 18:46:34 +00002310 want = example.want
2311 if want:
Tim Peters19397e52004-08-06 22:02:59 +00002312 if want[-1] == '\n':
2313 want = want[:-1]
Tim Peters8485b562004-08-04 18:46:34 +00002314 want = "\n# ".join(want.split("\n"))
2315 want = "\n# Expected:\n# %s" % want
2316 return want
Tim Petersdb3756d2003-06-29 05:30:48 +00002317
2318def testsource(module, name):
Tim Peters19397e52004-08-06 22:02:59 +00002319 """Extract the test sources from a doctest docstring as a script.
Tim Petersdb3756d2003-06-29 05:30:48 +00002320
2321 Provide the module (or dotted name of the module) containing the
Jim Fultona643b652004-07-14 19:06:50 +00002322 test to be debugged and the name (within the module) of the object
2323 with the doc string with tests to be debugged.
Tim Petersdb3756d2003-06-29 05:30:48 +00002324 """
Tim Peters8485b562004-08-04 18:46:34 +00002325 module = _normalize_module(module)
2326 tests = DocTestFinder().find(module)
2327 test = [t for t in tests if t.name == name]
Tim Petersdb3756d2003-06-29 05:30:48 +00002328 if not test:
2329 raise ValueError(name, "not found in tests")
2330 test = test[0]
Tim Peters19397e52004-08-06 22:02:59 +00002331 testsrc = script_from_examples(test.docstring)
Jim Fultona643b652004-07-14 19:06:50 +00002332 return testsrc
Tim Petersdb3756d2003-06-29 05:30:48 +00002333
Jim Fultona643b652004-07-14 19:06:50 +00002334def debug_src(src, pm=False, globs=None):
Tim Peters19397e52004-08-06 22:02:59 +00002335 """Debug a single doctest docstring, in argument `src`'"""
2336 testsrc = script_from_examples(src)
Tim Peters8485b562004-08-04 18:46:34 +00002337 debug_script(testsrc, pm, globs)
Tim Petersdb3756d2003-06-29 05:30:48 +00002338
Jim Fultona643b652004-07-14 19:06:50 +00002339def debug_script(src, pm=False, globs=None):
Tim Peters19397e52004-08-06 22:02:59 +00002340 "Debug a test script. `src` is the script, as a string."
Tim Petersdb3756d2003-06-29 05:30:48 +00002341 import pdb
Tim Petersdb3756d2003-06-29 05:30:48 +00002342
Tim Petersdb3756d2003-06-29 05:30:48 +00002343 srcfilename = tempfile.mktemp("doctestdebug.py")
Tim Peters8485b562004-08-04 18:46:34 +00002344 f = open(srcfilename, 'w')
2345 f.write(src)
2346 f.close()
2347
Jim Fultona643b652004-07-14 19:06:50 +00002348 if globs:
2349 globs = globs.copy()
2350 else:
2351 globs = {}
Tim Petersdb3756d2003-06-29 05:30:48 +00002352
Tim Peters8485b562004-08-04 18:46:34 +00002353 if pm:
2354 try:
2355 execfile(srcfilename, globs, globs)
2356 except:
2357 print sys.exc_info()[1]
2358 pdb.post_mortem(sys.exc_info()[2])
2359 else:
2360 # Note that %r is vital here. '%s' instead can, e.g., cause
2361 # backslashes to get treated as metacharacters on Windows.
2362 pdb.run("execfile(%r)" % srcfilename, globs, globs)
Tim Petersdb3756d2003-06-29 05:30:48 +00002363
Jim Fultona643b652004-07-14 19:06:50 +00002364def debug(module, name, pm=False):
Tim Peters19397e52004-08-06 22:02:59 +00002365 """Debug a single doctest docstring.
Jim Fultona643b652004-07-14 19:06:50 +00002366
2367 Provide the module (or dotted name of the module) containing the
2368 test to be debugged and the name (within the module) of the object
Tim Peters19397e52004-08-06 22:02:59 +00002369 with the docstring with tests to be debugged.
Jim Fultona643b652004-07-14 19:06:50 +00002370 """
Tim Peters8485b562004-08-04 18:46:34 +00002371 module = _normalize_module(module)
Jim Fultona643b652004-07-14 19:06:50 +00002372 testsrc = testsource(module, name)
2373 debug_script(testsrc, pm, module.__dict__)
2374
Tim Peters8485b562004-08-04 18:46:34 +00002375######################################################################
Edward Loper7c748462004-08-09 02:06:06 +00002376## 10. Example Usage
Tim Peters8485b562004-08-04 18:46:34 +00002377######################################################################
Tim Peters8a7d2d52001-01-16 07:10:57 +00002378class _TestClass:
2379 """
2380 A pointless class, for sanity-checking of docstring testing.
2381
2382 Methods:
2383 square()
2384 get()
2385
2386 >>> _TestClass(13).get() + _TestClass(-12).get()
2387 1
2388 >>> hex(_TestClass(13).square().get())
2389 '0xa9'
2390 """
2391
2392 def __init__(self, val):
2393 """val -> _TestClass object with associated value val.
2394
2395 >>> t = _TestClass(123)
2396 >>> print t.get()
2397 123
2398 """
2399
2400 self.val = val
2401
2402 def square(self):
2403 """square() -> square TestClass's associated value
2404
2405 >>> _TestClass(13).square().get()
2406 169
2407 """
2408
2409 self.val = self.val ** 2
2410 return self
2411
2412 def get(self):
2413 """get() -> return TestClass's associated value.
2414
2415 >>> x = _TestClass(-42)
2416 >>> print x.get()
2417 -42
2418 """
2419
2420 return self.val
2421
2422__test__ = {"_TestClass": _TestClass,
2423 "string": r"""
2424 Example of a string object, searched as-is.
2425 >>> x = 1; y = 2
2426 >>> x + y, x * y
2427 (3, 2)
Tim Peters6ebe61f2003-06-27 20:48:05 +00002428 """,
2429 "bool-int equivalence": r"""
2430 In 2.2, boolean expressions displayed
2431 0 or 1. By default, we still accept
2432 them. This can be disabled by passing
2433 DONT_ACCEPT_TRUE_FOR_1 to the new
2434 optionflags argument.
2435 >>> 4 == 4
2436 1
2437 >>> 4 == 4
2438 True
2439 >>> 4 > 4
2440 0
2441 >>> 4 > 4
2442 False
2443 """,
Tim Peters8485b562004-08-04 18:46:34 +00002444 "blank lines": r"""
2445 Blank lines can be marked with <BLANKLINE>:
2446 >>> print 'foo\n\nbar\n'
2447 foo
2448 <BLANKLINE>
2449 bar
2450 <BLANKLINE>
2451 """,
2452 }
2453# "ellipsis": r"""
2454# If the ellipsis flag is used, then '...' can be used to
2455# elide substrings in the desired output:
2456# >>> print range(1000)
2457# [0, 1, 2, ..., 999]
2458# """,
2459# "whitespace normalization": r"""
2460# If the whitespace normalization flag is used, then
2461# differences in whitespace are ignored.
2462# >>> print range(30)
2463# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
2464# 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
2465# 27, 28, 29]
2466# """,
2467# }
2468
2469def test1(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002470>>> warnings.filterwarnings("ignore", "class Tester", DeprecationWarning,
2471... "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002472>>> from doctest import Tester
2473>>> t = Tester(globs={'x': 42}, verbose=0)
2474>>> t.runstring(r'''
2475... >>> x = x * 2
2476... >>> print x
2477... 42
2478... ''', 'XYZ')
2479**********************************************************************
2480Failure in example: print x
2481from line #2 of XYZ
2482Expected: 42
2483Got: 84
2484(1, 2)
2485>>> t.runstring(">>> x = x * 2\n>>> print x\n84\n", 'example2')
2486(0, 2)
2487>>> t.summarize()
2488**********************************************************************
24891 items had failures:
2490 1 of 2 in XYZ
2491***Test Failed*** 1 failures.
2492(1, 4)
2493>>> t.summarize(verbose=1)
24941 items passed all tests:
2495 2 tests in example2
2496**********************************************************************
24971 items had failures:
2498 1 of 2 in XYZ
24994 tests in 2 items.
25003 passed and 1 failed.
2501***Test Failed*** 1 failures.
2502(1, 4)
2503"""
2504
2505def test2(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002506 >>> warnings.filterwarnings("ignore", "class Tester",
2507 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002508 >>> t = Tester(globs={}, verbose=1)
2509 >>> test = r'''
2510 ... # just an example
2511 ... >>> x = 1 + 2
2512 ... >>> x
2513 ... 3
2514 ... '''
2515 >>> t.runstring(test, "Example")
2516 Running string Example
2517 Trying: x = 1 + 2
2518 Expecting: nothing
2519 ok
2520 Trying: x
2521 Expecting: 3
2522 ok
2523 0 of 2 examples failed in string Example
2524 (0, 2)
2525"""
2526def test3(): r"""
Tim Peters3ddd60a2004-08-08 02:43:33 +00002527 >>> warnings.filterwarnings("ignore", "class Tester",
2528 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002529 >>> t = Tester(globs={}, verbose=0)
2530 >>> def _f():
2531 ... '''Trivial docstring example.
2532 ... >>> assert 2 == 2
2533 ... '''
2534 ... return 32
2535 ...
2536 >>> t.rundoc(_f) # expect 0 failures in 1 example
2537 (0, 1)
2538"""
2539def test4(): """
2540 >>> import new
2541 >>> m1 = new.module('_m1')
2542 >>> m2 = new.module('_m2')
2543 >>> test_data = \"""
2544 ... def _f():
2545 ... '''>>> assert 1 == 1
2546 ... '''
2547 ... def g():
2548 ... '''>>> assert 2 != 1
2549 ... '''
2550 ... class H:
2551 ... '''>>> assert 2 > 1
2552 ... '''
2553 ... def bar(self):
2554 ... '''>>> assert 1 < 2
2555 ... '''
2556 ... \"""
2557 >>> exec test_data in m1.__dict__
2558 >>> exec test_data in m2.__dict__
2559 >>> m1.__dict__.update({"f2": m2._f, "g2": m2.g, "h2": m2.H})
2560
2561 Tests that objects outside m1 are excluded:
2562
Tim Peters3ddd60a2004-08-08 02:43:33 +00002563 >>> warnings.filterwarnings("ignore", "class Tester",
2564 ... DeprecationWarning, "doctest", 0)
Tim Peters8485b562004-08-04 18:46:34 +00002565 >>> t = Tester(globs={}, verbose=0)
Tim Petersf727c6c2004-08-08 01:48:59 +00002566 >>> t.rundict(m1.__dict__, "rundict_test", m1) # f2 and g2 and h2 skipped
Tim Peters8485b562004-08-04 18:46:34 +00002567 (0, 4)
2568
Tim Petersf727c6c2004-08-08 01:48:59 +00002569 Once more, not excluding stuff outside m1:
Tim Peters8485b562004-08-04 18:46:34 +00002570
2571 >>> t = Tester(globs={}, verbose=0)
2572 >>> t.rundict(m1.__dict__, "rundict_test_pvt") # None are skipped.
2573 (0, 8)
2574
2575 The exclusion of objects from outside the designated module is
2576 meant to be invoked automagically by testmod.
2577
Tim Petersf727c6c2004-08-08 01:48:59 +00002578 >>> testmod(m1, verbose=False)
2579 (0, 4)
Tim Peters8485b562004-08-04 18:46:34 +00002580"""
Tim Peters8a7d2d52001-01-16 07:10:57 +00002581
2582def _test():
Tim Peters8485b562004-08-04 18:46:34 +00002583 #import doctest
2584 #doctest.testmod(doctest, verbose=False,
2585 # optionflags=ELLIPSIS | NORMALIZE_WHITESPACE |
2586 # UNIFIED_DIFF)
2587 #print '~'*70
2588 r = unittest.TextTestRunner()
2589 r.run(DocTestSuite())
Tim Peters8a7d2d52001-01-16 07:10:57 +00002590
2591if __name__ == "__main__":
2592 _test()