blob: 887833e11227a48fb24f4f4963aa75348c65a6b1 [file] [log] [blame]
Tim Peters8a7d2d52001-01-16 07:10:57 +00001# Module doctest version 0.9.6
2# Released to the public domain 16-Jan-2001,
3# by Tim Peters (tim.one@home.com).
4
5# Provided as-is; use at your own risk; no warranty; no promises; enjoy!
6
7"""Module doctest -- a framework for running examples in docstrings.
8
9NORMAL USAGE
10
11In normal use, end each module M with:
12
13def _test():
14 import doctest, M # replace M with your module's name
15 return doctest.testmod(M) # ditto
16
17if __name__ == "__main__":
18 _test()
19
20Then running the module as a script will cause the examples in the
21docstrings to get executed and verified:
22
23python M.py
24
25This won't display anything unless an example fails, in which case the
26failing example(s) and the cause(s) of the failure(s) are printed to stdout
27(why not stderr? because stderr is a lame hack <0.2 wink>), and the final
28line of output is "Test failed.".
29
30Run it with the -v switch instead:
31
32python M.py -v
33
34and a detailed report of all examples tried is printed to stdout, along
35with assorted summaries at the end.
36
37You can force verbose mode by passing "verbose=1" to testmod, or prohibit
38it by passing "verbose=0". In either of those cases, sys.argv is not
39examined by testmod.
40
41In any case, testmod returns a 2-tuple of ints (f, t), where f is the
42number of docstring examples that failed and t is the total number of
43docstring examples attempted.
44
45
46WHICH DOCSTRINGS ARE EXAMINED?
47
48+ M.__doc__.
49
50+ f.__doc__ for all functions f in M.__dict__.values(), except those
51 with private names.
52
53+ C.__doc__ for all classes C in M.__dict__.values(), except those with
54 private names.
55
56+ If M.__test__ exists and "is true", it must be a dict, and
57 each entry maps a (string) name to a function object, class object, or
58 string. Function and class object docstrings found from M.__test__
59 are searched even if the name is private, and strings are searched
60 directly as if they were docstrings. In output, a key K in M.__test__
61 appears with name
62 <name of M>.__test__.K
63
64Any classes found are recursively searched similarly, to test docstrings in
65their contained methods and nested classes. Private names reached from M's
66globals are skipped, but all names reached from M.__test__ are searched.
67
68By default, a name is considered to be private if it begins with an
69underscore (like "_my_func") but doesn't both begin and end with (at least)
70two underscores (like "__init__"). You can change the default by passing
71your own "isprivate" function to testmod.
72
73If you want to test docstrings in objects with private names too, stuff
74them into an M.__test__ dict, or see ADVANCED USAGE below (e.g., pass your
75own isprivate function to Tester's constructor, or call the rundoc method
76of a Tester instance).
77
78Warning: imports can cause trouble; e.g., if you do
79
80from XYZ import XYZclass
81
82then XYZclass is a name in M.__dict__ too, and doctest has no way to know
83that XYZclass wasn't *defined* in M. So it may try to execute the examples
84in XYZclass's docstring, and those in turn may require a different set of
85globals to work correctly. I prefer to do "import *"- friendly imports,
86a la
87
88import XYY
89_XYZclass = XYZ.XYZclass
90del XYZ
91
92or (Python 2.0)
93
94from XYZ import XYZclass as _XYZclass
95
96and then the leading underscore stops testmod from going nuts. You may
97prefer the method in the next section.
98
99
100WHAT'S THE EXECUTION CONTEXT?
101
102By default, each time testmod finds a docstring to test, it uses a *copy*
103of M's globals (so that running tests on a module doesn't change the
104module's real globals, and so that one test in M can't leave behind crumbs
105that accidentally allow another test to work). This means examples can
106freely use any names defined at top-level in M. It also means that sloppy
107imports (see above) can cause examples in external docstrings to use
108globals inappropriate for them.
109
110You can force use of your own dict as the execution context by passing
111"globs=your_dict" to testmod instead. Presumably this would be a copy of
112M.__dict__ merged with the globals from other imported modules.
113
114
115WHAT IF I WANT TO TEST A WHOLE PACKAGE?
116
117Piece o' cake, provided the modules do their testing from docstrings.
118Here's the test.py I use for the world's most elaborate Rational/
119floating-base-conversion pkg (which I'll distribute some day):
120
121from Rational import Cvt
122from Rational import Format
123from Rational import machprec
124from Rational import Rat
125from Rational import Round
126from Rational import utils
127
128modules = (Cvt,
129 Format,
130 machprec,
131 Rat,
132 Round,
133 utils)
134
135def _test():
136 import doctest
137 import sys
138 verbose = "-v" in sys.argv
139 for mod in modules:
140 doctest.testmod(mod, verbose=verbose, report=0)
141 doctest.master.summarize()
142
143if __name__ == "__main__":
144 _test()
145
146IOW, it just runs testmod on all the pkg modules. testmod remembers the
147names and outcomes (# of failures, # of tries) for each item it's seen, and
148passing "report=0" prevents it from printing a summary in verbose mode.
149Instead, the summary is delayed until all modules have been tested, and
150then "doctest.master.summarize()" forces the summary at the end.
151
152So this is very nice in practice: each module can be tested individually
153with almost no work beyond writing up docstring examples, and collections
154of modules can be tested too as a unit with no more work than the above.
155
156
157WHAT ABOUT EXCEPTIONS?
158
159No problem, as long as the only output generated by the example is the
160traceback itself. For example:
161
162 >>> 1/0
163 Traceback (innermost last):
164 File "<stdin>", line 1, in ?
165 ZeroDivisionError: integer division or modulo by zero
166 >>>
167
168Note that only the exception type and value are compared (specifically,
169only the last line in the traceback).
170
171
172ADVANCED USAGE
173
174doctest.testmod() captures the testing policy I find most useful most
175often. You may want other policies.
176
177testmod() actually creates a local instance of class doctest.Tester, runs
178appropriate methods of that class, and merges the results into global
179Tester instance doctest.master.
180
181You can create your own instances of doctest.Tester, and so build your own
182policies, or even run methods of doctest.master directly. See
183doctest.Tester.__doc__ for details.
184
185
186SO WHAT DOES A DOCSTRING EXAMPLE LOOK LIKE ALREADY!?
187
188Oh ya. It's easy! In most cases a copy-and-paste of an interactive
189console session works fine -- just make sure the leading whitespace is
190rigidly consistent (you can mix tabs and spaces if you're too lazy to do it
191right, but doctest is not in the business of guessing what you think a tab
192means).
193
194 >>> # comments are ignored
195 >>> x = 12
196 >>> x
197 12
198 >>> if x == 13:
199 ... print "yes"
200 ... else:
201 ... print "no"
202 ... print "NO"
203 ... print "NO!!!"
204 ...
205 no
206 NO
207 NO!!!
208 >>>
209
210Any expected output must immediately follow the final ">>>" or "..." line
211containing the code, and the expected output (if any) extends to the next
212">>>" or all-whitespace line. That's it.
213
214Bummers:
215
216+ Expected output cannot contain an all-whitespace line, since such a line
217 is taken to signal the end of expected output.
218
219+ Output to stdout is captured, but not output to stderr (exception
220 tracebacks are captured via a different means).
221
222+ If you continue a line via backslashing in an interactive session, or for
223 any other reason use a backslash, you need to double the backslash in the
224 docstring version. This is simply because you're in a string, and so the
225 backslash must be escaped for it to survive intact. Like:
226
227>>> if "yes" == \\
228... "y" + \\
229... "es": # in the source code you'll see the doubled backslashes
230... print 'yes'
231yes
232
233The starting column doesn't matter:
234
235>>> assert "Easy!"
236 >>> import math
237 >>> math.floor(1.9)
238 1.0
239
240and as many leading whitespace characters are stripped from the expected
241output as appeared in the initial ">>>" line that triggered it.
242
243If you execute this very file, the examples above will be found and
244executed, leading to this output in verbose mode:
245
246Running doctest.__doc__
247Trying: 1/0
248Expecting:
249Traceback (innermost last):
250 File "<stdin>", line 1, in ?
251ZeroDivisionError: integer division or modulo by zero
252ok
253Trying: x = 12
254Expecting: nothing
255ok
256Trying: x
257Expecting: 12
258ok
259Trying:
260if x == 13:
261 print "yes"
262else:
263 print "no"
264 print "NO"
265 print "NO!!!"
266Expecting:
267no
268NO
269NO!!!
270ok
271... and a bunch more like that, with this summary at the end:
272
2735 items had no tests:
274 doctest.Tester.__init__
275 doctest.Tester.run__test__
276 doctest.Tester.summarize
277 doctest.run_docstring_examples
278 doctest.testmod
27912 items passed all tests:
280 8 tests in doctest
281 6 tests in doctest.Tester
282 10 tests in doctest.Tester.merge
283 7 tests in doctest.Tester.rundict
284 3 tests in doctest.Tester.rundoc
285 3 tests in doctest.Tester.runstring
286 2 tests in doctest.__test__._TestClass
287 2 tests in doctest.__test__._TestClass.__init__
288 2 tests in doctest.__test__._TestClass.get
289 1 tests in doctest.__test__._TestClass.square
290 2 tests in doctest.__test__.string
291 7 tests in doctest.is_private
29253 tests in 17 items.
29353 passed and 0 failed.
294Test passed.
295"""
296
297# 0,0,1 06-Mar-1999
298# initial version posted
299# 0,0,2 06-Mar-1999
300# loosened parsing:
301# cater to stinkin' tabs
302# don't insist on a blank after PS2 prefix
303# so trailing "... " line from a compound stmt no longer
304# breaks if the file gets whitespace-trimmed
305# better error msgs for inconsistent leading whitespace
306# 0,9,1 08-Mar-1999
307# exposed the Tester class and added client methods
308# plus docstring examples of their use (eww - head-twisting!)
309# fixed logic error in reporting total # of tests & failures
310# added __test__ support to testmod (a pale reflection of Christian
311# Tismer's vision ...)
312# removed the "deep" argument; fiddle __test__ instead
313# simplified endcase logic for extracting tests, and running them.
314# before, if no output was expected but some was produced
315# anyway via an eval'ed result, the discrepancy wasn't caught
316# made TestClass private and used __test__ to get at it
317# many doc updates
318# speed _SpoofOut for long expected outputs
319# 0,9,2 09-Mar-1999
320# throw out comments from examples, enabling use of the much simpler
321# exec compile(... "single") ...
322# for simulating the runtime; that barfs on comment-only lines
323# used the traceback module to do a much better job of reporting
324# exceptions
325# run __doc__ values thru str(), "just in case"
326# privateness of names now determined by an overridable "isprivate"
327# function
328# by default a name now considered to be private iff it begins with
329# an underscore but doesn't both begin & end with two of 'em; so
330# e.g. Class.__init__ etc are searched now -- as they always
331# should have been
332# 0,9,3 18-Mar-1999
333# added .flush stub to _SpoofOut (JPython buglet diagnosed by
334# Hugh Emberson)
335# repaired ridiculous docs about backslashes in examples
336# minor internal changes
337# changed source to Unix line-end conventions
338# moved __test__ logic into new Tester.run__test__ method
339# 0,9,4 27-Mar-1999
340# report item name and line # in failing examples
341# 0,9,5 29-Jun-1999
342# allow straightforward exceptions in examples - thanks to Mark Hammond!
343# 0,9,6 16-Jan-2001
344# fiddling for changes in Python 2.0: some of the embedded docstring
345# examples no longer worked *exactly* as advertised, due to minor
346# language changes, and running doctest on itself pointed that out.
347# Hard to think of a better example of why this is useful <wink>.
348
349__version__ = 0, 9, 6
350
351import types
352_FunctionType = types.FunctionType
353_ClassType = types.ClassType
354_ModuleType = types.ModuleType
355_StringType = types.StringType
356del types
357
358import string
359_string_find = string.find
360_string_join = string.join
361_string_split = string.split
362_string_rindex = string.rindex
363del string
364
365import re
366PS1 = ">>>"
367PS2 = "..."
368_isPS1 = re.compile(r"(\s*)" + re.escape(PS1)).match
369_isPS2 = re.compile(r"(\s*)" + re.escape(PS2)).match
370_isEmpty = re.compile(r"\s*$").match
371_isComment = re.compile(r"\s*#").match
372del re
373
Skip Montanaroeccd02a2001-01-20 23:34:12 +0000374__all__ = []
375
Tim Peters8a7d2d52001-01-16 07:10:57 +0000376# Extract interactive examples from a string. Return a list of triples,
377# (source, outcome, lineno). "source" is the source code, and ends
378# with a newline iff the source spans more than one line. "outcome" is
379# the expected output if any, else an empty string. When not empty,
380# outcome always ends with a newline. "lineno" is the line number,
381# 0-based wrt the start of the string, of the first source line.
382
383def _extract_examples(s):
384 isPS1, isPS2 = _isPS1, _isPS2
385 isEmpty, isComment = _isEmpty, _isComment
386 examples = []
387 lines = _string_split(s, "\n")
388 i, n = 0, len(lines)
389 while i < n:
390 line = lines[i]
391 i = i + 1
392 m = isPS1(line)
393 if m is None:
394 continue
395 j = m.end(0) # beyond the prompt
396 if isEmpty(line, j) or isComment(line, j):
397 # a bare prompt or comment -- not interesting
398 continue
399 lineno = i - 1
400 if line[j] != " ":
401 raise ValueError("line " + `lineno` + " of docstring lacks "
402 "blank after " + PS1 + ": " + line)
403 j = j + 1
404 blanks = m.group(1)
405 nblanks = len(blanks)
406 # suck up this and following PS2 lines
407 source = []
408 while 1:
409 source.append(line[j:])
410 line = lines[i]
411 m = isPS2(line)
412 if m:
413 if m.group(1) != blanks:
414 raise ValueError("inconsistent leading whitespace "
415 "in line " + `i` + " of docstring: " + line)
416 i = i + 1
417 else:
418 break
419 if len(source) == 1:
420 source = source[0]
421 else:
422 # get rid of useless null line from trailing empty "..."
423 if source[-1] == "":
424 del source[-1]
425 source = _string_join(source, "\n") + "\n"
426 # suck up response
427 if isPS1(line) or isEmpty(line):
428 expect = ""
429 else:
430 expect = []
431 while 1:
432 if line[:nblanks] != blanks:
433 raise ValueError("inconsistent leading whitespace "
434 "in line " + `i` + " of docstring: " + line)
435 expect.append(line[nblanks:])
436 i = i + 1
437 line = lines[i]
438 if isPS1(line) or isEmpty(line):
439 break
440 expect = _string_join(expect, "\n") + "\n"
441 examples.append( (source, expect, lineno) )
442 return examples
443
444# Capture stdout when running examples.
445
446class _SpoofOut:
447 def __init__(self):
448 self.clear()
449 def write(self, s):
450 self.buf.append(s)
451 def get(self):
452 return _string_join(self.buf, "")
453 def clear(self):
454 self.buf = []
455 def flush(self):
456 # JPython calls flush
457 pass
458
459# Display some tag-and-msg pairs nicely, keeping the tag and its msg
460# on the same line when that makes sense.
461
462def _tag_out(printer, *tag_msg_pairs):
463 for tag, msg in tag_msg_pairs:
464 printer(tag + ":")
465 msg_has_nl = msg[-1:] == "\n"
466 msg_has_two_nl = msg_has_nl and \
467 _string_find(msg, "\n") < len(msg) - 1
468 if len(tag) + len(msg) < 76 and not msg_has_two_nl:
469 printer(" ")
470 else:
471 printer("\n")
472 printer(msg)
473 if not msg_has_nl:
474 printer("\n")
475
476# Run list of examples, in context globs. "out" can be used to display
477# stuff to "the real" stdout, and fakeout is an instance of _SpoofOut
478# that captures the examples' std output. Return (#failures, #tries).
479
480def _run_examples_inner(out, fakeout, examples, globs, verbose, name):
481 import sys, traceback
482 OK, BOOM, FAIL = range(3)
483 NADA = "nothing"
484 stderr = _SpoofOut()
485 failures = 0
486 for source, want, lineno in examples:
487 if verbose:
488 _tag_out(out, ("Trying", source),
489 ("Expecting", want or NADA))
490 fakeout.clear()
491 try:
492 exec compile(source, "<string>", "single") in globs
493 got = fakeout.get()
494 state = OK
495 except:
496 # See whether the exception was expected.
497 if _string_find(want, "Traceback (innermost last):\n") == 0:
498 # Only compare exception type and value - the rest of
499 # the traceback isn't necessary.
500 want = _string_split(want, '\n')[-2] + '\n'
501 exc_type, exc_val, exc_tb = sys.exc_info()
502 got = traceback.format_exception_only(exc_type, exc_val)[0]
503 state = OK
504 else:
505 # unexpected exception
506 stderr.clear()
507 traceback.print_exc(file=stderr)
508 state = BOOM
509
510 if state == OK:
511 if got == want:
512 if verbose:
513 out("ok\n")
514 continue
515 state = FAIL
516
517 assert state in (FAIL, BOOM)
518 failures = failures + 1
519 out("*" * 65 + "\n")
520 _tag_out(out, ("Failure in example", source))
521 out("from line #" + `lineno` + " of " + name + "\n")
522 if state == FAIL:
523 _tag_out(out, ("Expected", want or NADA), ("Got", got))
524 else:
525 assert state == BOOM
526 _tag_out(out, ("Exception raised", stderr.get()))
527
528 return failures, len(examples)
529
530# Run list of examples, in context globs. Return (#failures, #tries).
531
532def _run_examples(examples, globs, verbose, name):
533 import sys
534 saveout = sys.stdout
535 try:
536 sys.stdout = fakeout = _SpoofOut()
537 x = _run_examples_inner(saveout.write, fakeout, examples,
538 globs, verbose, name)
539 finally:
540 sys.stdout = saveout
541 return x
542
543def run_docstring_examples(f, globs, verbose=0, name="NoName"):
544 """f, globs, verbose=0, name="NoName" -> run examples from f.__doc__.
545
546 Use dict globs as the globals for execution.
547 Return (#failures, #tries).
548
549 If optional arg verbose is true, print stuff even if there are no
550 failures.
551 Use string name in failure msgs.
552 """
553
554 try:
555 doc = f.__doc__
556 if not doc:
557 # docstring empty or None
558 return 0, 0
559 # just in case CT invents a doc object that has to be forced
560 # to look like a string <0.9 wink>
561 doc = str(doc)
562 except:
563 return 0, 0
564
565 e = _extract_examples(doc)
566 if not e:
567 return 0, 0
568 return _run_examples(e, globs, verbose, name)
569
570def is_private(prefix, base):
571 """prefix, base -> true iff name prefix + "." + base is "private".
572
573 Prefix may be an empty string, and base does not contain a period.
574 Prefix is ignored (although functions you write conforming to this
575 protocol may make use of it).
576 Return true iff base begins with an (at least one) underscore, but
577 does not both begin and end with (at least) two underscores.
578
579 >>> is_private("a.b", "my_func")
580 0
581 >>> is_private("____", "_my_func")
582 1
583 >>> is_private("someclass", "__init__")
584 0
585 >>> is_private("sometypo", "__init_")
586 1
587 >>> is_private("x.y.z", "_")
588 1
589 >>> is_private("_x.y.z", "__")
590 0
591 >>> is_private("", "") # senseless but consistent
592 0
593 """
594
595 return base[:1] == "_" and not base[:2] == "__" == base[-2:]
596
597class Tester:
598 """Class Tester -- runs docstring examples and accumulates stats.
599
600In normal use, function doctest.testmod() hides all this from you,
601so use that if you can. Create your own instances of Tester to do
602fancier things.
603
604Methods:
605 runstring(s, name)
606 Search string s for examples to run; use name for logging.
607 Return (#failures, #tries).
608
609 rundoc(object, name=None)
610 Search object.__doc__ for examples to run; use name (or
611 object.__name__) for logging. Return (#failures, #tries).
612
613 rundict(d, name)
614 Search for examples in docstrings in all of d.values(); use name
615 for logging. Return (#failures, #tries).
616
617 run__test__(d, name)
618 Treat dict d like module.__test__. Return (#failures, #tries).
619
620 summarize(verbose=None)
621 Display summary of testing results, to stdout. Return
622 (#failures, #tries).
623
624 merge(other)
625 Merge in the test results from Tester instance "other".
626
627>>> from doctest import Tester
628>>> t = Tester(globs={'x': 42}, verbose=0)
629>>> t.runstring(r'''
630... >>> x = x * 2
631... >>> print x
632... 42
633... ''', 'XYZ')
634*****************************************************************
635Failure in example: print x
636from line #2 of XYZ
637Expected: 42
638Got: 84
639(1, 2)
640>>> t.runstring(">>> x = x * 2\\n>>> print x\\n84\\n", 'example2')
641(0, 2)
642>>> t.summarize()
6431 items had failures:
644 1 of 2 in XYZ
645***Test Failed*** 1 failures.
646(1, 4)
647>>> t.summarize(verbose=1)
6481 items passed all tests:
649 2 tests in example2
6501 items had failures:
651 1 of 2 in XYZ
6524 tests in 2 items.
6533 passed and 1 failed.
654***Test Failed*** 1 failures.
655(1, 4)
656>>>
657"""
658
659 def __init__(self, mod=None, globs=None, verbose=None,
660 isprivate=None):
661 """mod=None, globs=None, verbose=None, isprivate=None
662
663See doctest.__doc__ for an overview.
664
665Optional keyword arg "mod" is a module, whose globals are used for
666executing examples. If not specified, globs must be specified.
667
668Optional keyword arg "globs" gives a dict to be used as the globals
669when executing examples; if not specified, use the globals from
670module mod.
671
672In either case, a copy of the dict is used for each docstring
673examined.
674
675Optional keyword arg "verbose" prints lots of stuff if true, only
676failures if false; by default, it's true iff "-v" is in sys.argv.
677
678Optional keyword arg "isprivate" specifies a function used to determine
679whether a name is private. The default function is doctest.is_private;
680see its docs for details.
681"""
682
683 if mod is None and globs is None:
684 raise TypeError("Tester.__init__: must specify mod or globs")
685 if mod is not None and type(mod) is not _ModuleType:
686 raise TypeError("Tester.__init__: mod must be a module; " +
687 `mod`)
688 if globs is None:
689 globs = mod.__dict__
690 self.globs = globs
691
692 if verbose is None:
693 import sys
694 verbose = "-v" in sys.argv
695 self.verbose = verbose
696
697 if isprivate is None:
698 isprivate = is_private
699 self.isprivate = isprivate
700
701 self.name2ft = {} # map name to (#failures, #trials) pair
702
703 def runstring(self, s, name):
704 """
705 s, name -> search string s for examples to run, logging as name.
706
707 Use string name as the key for logging the outcome.
708 Return (#failures, #examples).
709
710 >>> t = Tester(globs={}, verbose=1)
711 >>> test = r'''
712 ... # just an example
713 ... >>> x = 1 + 2
714 ... >>> x
715 ... 3
716 ... '''
717 >>> t.runstring(test, "Example")
718 Running string Example
719 Trying: x = 1 + 2
720 Expecting: nothing
721 ok
722 Trying: x
723 Expecting: 3
724 ok
725 0 of 2 examples failed in string Example
726 (0, 2)
727 """
728
729 if self.verbose:
730 print "Running string", name
731 f = t = 0
732 e = _extract_examples(s)
733 if e:
734 f, t = _run_examples(e, self.globs.copy(), self.verbose, name)
735 if self.verbose:
736 print f, "of", t, "examples failed in string", name
737 self.__record_outcome(name, f, t)
738 return f, t
739
740 def rundoc(self, object, name=None):
741 """
742 object, name=None -> search object.__doc__ for examples to run.
743
744 Use optional string name as the key for logging the outcome;
745 by default use object.__name__.
746 Return (#failures, #examples).
747 If object is a class object, search recursively for method
748 docstrings too.
749 object.__doc__ is examined regardless of name, but if object is
750 a class, whether private names reached from object are searched
751 depends on the constructor's "isprivate" argument.
752
753 >>> t = Tester(globs={}, verbose=0)
754 >>> def _f():
755 ... '''Trivial docstring example.
756 ... >>> assert 2 == 2
757 ... '''
758 ... return 32
759 ...
760 >>> t.rundoc(_f) # expect 0 failures in 1 example
761 (0, 1)
762 """
763
764 if name is None:
765 try:
766 name = object.__name__
767 except AttributeError:
768 raise ValueError("Tester.rundoc: name must be given "
769 "when object.__name__ doesn't exist; " + `object`)
770 if self.verbose:
771 print "Running", name + ".__doc__"
772 f, t = run_docstring_examples(object, self.globs.copy(),
773 self.verbose, name)
774 if self.verbose:
775 print f, "of", t, "examples failed in", name + ".__doc__"
776 self.__record_outcome(name, f, t)
777 if type(object) is _ClassType:
778 f2, t2 = self.rundict(object.__dict__, name)
779 f = f + f2
780 t = t + t2
781 return f, t
782
783 def rundict(self, d, name):
784 """
785 d. name -> search for docstring examples in all of d.values().
786
787 For k, v in d.items() such that v is a function or class,
788 do self.rundoc(v, name + "." + k). Whether this includes
789 objects with private names depends on the constructor's
790 "isprivate" argument.
791 Return aggregate (#failures, #examples).
792
793 >>> def _f():
794 ... '''>>> assert 1 == 1
795 ... '''
796 >>> def g():
797 ... '''>>> assert 2 != 1
798 ... '''
799 >>> d = {"_f": _f, "g": g}
800 >>> t = Tester(globs={}, verbose=0)
801 >>> t.rundict(d, "rundict_test") # _f is skipped
802 (0, 1)
803 >>> t = Tester(globs={}, verbose=0, isprivate=lambda x,y: 0)
804 >>> t.rundict(d, "rundict_test_pvt") # both are searched
805 (0, 2)
806 """
807
808 if not hasattr(d, "items"):
809 raise TypeError("Tester.rundict: d must support .items(); " +
810 `d`)
811 f = t = 0
812 for thisname, value in d.items():
813 if type(value) in (_FunctionType, _ClassType):
814 f2, t2 = self.__runone(value, name + "." + thisname)
815 f = f + f2
816 t = t + t2
817 return f, t
818
819 def run__test__(self, d, name):
820 """d, name -> Treat dict d like module.__test__.
821
822 Return (#failures, #tries).
823 See testmod.__doc__ for details.
824 """
825
826 failures = tries = 0
827 prefix = name + "."
828 savepvt = self.isprivate
829 try:
830 self.isprivate = lambda *args: 0
831 for k, v in d.items():
832 thisname = prefix + k
833 if type(v) is _StringType:
834 f, t = self.runstring(v, thisname)
835 elif type(v) in (_FunctionType, _ClassType):
836 f, t = self.rundoc(v, thisname)
837 else:
838 raise TypeError("Tester.run__test__: values in "
839 "dict must be strings, functions "
840 "or classes; " + `v`)
841 failures = failures + f
842 tries = tries + t
843 finally:
844 self.isprivate = savepvt
845 return failures, tries
846
847 def summarize(self, verbose=None):
848 """
849 verbose=None -> summarize results, return (#failures, #tests).
850
851 Print summary of test results to stdout.
852 Optional arg 'verbose' controls how wordy this is. By
853 default, use the verbose setting established by the
854 constructor.
855 """
856
857 if verbose is None:
858 verbose = self.verbose
859 notests = []
860 passed = []
861 failed = []
862 totalt = totalf = 0
863 for x in self.name2ft.items():
864 name, (f, t) = x
865 assert f <= t
866 totalt = totalt + t
867 totalf = totalf + f
868 if t == 0:
869 notests.append(name)
870 elif f == 0:
871 passed.append( (name, t) )
872 else:
873 failed.append(x)
874 if verbose:
875 if notests:
876 print len(notests), "items had no tests:"
877 notests.sort()
878 for thing in notests:
879 print " ", thing
880 if passed:
881 print len(passed), "items passed all tests:"
882 passed.sort()
883 for thing, count in passed:
884 print " %3d tests in %s" % (count, thing)
885 if failed:
886 print len(failed), "items had failures:"
887 failed.sort()
888 for thing, (f, t) in failed:
889 print " %3d of %3d in %s" % (f, t, thing)
890 if verbose:
891 print totalt, "tests in", len(self.name2ft), "items."
892 print totalt - totalf, "passed and", totalf, "failed."
893 if totalf:
894 print "***Test Failed***", totalf, "failures."
895 elif verbose:
896 print "Test passed."
897 return totalf, totalt
898
899 def merge(self, other):
900 """
901 other -> merge in test results from the other Tester instance.
902
903 If self and other both have a test result for something
904 with the same name, the (#failures, #tests) results are
905 summed, and a warning is printed to stdout.
906
907 >>> from doctest import Tester
908 >>> t1 = Tester(globs={}, verbose=0)
909 >>> t1.runstring('''
910 ... >>> x = 12
911 ... >>> print x
912 ... 12
913 ... ''', "t1example")
914 (0, 2)
915 >>>
916 >>> t2 = Tester(globs={}, verbose=0)
917 >>> t2.runstring('''
918 ... >>> x = 13
919 ... >>> print x
920 ... 13
921 ... ''', "t2example")
922 (0, 2)
923 >>> common = ">>> assert 1 + 2 == 3\\n"
924 >>> t1.runstring(common, "common")
925 (0, 1)
926 >>> t2.runstring(common, "common")
927 (0, 1)
928 >>> t1.merge(t2)
929 *** Tester.merge: 'common' in both testers; summing outcomes.
930 >>> t1.summarize(1)
931 3 items passed all tests:
932 2 tests in common
933 2 tests in t1example
934 2 tests in t2example
935 6 tests in 3 items.
936 6 passed and 0 failed.
937 Test passed.
938 (0, 6)
939 >>>
940 """
941
942 d = self.name2ft
943 for name, (f, t) in other.name2ft.items():
944 if d.has_key(name):
945 print "*** Tester.merge: '" + name + "' in both" \
946 " testers; summing outcomes."
947 f2, t2 = d[name]
948 f = f + f2
949 t = t + t2
950 d[name] = f, t
951
952 def __record_outcome(self, name, f, t):
953 if self.name2ft.has_key(name):
954 print "*** Warning: '" + name + "' was tested before;", \
955 "summing outcomes."
956 f2, t2 = self.name2ft[name]
957 f = f + f2
958 t = t + t2
959 self.name2ft[name] = f, t
960
961 def __runone(self, target, name):
962 if "." in name:
963 i = _string_rindex(name, ".")
964 prefix, base = name[:i], name[i+1:]
965 else:
966 prefix, base = "", base
967 if self.isprivate(prefix, base):
968 return 0, 0
969 return self.rundoc(target, name)
970
971master = None
972
973def testmod(m, name=None, globs=None, verbose=None, isprivate=None,
974 report=1):
975 """m, name=None, globs=None, verbose=None, isprivate=None, report=1
976
977 Test examples in docstrings in functions and classes reachable from
978 module m, starting with m.__doc__. Private names are skipped.
979
980 Also test examples reachable from dict m.__test__ if it exists and is
981 not None. m.__dict__ maps names to functions, classes and strings;
982 function and class docstrings are tested even if the name is private;
983 strings are tested directly, as if they were docstrings.
984
985 Return (#failures, #tests).
986
987 See doctest.__doc__ for an overview.
988
989 Optional keyword arg "name" gives the name of the module; by default
990 use m.__name__.
991
992 Optional keyword arg "globs" gives a dict to be used as the globals
993 when executing examples; by default, use m.__dict__. A copy of this
994 dict is actually used for each docstring, so that each docstring's
995 examples start with a clean slate.
996
997 Optional keyword arg "verbose" prints lots of stuff if true, prints
998 only failures if false; by default, it's true iff "-v" is in sys.argv.
999
1000 Optional keyword arg "isprivate" specifies a function used to
1001 determine whether a name is private. The default function is
1002 doctest.is_private; see its docs for details.
1003
1004 Optional keyword arg "report" prints a summary at the end when true,
1005 else prints nothing at the end. In verbose mode, the summary is
1006 detailed, else very brief (in fact, empty if all tests passed).
1007
1008 Advanced tomfoolery: testmod runs methods of a local instance of
1009 class doctest.Tester, then merges the results into (or creates)
1010 global Tester instance doctest.master. Methods of doctest.master
1011 can be called directly too, if you want to do something unusual.
1012 Passing report=0 to testmod is especially useful then, to delay
1013 displaying a summary. Invoke doctest.master.summarize(verbose)
1014 when you're done fiddling.
1015 """
1016
1017 global master
1018
1019 if type(m) is not _ModuleType:
1020 raise TypeError("testmod: module required; " + `m`)
1021 if name is None:
1022 name = m.__name__
1023 tester = Tester(m, globs=globs, verbose=verbose, isprivate=isprivate)
1024 failures, tries = tester.rundoc(m, name)
1025 f, t = tester.rundict(m.__dict__, name)
1026 failures = failures + f
1027 tries = tries + t
1028 if hasattr(m, "__test__"):
1029 testdict = m.__test__
1030 if testdict:
1031 if not hasattr(testdict, "items"):
1032 raise TypeError("testmod: module.__test__ must support "
1033 ".items(); " + `testdict`)
1034 f, t = tester.run__test__(testdict, name + ".__test__")
1035 failures = failures + f
1036 tries = tries + t
1037 if report:
1038 tester.summarize()
1039 if master is None:
1040 master = tester
1041 else:
1042 master.merge(tester)
1043 return failures, tries
1044
1045class _TestClass:
1046 """
1047 A pointless class, for sanity-checking of docstring testing.
1048
1049 Methods:
1050 square()
1051 get()
1052
1053 >>> _TestClass(13).get() + _TestClass(-12).get()
1054 1
1055 >>> hex(_TestClass(13).square().get())
1056 '0xa9'
1057 """
1058
1059 def __init__(self, val):
1060 """val -> _TestClass object with associated value val.
1061
1062 >>> t = _TestClass(123)
1063 >>> print t.get()
1064 123
1065 """
1066
1067 self.val = val
1068
1069 def square(self):
1070 """square() -> square TestClass's associated value
1071
1072 >>> _TestClass(13).square().get()
1073 169
1074 """
1075
1076 self.val = self.val ** 2
1077 return self
1078
1079 def get(self):
1080 """get() -> return TestClass's associated value.
1081
1082 >>> x = _TestClass(-42)
1083 >>> print x.get()
1084 -42
1085 """
1086
1087 return self.val
1088
1089__test__ = {"_TestClass": _TestClass,
1090 "string": r"""
1091 Example of a string object, searched as-is.
1092 >>> x = 1; y = 2
1093 >>> x + y, x * y
1094 (3, 2)
1095 """
1096 }
1097
1098def _test():
1099 import doctest
1100 return doctest.testmod(doctest)
1101
1102if __name__ == "__main__":
1103 _test()