| \section{\module{unittest} --- |
| Unit testing framework} |
| |
| \declaremodule{standard}{unittest} |
| \modulesynopsis{Unit testing framework for Python.} |
| \moduleauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com} |
| \sectionauthor{Steve Purcell}{stephen\textunderscore{}purcell@yahoo.com} |
| \sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org} |
| \sectionauthor{Raymond Hettinger}{python@rcn.com} |
| |
| \versionadded{2.1} |
| |
| The Python unit testing framework, often referred to as ``PyUnit,'' is |
| a Python language version of JUnit, by Kent Beck and Erich Gamma. |
| JUnit is, in turn, a Java version of Kent's Smalltalk testing |
| framework. Each is the de facto standard unit testing framework for |
| its respective language. |
| |
| PyUnit supports test automation, sharing of setup and shutdown code |
| for tests, aggregation of tests into collections, and independence of |
| the tests from the reporting framework. The \module{unittest} module |
| provides classes that make it easy to support these qualities for a |
| set of tests. |
| |
| To achieve this, PyUnit supports some important concepts: |
| |
| \begin{definitions} |
| \term{test fixture} |
| A \dfn{test fixture} represents the preparation needed to perform one |
| or more tests, and any associate cleanup actions. This may involve, |
| for example, creating temporary or proxy databases, directories, or |
| starting a server process. |
| |
| \term{test case} |
| A \dfn{test case} is the smallest unit of testing. It checks for a |
| specific response to a particular set of inputs. PyUnit provides a |
| base class, \class{TestCase}, which may be used to create new test |
| cases. You may provide your own implementation that does not subclass |
| from \class{TestCase}, of course. |
| |
| \term{test suite} |
| A \dfn{test suite} is a collection of test cases, test suites, or |
| both. It is used to aggregate tests that should be executed |
| together. |
| |
| \term{test runner} |
| A \dfn{test runner} is a component which orchestrates the execution of |
| tests and provides the outcome to the user. The runner may use a |
| graphical interface, a textual interface, or return a special value to |
| indicate the results of executing the tests. |
| \end{definitions} |
| |
| |
| The test case and test fixture concepts are supported through the |
| \class{TestCase} and \class{FunctionTestCase} classes; the former |
| should be used when creating new tests, and the latter can be used when |
| integrating existing test code with a PyUnit-driven framework. When |
| building test fixtures using \class{TestCase}, the \method{setUp()} |
| and \method{tearDown()} methods can be overridden to provide |
| initialization and cleanup for the fixture. With |
| \class{FunctionTestCase}, existing functions can be passed to the |
| constructor for these purposes. When the test is run, the |
| fixture initialization is run first; if it succeeds, the cleanup |
| method is run after the test has been executed, regardless of the |
| outcome of the test. Each instance of the \class{TestCase} will only |
| be used to run a single test method, so a new fixture is created for |
| each test. |
| |
| Test suites are implemented by the \class{TestSuite} class. This |
| class allows individual tests and test suites to be aggregated; when |
| the suite is executed, all tests added directly to the suite and in |
| ``child'' test suites are run. |
| |
| A test runner is an object that provides a single method, |
| \method{run()}, which accepts a \class{TestCase} or \class{TestSuite} |
| object as a parameter, and returns a result object. The class |
| \class{TestResult} is provided for use as the result object. PyUnit |
| provide the \class{TextTestRunner} as an example test runner which |
| reports test results on the standard error stream by default. |
| Alternate runners can be implemented for other environments (such as |
| graphical environments) without any need to derive from a specific |
| class. |
| |
| |
| \begin{seealso} |
| \seemodule{doctest}{Another test-support module with a very |
| different flavor.} |
| \seetitle[http://pyunit.sourceforge.net/]{PyUnit Web Site}{The |
| source for further information on PyUnit.} |
| \seetitle[http://www.XProgramming.com/testfram.htm]{Simple Smalltalk |
| Testing: With Patterns}{Kent Beck's original paper on |
| testing frameworks using the pattern shared by |
| \module{unittest}.} |
| \end{seealso} |
| |
| |
| \subsection{Basic example \label{minimal-example}} |
| |
| The \module{unittest} module provides a rich set of tools for |
| constructing and running tests. This section demonstrates that a |
| small subset of the tools suffice to meet the needs of most users. |
| |
| Here is a short script to test three functions from the |
| \refmodule{random} module: |
| |
| \begin{verbatim} |
| import random |
| import unittest |
| |
| class TestSequenceFunctions(unittest.TestCase): |
| |
| def setUp(self): |
| self.seq = range(10) |
| |
| def testshuffle(self): |
| # make sure the shuffled sequence does not lose any elements |
| random.shuffle(self.seq) |
| self.seq.sort() |
| self.assertEqual(self.seq, range(10)) |
| |
| def testchoice(self): |
| element = random.choice(self.seq) |
| self.assert_(element in self.seq) |
| |
| def testsample(self): |
| self.assertRaises(ValueError, random.sample, self.seq, 20) |
| for element in random.sample(self.seq, 5): |
| self.assert_(element in self.seq) |
| |
| if __name__ == '__main__': |
| unittest.main() |
| \end{verbatim} |
| |
| A testcase is created by subclassing \class{unittest.TestCase}. |
| The three individual tests are defined with methods whose names start with |
| the letters \samp{test}. This naming convention informs the test runner |
| about which methods represent tests. |
| |
| The crux of each test is a call to \method{assertEqual()} to |
| check for an expected result; \method{assert_()} to verify a condition; |
| or \method{assertRaises()} to verify that an expected exception gets |
| raised. These methods are used instead of the \keyword{assert} statement |
| so the test runner can accumulate all test results and produce a report. |
| |
| When a \method{setUp()} method is defined, the test runner will run that |
| method prior to each test. Likewise, if a \method{tearDown()} method is |
| defined, the test runner will invoke that method after each test. In the |
| example, \method{setUp()} was used to create a fresh sequence for each test. |
| |
| The final block shows a simple way to run the tests. |
| \function{unittest.main()} provides a command line interface to the |
| test script. When run from the command line, the above script |
| produces an output that looks like this: |
| |
| \begin{verbatim} |
| ... |
| ---------------------------------------------------------------------- |
| Ran 3 tests in 0.000s |
| |
| OK |
| \end{verbatim} |
| |
| Instead of \function{unittest.main()}, there are other ways to run the tests |
| with a finer level of control, less terse output, and no requirement to be |
| run from the command line. For example, the last two lines may be replaced |
| with: |
| |
| \begin{verbatim} |
| suite = unittest.makeSuite(TestSequenceFunctions) |
| unittest.TextTestRunner(verbosity=2).run(suite) |
| \end{verbatim} |
| |
| Running the revised script from the interpreter or another script |
| produces the following output: |
| |
| \begin{verbatim} |
| testchoice (__main__.TestSequenceFunctions) ... ok |
| testsample (__main__.TestSequenceFunctions) ... ok |
| testshuffle (__main__.TestSequenceFunctions) ... ok |
| |
| ---------------------------------------------------------------------- |
| Ran 3 tests in 0.110s |
| |
| OK |
| \end{verbatim} |
| |
| The above examples show the most commonly used \module{unittest} features |
| which are sufficient to meet many everyday testing needs. The remainder |
| of the documentation explores the full feature set from first principles. |
| |
| |
| \subsection{Organizing test code |
| \label{organizing-tests}} |
| |
| The basic building blocks of unit testing are \dfn{test cases} --- |
| single scenarios that must be set up and checked for correctness. In |
| PyUnit, test cases are represented by instances of the |
| \class{TestCase} class in the \refmodule{unittest} module. To make |
| your own test cases you must write subclasses of \class{TestCase}, or |
| use \class{FunctionTestCase}. |
| |
| An instance of a \class{TestCase}-derived class is an object that can |
| completely run a single test method, together with optional set-up |
| and tidy-up code. |
| |
| The testing code of a \class{TestCase} instance should be entirely |
| self contained, such that it can be run either in isolation or in |
| arbitrary combination with any number of other test cases. |
| |
| The simplest test case subclass will simply override the |
| \method{runTest()} method in order to perform specific testing code: |
| |
| \begin{verbatim} |
| import unittest |
| |
| class DefaultWidgetSizeTestCase(unittest.TestCase): |
| def runTest(self): |
| widget = Widget("The widget") |
| self.failUnless(widget.size() == (50,50), 'incorrect default size') |
| \end{verbatim} |
| |
| Note that in order to test something, we use the one of the |
| \method{assert*()} or \method{fail*()} methods provided by the |
| \class{TestCase} base class. If the test fails when the test case |
| runs, an exception will be raised, and the testing framework will |
| identify the test case as a \dfn{failure}. Other exceptions that do |
| not arise from checks made through the \method{assert*()} and |
| \method{fail*()} methods are identified by the testing framework as |
| dfn{errors}. |
| |
| The way to run a test case will be described later. For now, note |
| that to construct an instance of such a test case, we call its |
| constructor without arguments: |
| |
| \begin{verbatim} |
| testCase = DefaultWidgetSizeTestCase() |
| \end{verbatim} |
| |
| Now, such test cases can be numerous, and their set-up can be |
| repetitive. In the above case, constructing a ``Widget'' in each of |
| 100 Widget test case subclasses would mean unsightly duplication. |
| |
| Luckily, we can factor out such set-up code by implementing a method |
| called \method{setUp()}, which the testing framework will |
| automatically call for us when we run the test: |
| |
| \begin{verbatim} |
| import unittest |
| |
| class SimpleWidgetTestCase(unittest.TestCase): |
| def setUp(self): |
| self.widget = Widget("The widget") |
| |
| class DefaultWidgetSizeTestCase(SimpleWidgetTestCase): |
| def runTest(self): |
| self.failUnless(self.widget.size() == (50,50), |
| 'incorrect default size') |
| |
| class WidgetResizeTestCase(SimpleWidgetTestCase): |
| def runTest(self): |
| self.widget.resize(100,150) |
| self.failUnless(self.widget.size() == (100,150), |
| 'wrong size after resize') |
| \end{verbatim} |
| |
| If the \method{setUp()} method raises an exception while the test is |
| running, the framework will consider the test to have suffered an |
| error, and the \method{runTest()} method will not be executed. |
| |
| Similarly, we can provide a \method{tearDown()} method that tidies up |
| after the \method{runTest()} method has been run: |
| |
| \begin{verbatim} |
| import unittest |
| |
| class SimpleWidgetTestCase(unittest.TestCase): |
| def setUp(self): |
| self.widget = Widget("The widget") |
| |
| def tearDown(self): |
| self.widget.dispose() |
| self.widget = None |
| \end{verbatim} |
| |
| If \method{setUp()} succeeded, the \method{tearDown()} method will be |
| run regardless of whether or not \method{runTest()} succeeded. |
| |
| Such a working environment for the testing code is called a |
| \dfn{fixture}. |
| |
| Often, many small test cases will use the same fixture. In this case, |
| we would end up subclassing \class{SimpleWidgetTestCase} into many |
| small one-method classes such as |
| \class{DefaultWidgetSizeTestCase}. This is time-consuming and |
| discouraging, so in the same vein as JUnit, PyUnit provides a simpler |
| mechanism: |
| |
| \begin{verbatim} |
| import unittest |
| |
| class WidgetTestCase(unittest.TestCase): |
| def setUp(self): |
| self.widget = Widget("The widget") |
| |
| def tearDown(self): |
| self.widget.dispose() |
| self.widget = None |
| |
| def testDefaultSize(self): |
| self.failUnless(self.widget.size() == (50,50), |
| 'incorrect default size') |
| |
| def testResize(self): |
| self.widget.resize(100,150) |
| self.failUnless(self.widget.size() == (100,150), |
| 'wrong size after resize') |
| \end{verbatim} |
| |
| Here we have not provided a \method{runTest()} method, but have |
| instead provided two different test methods. Class instances will now |
| each run one of the \method{test*()} methods, with \code{self.widget} |
| created and destroyed separately for each instance. When creating an |
| instance we must specify the test method it is to run. We do this by |
| passing the method name in the constructor: |
| |
| \begin{verbatim} |
| defaultSizeTestCase = WidgetTestCase("testDefaultSize") |
| resizeTestCase = WidgetTestCase("testResize") |
| \end{verbatim} |
| |
| Test case instances are grouped together according to the features |
| they test. PyUnit provides a mechanism for this: the \class{test |
| suite}, represented by the class \class{TestSuite} in the |
| \refmodule{unittest} module: |
| |
| \begin{verbatim} |
| widgetTestSuite = unittest.TestSuite() |
| widgetTestSuite.addTest(WidgetTestCase("testDefaultSize")) |
| widgetTestSuite.addTest(WidgetTestCase("testResize")) |
| \end{verbatim} |
| |
| For the ease of running tests, as we will see later, it is a good |
| idea to provide in each test module a callable object that returns a |
| pre-built test suite: |
| |
| \begin{verbatim} |
| def suite(): |
| suite = unittest.TestSuite() |
| suite.addTest(WidgetTestCase("testDefaultSize")) |
| suite.addTest(WidgetTestCase("testResize")) |
| return suite |
| \end{verbatim} |
| |
| or even: |
| |
| \begin{verbatim} |
| class WidgetTestSuite(unittest.TestSuite): |
| def __init__(self): |
| unittest.TestSuite.__init__(self,map(WidgetTestCase, |
| ("testDefaultSize", |
| "testResize"))) |
| \end{verbatim} |
| |
| (The latter is admittedly not for the faint-hearted!) |
| |
| Since it is a common pattern to create a \class{TestCase} subclass |
| with many similarly named test functions, there is a convenience |
| function called \function{makeSuite()} that constructs a test suite |
| that comprises all of the test cases in a test case class: |
| |
| \begin{verbatim} |
| suite = unittest.makeSuite(WidgetTestCase) |
| \end{verbatim} |
| |
| Note that when using the \function{makeSuite()} function, the order in |
| which the various test cases will be run by the test suite is the |
| order determined by sorting the test function names using the |
| \function{cmp()} built-in function. |
| |
| Often it is desirable to group suites of test cases together, so as to |
| run tests for the whole system at once. This is easy, since |
| \class{TestSuite} instances can be added to a \class{TestSuite} just |
| as \class{TestCase} instances can be added to a \class{TestSuite}: |
| |
| \begin{verbatim} |
| suite1 = module1.TheTestSuite() |
| suite2 = module2.TheTestSuite() |
| alltests = unittest.TestSuite((suite1, suite2)) |
| \end{verbatim} |
| |
| You can place the definitions of test cases and test suites in the |
| same modules as the code they are to test (such as \file{widget.py}), |
| but there are several advantages to placing the test code in a |
| separate module, such as \file{widgettests.py}: |
| |
| \begin{itemize} |
| \item The test module can be run standalone from the command line. |
| \item The test code can more easily be separated from shipped code. |
| \item There is less temptation to change test code to fit the code |
| it tests without a good reason. |
| \item Test code should be modified much less frequently than the |
| code it tests. |
| \item Tested code can be refactored more easily. |
| \item Tests for modules written in C must be in separate modules |
| anyway, so why not be consistent? |
| \item If the testing strategy changes, there is no need to change |
| the source code. |
| \end{itemize} |
| |
| |
| \subsection{Re-using old test code |
| \label{legacy-unit-tests}} |
| |
| Some users will find that they have existing test code that they would |
| like to run from PyUnit, without converting every old test function to |
| a \class{TestCase} subclass. |
| |
| For this reason, PyUnit provides a \class{FunctionTestCase} class. |
| This subclass of \class{TestCase} can be used to wrap an existing test |
| function. Set-up and tear-down functions can also optionally be |
| wrapped. |
| |
| Given the following test function: |
| |
| \begin{verbatim} |
| def testSomething(): |
| something = makeSomething() |
| assert something.name is not None |
| # ... |
| \end{verbatim} |
| |
| one can create an equivalent test case instance as follows: |
| |
| \begin{verbatim} |
| testcase = unittest.FunctionTestCase(testSomething) |
| \end{verbatim} |
| |
| If there are additional set-up and tear-down methods that should be |
| called as part of the test case's operation, they can also be provided: |
| |
| \begin{verbatim} |
| testcase = unittest.FunctionTestCase(testSomething, |
| setUp=makeSomethingDB, |
| tearDown=deleteSomethingDB) |
| \end{verbatim} |
| |
| \note{PyUnit supports the use of \exception{AssertionError} |
| as an indicator of test failure, but does not recommend it. Future |
| versions may treat \exception{AssertionError} differently.} |
| |
| |
| \subsection{Classes and functions |
| \label{unittest-contents}} |
| |
| \begin{classdesc}{TestCase}{} |
| Instances of the \class{TestCase} class represent the smallest |
| testable units in a set of tests. This class is intended to be used |
| as a base class, with specific tests being implemented by concrete |
| subclasses. This class implements the interface needed by the test |
| runner to allow it to drive the test, and methods that the test code |
| can use to check for and report various kinds of failures. |
| \end{classdesc} |
| |
| \begin{classdesc}{FunctionTestCase}{testFunc\optional{, |
| setUp\optional{, tearDown\optional{, description}}}} |
| This class implements the portion of the \class{TestCase} interface |
| which allows the test runner to drive the test, but does not provide |
| the methods which test code can use to check and report errors. |
| This is used to create test cases using legacy test code, allowing |
| it to be integrated into a \refmodule{unittest}-based test |
| framework. |
| \end{classdesc} |
| |
| \begin{classdesc}{TestSuite}{\optional{tests}} |
| This class represents an aggregation of individual tests cases and |
| test suites. The class presents the interface needed by the test |
| runner to allow it to be run as any other test case, but all the |
| contained tests and test suites are executed. Additional methods |
| are provided to add test cases and suites to the aggregation. If |
| \var{tests} is given, it must be a sequence of individual tests that |
| will be added to the suite. |
| \end{classdesc} |
| |
| \begin{classdesc}{TestLoader}{} |
| This class is responsible for loading tests according to various |
| criteria and returning them wrapped in a \class{TestSuite}. |
| It can load all tests within a given module or \class{TestCase} |
| class. When loading from a module, it considers all |
| \class{TestCase}-derived classes. For each such class, it creates |
| an instance for each method with a name beginning with the string |
| \samp{test}. |
| \end{classdesc} |
| |
| \begin{datadesc}{defaultTestLoader} |
| Instance of the \class{TestLoader} class which can be shared. If no |
| customization of the \class{TestLoader} is needed, this instance can |
| always be used instead of creating new instances. |
| \end{datadesc} |
| |
| \begin{classdesc}{TextTestRunner}{\optional{stream\optional{, |
| descriptions\optional{, verbosity}}}} |
| A basic test runner implementation which prints results on standard |
| output. It has a few configurable parameters, but is essentially |
| very simple. Graphical applications which run test suites should |
| provide alternate implementations. |
| \end{classdesc} |
| |
| \begin{funcdesc}{main}{\optional{module\optional{, |
| defaultTest\optional{, argv\optional{, |
| testRunner\optional{, testRunner}}}}}} |
| A command-line program that runs a set of tests; this is primarily |
| for making test modules conveniently executable. The simplest use |
| for this function is: |
| |
| \begin{verbatim} |
| if __name__ == '__main__': |
| unittest.main() |
| \end{verbatim} |
| \end{funcdesc} |
| |
| In some cases, the existing tests may have be written using the |
| \refmodule{doctest} module. If so, that module provides a |
| \class{DocTestSuite} class that can automatically build |
| \class{unittest.TestSuite} instances from the existing test code. |
| \versionadded{2.3} |
| |
| |
| \subsection{TestCase Objects |
| \label{testcase-objects}} |
| |
| Each \class{TestCase} instance represents a single test, but each |
| concrete subclass may be used to define multiple tests --- the |
| concrete class represents a single test fixture. The fixture is |
| created and cleaned up for each test case. |
| |
| \class{TestCase} instances provide three groups of methods: one group |
| used to run the test, another used by the test implementation to |
| check conditions and report failures, and some inquiry methods |
| allowing information about the test itself to be gathered. |
| |
| Methods in the first group are: |
| |
| \begin{methoddesc}[TestCase]{setUp}{} |
| Method called to prepare the test fixture. This is called |
| immediately before calling the test method; any exception raised by |
| this method will be considered an error rather than a test failure. |
| The default implementation does nothing. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{tearDown}{} |
| Method called immediately after the test method has been called and |
| the result recorded. This is called even if the test method raised |
| an exception, so the implementation in subclasses may need to be |
| particularly careful about checking internal state. Any exception |
| raised by this method will be considered an error rather than a test |
| failure. This method will only be called if the \method{setUp()} |
| succeeds, regardless of the outcome of the test method. |
| The default implementation does nothing. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{run}{\optional{result}} |
| Run the test, collecting the result into the test result object |
| passed as \var{result}. If \var{result} is omitted or \constant{None}, |
| a temporary result object is created and used, but is not made |
| available to the caller. This is equivalent to simply calling the |
| \class{TestCase} instance. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{debug}{} |
| Run the test without collecting the result. This allows exceptions |
| raised by the test to be propogated to the caller, and can be used |
| to support running tests under a debugger. |
| \end{methoddesc} |
| |
| |
| The test code can use any of the following methods to check for and |
| report failures. |
| |
| \begin{methoddesc}[TestCase]{assert_}{expr\optional{, msg}} |
| \methodline{failUnless}{expr\optional{, msg}} |
| Signal a test failure if \var{expr} is false; the explanation for |
| the error will be \var{msg} if given, otherwise it will be |
| \constant{None}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{assertEqual}{first, second\optional{, msg}} |
| \methodline{failUnlessEqual}{first, second\optional{, msg}} |
| Test that \var{first} and \var{second} are equal. If the values do |
| not compare equal, the test will fail with the explanation given by |
| \var{msg}, or \constant{None}. Note that using \method{failUnlessEqual()} |
| improves upon doing the comparison as the first parameter to |
| \method{failUnless()}: the default value for \var{msg} can be |
| computed to include representations of both \var{first} and |
| \var{second}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{assertNotEqual}{first, second\optional{, msg}} |
| \methodline{failIfEqual}{first, second\optional{, msg}} |
| Test that \var{first} and \var{second} are not equal. If the values |
| do compare equal, the test will fail with the explanation given by |
| \var{msg}, or \constant{None}. Note that using \method{failIfEqual()} |
| improves upon doing the comparison as the first parameter to |
| \method{failUnless()} is that the default value for \var{msg} can be |
| computed to include representations of both \var{first} and |
| \var{second}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{assertAlmostEqual}{first, second\optional{, |
| places\optional{, msg}}} |
| \methodline{failUnlessAlmostEqual}{first, second\optional{, |
| places\optional{, msg}}} |
| Test that \var{first} and \var{second} are approximately equal |
| by computing the difference, rounding to the given number of \var{places}, |
| and comparing to zero. Note that comparing a given number of decimal places |
| is not the same as comparing a given number of significant digits. |
| If the values do not compare equal, the test will fail with the explanation |
| given by \var{msg}, or \constant{None}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{assertNotAlmostEqual}{first, second\optional{, |
| places\optional{, msg}}} |
| \methodline{failIfAlmostEqual}{first, second\optional{, |
| places\optional{, msg}}} |
| Test that \var{first} and \var{second} are not approximately equal |
| by computing the difference, rounding to the given number of \var{places}, |
| and comparing to zero. Note that comparing a given number of decimal places |
| is not the same as comparing a given number of significant digits. |
| If the values do not compare equal, the test will fail with the explanation |
| given by \var{msg}, or \constant{None}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{assertRaises}{exception, callable, \moreargs} |
| \methodline{failUnlessRaises}{exception, callable, \moreargs} |
| Test that an exception is raised when \var{callable} is called with |
| any positional or keyword arguments that are also passed to |
| \method{assertRaises()}. The test passes if \var{exception} is |
| raised, is an error if another exception is raised, or fails if no |
| exception is raised. To catch any of a group of exceptions, a tuple |
| containing the exception classes may be passed as \var{exception}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{failIf}{expr\optional{, msg}} |
| The inverse of the \method{failUnless()} method is the |
| \method{failIf()} method. This signals a test failure if \var{expr} |
| is true, with \var{msg} or \constant{None} for the error message. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{fail}{\optional{msg}} |
| Signals a test failure unconditionally, with \var{msg} or |
| \constant{None} for the error message. |
| \end{methoddesc} |
| |
| \begin{memberdesc}[TestCase]{failureException} |
| This class attribute gives the exception raised by the |
| \method{test()} method. If a test framework needs to use a |
| specialized exception, possibly to carry additional information, it |
| must subclass this exception in order to ``play fair'' with the |
| framework. The initial value of this attribute is |
| \exception{AssertionError}. |
| \end{memberdesc} |
| |
| |
| Testing frameworks can use the following methods to collect |
| information on the test: |
| |
| \begin{methoddesc}[TestCase]{countTestCases}{} |
| Return the number of tests represented by the this test object. For |
| \class{TestCase} instances, this will always be \code{1}, but this |
| method is also implemented by the \class{TestSuite} class, which can |
| return larger values. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{defaultTestResult}{} |
| Return the default type of test result object to be used to run this |
| test. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{id}{} |
| Return a string identifying the specific test case. This is usually |
| the full name of the test method, including the module and class |
| names. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestCase]{shortDescription}{} |
| Returns a one-line description of the test, or \constant{None} if no |
| description has been provided. The default implementation of this |
| method returns the first line of the test method's docstring, if |
| available, or \constant{None}. |
| \end{methoddesc} |
| |
| |
| \subsection{TestSuite Objects |
| \label{testsuite-objects}} |
| |
| \class{TestSuite} objects behave much like \class{TestCase} objects, |
| except they do not actually implement a test. Instead, they are used |
| to aggregate tests into groups that should be run together. Some |
| additional methods are available to add tests to \class{TestSuite} |
| instances: |
| |
| \begin{methoddesc}[TestSuite]{addTest}{test} |
| Add a \class{TestCase} or \class{TestSuite} to the set of tests that |
| make up the suite. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestSuite]{addTests}{tests} |
| Add all the tests from a sequence of \class{TestCase} and |
| \class{TestSuite} instances to this test suite. |
| \end{methoddesc} |
| |
| The \method{run()} method is also slightly different: |
| |
| \begin{methoddesc}[TestSuite]{run}{result} |
| Run the tests associated with this suite, collecting the result into |
| the test result object passed as \var{result}. Note that unlike |
| \method{TestCase.run()}, \method{TestSuite.run()} requires the |
| result object to be passed in. |
| \end{methoddesc} |
| |
| In the typical usage of a \class{TestSuite} object, the \method{run()} |
| method is invoked by a \class{TestRunner} rather than by the end-user |
| test harness. |
| |
| |
| \subsection{TestResult Objects |
| \label{testresult-objects}} |
| |
| A \class{TestResult} object stores the results of a set of tests. The |
| \class{TestCase} and \class{TestSuite} classes ensure that results are |
| properly stored; test authors do not need to worry about recording the |
| outcome of tests. |
| |
| Testing frameworks built on top of \refmodule{unittest} may want |
| access to the \class{TestResult} object generated by running a set of |
| tests for reporting purposes; a \class{TestResult} instance is |
| returned by the \method{TestRunner.run()} method for this purpose. |
| |
| Each instance holds the total number of tests run, and collections of |
| failures and errors that occurred among those test runs. The |
| collections contain tuples of \code{(\var{testcase}, |
| \var{traceback})}, where \var{traceback} is a string containing a |
| formatted version of the traceback for the exception. |
| |
| \class{TestResult} instances have the following attributes that will |
| be of interest when inspecting the results of running a set of tests: |
| |
| \begin{memberdesc}[TestResult]{errors} |
| A list containing pairs of \class{TestCase} instances and the |
| formatted tracebacks for tests which raised an exception but did not |
| signal a test failure. |
| \versionchanged[Contains formatted tracebacks instead of |
| \function{sys.exc_info()} results]{2.2} |
| \end{memberdesc} |
| |
| \begin{memberdesc}[TestResult]{failures} |
| A list containing pairs of \class{TestCase} instances and the |
| formatted tracebacks for tests which signalled a failure in the code |
| under test. |
| \versionchanged[Contains formatted tracebacks instead of |
| \function{sys.exc_info()} results]{2.2} |
| \end{memberdesc} |
| |
| \begin{memberdesc}[TestResult]{testsRun} |
| The number of tests which have been started. |
| \end{memberdesc} |
| |
| \begin{methoddesc}[TestResult]{wasSuccessful}{} |
| Returns true if all tests run so far have passed, otherwise returns |
| false. |
| \end{methoddesc} |
| |
| |
| The following methods of the \class{TestResult} class are used to |
| maintain the internal data structures, and may be extended in |
| subclasses to support additional reporting requirements. This is |
| particularly useful in building tools which support interactive |
| reporting while tests are being run. |
| |
| \begin{methoddesc}[TestResult]{startTest}{test} |
| Called when the test case \var{test} is about to be run. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestResult]{stopTest}{test} |
| Called when the test case \var{test} has been executed, regardless |
| of the outcome. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestResult]{addError}{test, err} |
| Called when the test case \var{test} raises an exception without |
| signalling a test failure. \var{err} is a tuple of the form |
| returned by \function{sys.exc_info()}: \code{(\var{type}, |
| \var{value}, \var{traceback})}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestResult]{addFailure}{test, err} |
| Called when the test case \var{test} signals a failure. |
| \var{err} is a tuple of the form returned by |
| \function{sys.exc_info()}: \code{(\var{type}, \var{value}, |
| \var{traceback})}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestResult]{addSuccess}{test} |
| This method is called for a test that does not fail; \var{test} is |
| the test case object. |
| \end{methoddesc} |
| |
| |
| One additional method is available for \class{TestResult} objects: |
| |
| \begin{methoddesc}[TestResult]{stop}{} |
| This method can be called to signal that the set of tests being run |
| should be aborted. Once this has been called, the |
| \class{TestRunner} object return to its caller without running any |
| additional tests. This is used by the \class{TextTestRunner} class |
| to stop the test framework when the user signals an interrupt from |
| the keyboard. Interactive tools which provide runners can use this |
| in a similar manner. |
| \end{methoddesc} |
| |
| |
| \subsection{TestLoader Objects |
| \label{testloader-objects}} |
| |
| The \class{TestLoader} class is used to create test suites from |
| classes and modules. Normally, there is no need to create an instance |
| of this class; the \refmodule{unittest} module provides an instance |
| that can be shared as the \code{defaultTestLoader} module attribute. |
| Using a subclass or instance would allow customization of some |
| configurable properties. |
| |
| \class{TestLoader} objects have the following methods: |
| |
| \begin{methoddesc}[TestLoader]{loadTestsFromTestCase}{testCaseClass} |
| Return a suite of all tests cases contained in the |
| \class{TestCase}-derived class \class{testCaseClass}. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestLoader]{loadTestsFromModule}{module} |
| Return a suite of all tests cases contained in the given module. |
| This method searches \var{module} for classes derived from |
| \class{TestCase} and creates an instance of the class for each test |
| method defined for the class. |
| |
| \warning{While using a hierarchy of |
| \class{Testcase}-derived classes can be convenient in sharing |
| fixtures and helper functions, defining test methods on base classes |
| that are not intended to be instantiated directly does not play well |
| with this method. Doing so, however, can be useful when the |
| fixtures are different and defined in subclasses.} |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestLoader]{loadTestsFromName}{name\optional{, module}} |
| Return a suite of all tests cases given a string specifier. |
| |
| The specifier \var{name} is a ``dotted name'' that may resolve |
| either to a module, a test case class, a test method within a test |
| case class, or a callable object which returns a \class{TestCase} or |
| \class{TestSuite} instance. For example, if you have a module |
| \module{SampleTests} containing a \class{TestCase}-derived class |
| \class{SampleTestCase} with three test methods (\method{test_one()}, |
| \method{test_two()}, and \method{test_three()}), the specifier |
| \code{'SampleTests.SampleTestCase'} would cause this method to |
| return a suite which will run all three test methods. Using the |
| specifier \code{'SampleTests.SampleTestCase.test_two'} would cause |
| it to return a test suite which will run only the |
| \method{test_two()} test method. The specifier can refer to modules |
| and packages which have not been imported; they will be imported as |
| a side-effect. |
| |
| The method optionally resolves \var{name} relative to a given module. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestLoader]{loadTestsFromNames}{names\optional{, module}} |
| Similar to \method{loadTestsFromName()}, but takes a sequence of |
| names rather than a single name. The return value is a test suite |
| which supports all the tests defined for each name. |
| \end{methoddesc} |
| |
| \begin{methoddesc}[TestLoader]{getTestCaseNames}{testCaseClass} |
| Return a sorted sequence of method names found within |
| \var{testCaseClass}. |
| \end{methoddesc} |
| |
| |
| The following attributes of a \class{TestLoader} can be configured |
| either by subclassing or assignment on an instance: |
| |
| \begin{memberdesc}[TestLoader]{testMethodPrefix} |
| String giving the prefix of method names which will be interpreted |
| as test methods. The default value is \code{'test'}. |
| \end{memberdesc} |
| |
| \begin{memberdesc}[TestLoader]{sortTestMethodsUsing} |
| Function to be used to compare method names when sorting them in |
| \method{getTestCaseNames()}. The default value is the built-in |
| \function{cmp()} function; it can be set to \constant{None} to disable |
| the sort. |
| \end{memberdesc} |
| |
| \begin{memberdesc}[TestLoader]{suiteClass} |
| Callable object that constructs a test suite from a list of tests. |
| No methods on the resulting object are needed. The default value is |
| the \class{TestSuite} class. |
| \end{memberdesc} |