Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`unittest` --- Unit testing framework |
| 2 | ========================================== |
| 3 | |
| 4 | .. module:: unittest |
| 5 | :synopsis: Unit testing framework for Python. |
| 6 | .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com> |
| 7 | .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com> |
| 8 | .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 9 | .. sectionauthor:: Raymond Hettinger <python@rcn.com> |
| 10 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 11 | (If you are already familiar with the basic concepts of testing, you might want |
| 12 | to skip to :ref:`the list of assert methods <assert-methods>`.) |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 13 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 14 | The Python unit testing framework, sometimes referred to as "PyUnit," is a |
| 15 | Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in |
| 16 | turn, a Java version of Kent's Smalltalk testing framework. Each is the de |
| 17 | facto standard unit testing framework for its respective language. |
| 18 | |
| 19 | :mod:`unittest` supports test automation, sharing of setup and shutdown code for |
| 20 | tests, aggregation of tests into collections, and independence of the tests from |
| 21 | the reporting framework. The :mod:`unittest` module provides classes that make |
| 22 | it easy to support these qualities for a set of tests. |
| 23 | |
| 24 | To achieve this, :mod:`unittest` supports some important concepts: |
| 25 | |
| 26 | test fixture |
| 27 | A :dfn:`test fixture` represents the preparation needed to perform one or more |
| 28 | tests, and any associate cleanup actions. This may involve, for example, |
| 29 | creating temporary or proxy databases, directories, or starting a server |
| 30 | process. |
| 31 | |
| 32 | test case |
| 33 | A :dfn:`test case` is the smallest unit of testing. It checks for a specific |
| 34 | response to a particular set of inputs. :mod:`unittest` provides a base class, |
| 35 | :class:`TestCase`, which may be used to create new test cases. |
| 36 | |
| 37 | test suite |
| 38 | A :dfn:`test suite` is a collection of test cases, test suites, or both. It is |
| 39 | used to aggregate tests that should be executed together. |
| 40 | |
| 41 | test runner |
| 42 | A :dfn:`test runner` is a component which orchestrates the execution of tests |
| 43 | and provides the outcome to the user. The runner may use a graphical interface, |
| 44 | a textual interface, or return a special value to indicate the results of |
| 45 | executing the tests. |
| 46 | |
| 47 | The test case and test fixture concepts are supported through the |
| 48 | :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be |
| 49 | used when creating new tests, and the latter can be used when integrating |
| 50 | existing test code with a :mod:`unittest`\ -driven framework. When building test |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 51 | fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and |
| 52 | :meth:`~TestCase.tearDown` methods can be overridden to provide initialization |
| 53 | and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions |
| 54 | can be passed to the constructor for these purposes. When the test is run, the |
| 55 | fixture initialization is run first; if it succeeds, the cleanup method is run |
| 56 | after the test has been executed, regardless of the outcome of the test. Each |
| 57 | instance of the :class:`TestCase` will only be used to run a single test method, |
| 58 | so a new fixture is created for each test. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 59 | |
| 60 | Test suites are implemented by the :class:`TestSuite` class. This class allows |
| 61 | individual tests and test suites to be aggregated; when the suite is executed, |
Benjamin Peterson | 14a3dd7 | 2009-05-25 00:51:58 +0000 | [diff] [blame] | 62 | all tests added directly to the suite and in "child" test suites are run. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 63 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 64 | A test runner is an object that provides a single method, |
| 65 | :meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite` |
| 66 | object as a parameter, and returns a result object. The class |
| 67 | :class:`TestResult` is provided for use as the result object. :mod:`unittest` |
| 68 | provides the :class:`TextTestRunner` as an example test runner which reports |
| 69 | test results on the standard error stream by default. Alternate runners can be |
| 70 | implemented for other environments (such as graphical environments) without any |
| 71 | need to derive from a specific class. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 72 | |
| 73 | |
| 74 | .. seealso:: |
| 75 | |
| 76 | Module :mod:`doctest` |
| 77 | Another test-support module with a very different flavor. |
| 78 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 79 | `unittest2: A backport of new unittest features for Python 2.4-2.6 <http://pypi.python.org/pypi/unittest2>`_ |
| 80 | Many new features were added to unittest in Python 2.7, including test |
| 81 | discovery. unittest2 allows you to use these features with earlier |
| 82 | versions of Python. |
| 83 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 84 | `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_ |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 85 | Kent Beck's original paper on testing frameworks using the pattern shared |
| 86 | by :mod:`unittest`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 87 | |
Raymond Hettinger | 6b232cd | 2009-03-24 00:22:53 +0000 | [diff] [blame] | 88 | `Nose <http://code.google.com/p/python-nose/>`_ and `py.test <http://pytest.org>`_ |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 89 | Third-party unittest frameworks with a lighter-weight syntax for writing |
| 90 | tests. For example, ``assert func(10) == 42``. |
Raymond Hettinger | 6b232cd | 2009-03-24 00:22:53 +0000 | [diff] [blame] | 91 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 92 | `The Python Testing Tools Taxonomy <http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy>`_ |
| 93 | An extensive list of Python testing tools including functional testing |
| 94 | frameworks and mock object libraries. |
| 95 | |
| 96 | `Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_ |
| 97 | A special-interest-group for discussion of testing, and testing tools, |
| 98 | in Python. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 99 | |
| 100 | .. _unittest-minimal-example: |
| 101 | |
| 102 | Basic example |
| 103 | ------------- |
| 104 | |
| 105 | The :mod:`unittest` module provides a rich set of tools for constructing and |
| 106 | running tests. This section demonstrates that a small subset of the tools |
| 107 | suffice to meet the needs of most users. |
| 108 | |
| 109 | Here is a short script to test three functions from the :mod:`random` module:: |
| 110 | |
| 111 | import random |
| 112 | import unittest |
| 113 | |
| 114 | class TestSequenceFunctions(unittest.TestCase): |
| 115 | |
| 116 | def setUp(self): |
Georg Brandl | a08a3c5 | 2009-08-13 08:40:50 +0000 | [diff] [blame] | 117 | self.seq = list(range(10)) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 118 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 119 | def test_shuffle(self): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 120 | # make sure the shuffled sequence does not lose any elements |
| 121 | random.shuffle(self.seq) |
| 122 | self.seq.sort() |
Georg Brandl | a08a3c5 | 2009-08-13 08:40:50 +0000 | [diff] [blame] | 123 | self.assertEqual(self.seq, list(range(10))) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 124 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 125 | # should raise an exception for an immutable sequence |
| 126 | self.assertRaises(TypeError, random.shuffle, (1,2,3)) |
| 127 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 128 | def test_choice(self): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 129 | element = random.choice(self.seq) |
Ezio Melotti | d503180 | 2010-02-04 20:29:01 +0000 | [diff] [blame] | 130 | self.assertIn(element, self.seq) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 131 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 132 | def test_sample(self): |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 133 | with self.assertRaises(ValueError): |
| 134 | random.sample(self.seq, 20) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 135 | for element in random.sample(self.seq, 5): |
Ezio Melotti | d503180 | 2010-02-04 20:29:01 +0000 | [diff] [blame] | 136 | self.assertIn(element, self.seq) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 137 | |
| 138 | if __name__ == '__main__': |
| 139 | unittest.main() |
| 140 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 141 | A testcase is created by subclassing :class:`unittest.TestCase`. The three |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 142 | individual tests are defined with methods whose names start with the letters |
| 143 | ``test``. This naming convention informs the test runner about which methods |
| 144 | represent tests. |
| 145 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 146 | The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 147 | expected result; :meth:`~TestCase.assertTrue` to verify a condition; or |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 148 | :meth:`~TestCase.assertRaises` to verify that an expected exception gets raised. |
| 149 | These methods are used instead of the :keyword:`assert` statement so the test |
| 150 | runner can accumulate all test results and produce a report. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 151 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 152 | When a :meth:`~TestCase.setUp` method is defined, the test runner will run that |
| 153 | method prior to each test. Likewise, if a :meth:`~TestCase.tearDown` method is |
| 154 | defined, the test runner will invoke that method after each test. In the |
| 155 | example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each |
| 156 | test. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 157 | |
| 158 | The final block shows a simple way to run the tests. :func:`unittest.main` |
Éric Araujo | 3efdf06 | 2010-12-16 03:16:29 +0000 | [diff] [blame] | 159 | provides a command-line interface to the test script. When run from the command |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 160 | line, the above script produces an output that looks like this:: |
| 161 | |
| 162 | ... |
| 163 | ---------------------------------------------------------------------- |
| 164 | Ran 3 tests in 0.000s |
| 165 | |
| 166 | OK |
| 167 | |
| 168 | Instead of :func:`unittest.main`, there are other ways to run the tests with a |
| 169 | finer level of control, less terse output, and no requirement to be run from the |
| 170 | command line. For example, the last two lines may be replaced with:: |
| 171 | |
| 172 | suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions) |
| 173 | unittest.TextTestRunner(verbosity=2).run(suite) |
| 174 | |
| 175 | Running the revised script from the interpreter or another script produces the |
| 176 | following output:: |
| 177 | |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 178 | test_choice (__main__.TestSequenceFunctions) ... ok |
| 179 | test_sample (__main__.TestSequenceFunctions) ... ok |
| 180 | test_shuffle (__main__.TestSequenceFunctions) ... ok |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 181 | |
| 182 | ---------------------------------------------------------------------- |
| 183 | Ran 3 tests in 0.110s |
| 184 | |
| 185 | OK |
| 186 | |
| 187 | The above examples show the most commonly used :mod:`unittest` features which |
| 188 | are sufficient to meet many everyday testing needs. The remainder of the |
| 189 | documentation explores the full feature set from first principles. |
| 190 | |
| 191 | |
Éric Araujo | 3efdf06 | 2010-12-16 03:16:29 +0000 | [diff] [blame] | 192 | .. _unittest-command-line-interface: |
| 193 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 194 | Command Line Interface |
Éric Araujo | 3efdf06 | 2010-12-16 03:16:29 +0000 | [diff] [blame] | 195 | ---------------------- |
| 196 | |
| 197 | The unittest module can be used from the command line to run tests from |
| 198 | modules, classes or even individual test methods:: |
| 199 | |
| 200 | python -m unittest test_module1 test_module2 |
| 201 | python -m unittest test_module.TestClass |
| 202 | python -m unittest test_module.TestClass.test_method |
| 203 | |
| 204 | You can pass in a list with any combination of module names, and fully |
| 205 | qualified class or method names. |
| 206 | |
| 207 | You can run tests with more detail (higher verbosity) by passing in the -v flag:: |
| 208 | |
| 209 | python -m unittest -v test_module |
| 210 | |
| 211 | For a list of all the command-line options:: |
| 212 | |
| 213 | python -m unittest -h |
| 214 | |
| 215 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 216 | .. _organizing-tests: |
| 217 | |
| 218 | Organizing test code |
| 219 | -------------------- |
| 220 | |
| 221 | The basic building blocks of unit testing are :dfn:`test cases` --- single |
| 222 | scenarios that must be set up and checked for correctness. In :mod:`unittest`, |
| 223 | test cases are represented by instances of :mod:`unittest`'s :class:`TestCase` |
| 224 | class. To make your own test cases you must write subclasses of |
| 225 | :class:`TestCase`, or use :class:`FunctionTestCase`. |
| 226 | |
| 227 | An instance of a :class:`TestCase`\ -derived class is an object that can |
| 228 | completely run a single test method, together with optional set-up and tidy-up |
| 229 | code. |
| 230 | |
| 231 | The testing code of a :class:`TestCase` instance should be entirely self |
| 232 | contained, such that it can be run either in isolation or in arbitrary |
| 233 | combination with any number of other test cases. |
| 234 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 235 | The simplest :class:`TestCase` subclass will simply override the |
| 236 | :meth:`~TestCase.runTest` method in order to perform specific testing code:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 237 | |
| 238 | import unittest |
| 239 | |
| 240 | class DefaultWidgetSizeTestCase(unittest.TestCase): |
| 241 | def runTest(self): |
| 242 | widget = Widget('The widget') |
| 243 | self.assertEqual(widget.size(), (50, 50), 'incorrect default size') |
| 244 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 245 | Note that in order to test something, we use the one of the :meth:`assert\*` |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 246 | methods provided by the :class:`TestCase` base class. If the test fails, an |
| 247 | exception will be raised, and :mod:`unittest` will identify the test case as a |
| 248 | :dfn:`failure`. Any other exceptions will be treated as :dfn:`errors`. This |
| 249 | helps you identify where the problem is: :dfn:`failures` are caused by incorrect |
| 250 | results - a 5 where you expected a 6. :dfn:`Errors` are caused by incorrect |
| 251 | code - e.g., a :exc:`TypeError` caused by an incorrect function call. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 252 | |
| 253 | The way to run a test case will be described later. For now, note that to |
| 254 | construct an instance of such a test case, we call its constructor without |
| 255 | arguments:: |
| 256 | |
| 257 | testCase = DefaultWidgetSizeTestCase() |
| 258 | |
| 259 | Now, such test cases can be numerous, and their set-up can be repetitive. In |
| 260 | the above case, constructing a :class:`Widget` in each of 100 Widget test case |
| 261 | subclasses would mean unsightly duplication. |
| 262 | |
| 263 | Luckily, we can factor out such set-up code by implementing a method called |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 264 | :meth:`~TestCase.setUp`, which the testing framework will automatically call for |
| 265 | us when we run the test:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 266 | |
| 267 | import unittest |
| 268 | |
| 269 | class SimpleWidgetTestCase(unittest.TestCase): |
| 270 | def setUp(self): |
| 271 | self.widget = Widget('The widget') |
| 272 | |
| 273 | class DefaultWidgetSizeTestCase(SimpleWidgetTestCase): |
| 274 | def runTest(self): |
Ezio Melotti | d503180 | 2010-02-04 20:29:01 +0000 | [diff] [blame] | 275 | self.assertEqual(self.widget.size(), (50,50), |
| 276 | 'incorrect default size') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 277 | |
| 278 | class WidgetResizeTestCase(SimpleWidgetTestCase): |
| 279 | def runTest(self): |
| 280 | self.widget.resize(100,150) |
Ezio Melotti | d503180 | 2010-02-04 20:29:01 +0000 | [diff] [blame] | 281 | self.assertEqual(self.widget.size(), (100,150), |
| 282 | 'wrong size after resize') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 283 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 284 | If the :meth:`~TestCase.setUp` method raises an exception while the test is |
| 285 | running, the framework will consider the test to have suffered an error, and the |
| 286 | :meth:`~TestCase.runTest` method will not be executed. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 287 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 288 | Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up |
| 289 | after the :meth:`~TestCase.runTest` method has been run:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 290 | |
| 291 | import unittest |
| 292 | |
| 293 | class SimpleWidgetTestCase(unittest.TestCase): |
| 294 | def setUp(self): |
| 295 | self.widget = Widget('The widget') |
| 296 | |
| 297 | def tearDown(self): |
| 298 | self.widget.dispose() |
| 299 | self.widget = None |
| 300 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 301 | If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will |
| 302 | be run whether :meth:`~TestCase.runTest` succeeded or not. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 303 | |
| 304 | Such a working environment for the testing code is called a :dfn:`fixture`. |
| 305 | |
| 306 | Often, many small test cases will use the same fixture. In this case, we would |
| 307 | end up subclassing :class:`SimpleWidgetTestCase` into many small one-method |
| 308 | classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 309 | discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler |
| 310 | mechanism:: |
| 311 | |
| 312 | import unittest |
| 313 | |
| 314 | class WidgetTestCase(unittest.TestCase): |
| 315 | def setUp(self): |
| 316 | self.widget = Widget('The widget') |
| 317 | |
| 318 | def tearDown(self): |
| 319 | self.widget.dispose() |
| 320 | self.widget = None |
| 321 | |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 322 | def test_default_size(self): |
Ezio Melotti | d503180 | 2010-02-04 20:29:01 +0000 | [diff] [blame] | 323 | self.assertEqual(self.widget.size(), (50,50), |
| 324 | 'incorrect default size') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 325 | |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 326 | def test_resize(self): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 327 | self.widget.resize(100,150) |
Ezio Melotti | d503180 | 2010-02-04 20:29:01 +0000 | [diff] [blame] | 328 | self.assertEqual(self.widget.size(), (100,150), |
| 329 | 'wrong size after resize') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 330 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 331 | Here we have not provided a :meth:`~TestCase.runTest` method, but have instead |
| 332 | provided two different test methods. Class instances will now each run one of |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 333 | the :meth:`test_\*` methods, with ``self.widget`` created and destroyed |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 334 | separately for each instance. When creating an instance we must specify the |
| 335 | test method it is to run. We do this by passing the method name in the |
| 336 | constructor:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 337 | |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 338 | defaultSizeTestCase = WidgetTestCase('test_default_size') |
| 339 | resizeTestCase = WidgetTestCase('test_resize') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 340 | |
| 341 | Test case instances are grouped together according to the features they test. |
| 342 | :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`, |
| 343 | represented by :mod:`unittest`'s :class:`TestSuite` class:: |
| 344 | |
| 345 | widgetTestSuite = unittest.TestSuite() |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 346 | widgetTestSuite.addTest(WidgetTestCase('test_default_size')) |
| 347 | widgetTestSuite.addTest(WidgetTestCase('test_resize')) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 348 | |
| 349 | For the ease of running tests, as we will see later, it is a good idea to |
| 350 | provide in each test module a callable object that returns a pre-built test |
| 351 | suite:: |
| 352 | |
| 353 | def suite(): |
| 354 | suite = unittest.TestSuite() |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 355 | suite.addTest(WidgetTestCase('test_default_size')) |
| 356 | suite.addTest(WidgetTestCase('test_resize')) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 357 | return suite |
| 358 | |
| 359 | or even:: |
| 360 | |
| 361 | def suite(): |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 362 | tests = ['test_default_size', 'test_resize'] |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 363 | |
| 364 | return unittest.TestSuite(map(WidgetTestCase, tests)) |
| 365 | |
| 366 | Since it is a common pattern to create a :class:`TestCase` subclass with many |
| 367 | similarly named test functions, :mod:`unittest` provides a :class:`TestLoader` |
| 368 | class that can be used to automate the process of creating a test suite and |
| 369 | populating it with individual tests. For example, :: |
| 370 | |
| 371 | suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase) |
| 372 | |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 373 | will create a test suite that will run ``WidgetTestCase.test_default_size()`` and |
| 374 | ``WidgetTestCase.test_resize``. :class:`TestLoader` uses the ``'test'`` method |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 375 | name prefix to identify test methods automatically. |
| 376 | |
Mark Dickinson | c48d834 | 2009-02-01 14:18:10 +0000 | [diff] [blame] | 377 | Note that the order in which the various test cases will be run is |
| 378 | determined by sorting the test function names with respect to the |
| 379 | built-in ordering for strings. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 380 | |
| 381 | Often it is desirable to group suites of test cases together, so as to run tests |
| 382 | for the whole system at once. This is easy, since :class:`TestSuite` instances |
| 383 | can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be |
| 384 | added to a :class:`TestSuite`:: |
| 385 | |
| 386 | suite1 = module1.TheTestSuite() |
| 387 | suite2 = module2.TheTestSuite() |
| 388 | alltests = unittest.TestSuite([suite1, suite2]) |
| 389 | |
| 390 | You can place the definitions of test cases and test suites in the same modules |
| 391 | as the code they are to test (such as :file:`widget.py`), but there are several |
| 392 | advantages to placing the test code in a separate module, such as |
| 393 | :file:`test_widget.py`: |
| 394 | |
| 395 | * The test module can be run standalone from the command line. |
| 396 | |
| 397 | * The test code can more easily be separated from shipped code. |
| 398 | |
| 399 | * There is less temptation to change test code to fit the code it tests without |
| 400 | a good reason. |
| 401 | |
| 402 | * Test code should be modified much less frequently than the code it tests. |
| 403 | |
| 404 | * Tested code can be refactored more easily. |
| 405 | |
| 406 | * Tests for modules written in C must be in separate modules anyway, so why not |
| 407 | be consistent? |
| 408 | |
| 409 | * If the testing strategy changes, there is no need to change the source code. |
| 410 | |
| 411 | |
| 412 | .. _legacy-unit-tests: |
| 413 | |
| 414 | Re-using old test code |
| 415 | ---------------------- |
| 416 | |
| 417 | Some users will find that they have existing test code that they would like to |
| 418 | run from :mod:`unittest`, without converting every old test function to a |
| 419 | :class:`TestCase` subclass. |
| 420 | |
| 421 | For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class. |
| 422 | This subclass of :class:`TestCase` can be used to wrap an existing test |
| 423 | function. Set-up and tear-down functions can also be provided. |
| 424 | |
| 425 | Given the following test function:: |
| 426 | |
| 427 | def testSomething(): |
| 428 | something = makeSomething() |
| 429 | assert something.name is not None |
| 430 | # ... |
| 431 | |
| 432 | one can create an equivalent test case instance as follows:: |
| 433 | |
| 434 | testcase = unittest.FunctionTestCase(testSomething) |
| 435 | |
| 436 | If there are additional set-up and tear-down methods that should be called as |
| 437 | part of the test case's operation, they can also be provided like so:: |
| 438 | |
| 439 | testcase = unittest.FunctionTestCase(testSomething, |
| 440 | setUp=makeSomethingDB, |
| 441 | tearDown=deleteSomethingDB) |
| 442 | |
| 443 | To make migrating existing test suites easier, :mod:`unittest` supports tests |
| 444 | raising :exc:`AssertionError` to indicate test failure. However, it is |
| 445 | recommended that you use the explicit :meth:`TestCase.fail\*` and |
| 446 | :meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest` |
| 447 | may treat :exc:`AssertionError` differently. |
| 448 | |
| 449 | .. note:: |
| 450 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 451 | Even though :class:`FunctionTestCase` can be used to quickly convert an |
| 452 | existing test base over to a :mod:`unittest`\ -based system, this approach is |
| 453 | not recommended. Taking the time to set up proper :class:`TestCase` |
| 454 | subclasses will make future test refactorings infinitely easier. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 455 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 456 | In some cases, the existing tests may have been written using the :mod:`doctest` |
| 457 | module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can |
| 458 | automatically build :class:`unittest.TestSuite` instances from the existing |
| 459 | :mod:`doctest`\ -based tests. |
| 460 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 461 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 462 | .. _unittest-skipping: |
| 463 | |
| 464 | Skipping tests and expected failures |
| 465 | ------------------------------------ |
| 466 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 467 | .. versionadded:: 3.1 |
| 468 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 469 | Unittest supports skipping individual test methods and even whole classes of |
| 470 | tests. In addition, it supports marking a test as a "expected failure," a test |
| 471 | that is broken and will fail, but shouldn't be counted as a failure on a |
| 472 | :class:`TestResult`. |
| 473 | |
| 474 | Skipping a test is simply a matter of using the :func:`skip` :term:`decorator` |
| 475 | or one of its conditional variants. |
| 476 | |
| 477 | Basic skipping looks like this: :: |
| 478 | |
| 479 | class MyTestCase(unittest.TestCase): |
| 480 | |
| 481 | @unittest.skip("demonstrating skipping") |
| 482 | def test_nothing(self): |
| 483 | self.fail("shouldn't happen") |
| 484 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 485 | @unittest.skipIf(mylib.__version__ < (1, 3), |
| 486 | "not supported in this library version") |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 487 | def test_format(self): |
| 488 | # Tests that work for only a certain version of the library. |
| 489 | pass |
| 490 | |
| 491 | @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") |
| 492 | def test_windows_support(self): |
| 493 | # windows specific testing code |
| 494 | pass |
| 495 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 496 | This is the output of running the example above in verbose mode: :: |
| 497 | |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 498 | test_format (__main__.MyTestCase) ... skipped 'not supported in this library version' |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 499 | test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping' |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 500 | test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows' |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 501 | |
| 502 | ---------------------------------------------------------------------- |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 503 | Ran 3 tests in 0.005s |
| 504 | |
| 505 | OK (skipped=3) |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 506 | |
| 507 | Classes can be skipped just like methods: :: |
| 508 | |
| 509 | @skip("showing class skipping") |
| 510 | class MySkippedTestCase(unittest.TestCase): |
| 511 | def test_not_run(self): |
| 512 | pass |
| 513 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 514 | :meth:`TestCase.setUp` can also skip the test. This is useful when a resource |
| 515 | that needs to be set up is not available. |
| 516 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 517 | Expected failures use the :func:`expectedFailure` decorator. :: |
| 518 | |
| 519 | class ExpectedFailureTestCase(unittest.TestCase): |
| 520 | @unittest.expectedFailure |
| 521 | def test_fail(self): |
| 522 | self.assertEqual(1, 0, "broken") |
| 523 | |
| 524 | It's easy to roll your own skipping decorators by making a decorator that calls |
| 525 | :func:`skip` on the test when it wants it to be skipped. This decorator skips |
| 526 | the test unless the passed object has a certain attribute: :: |
| 527 | |
| 528 | def skipUnlessHasattr(obj, attr): |
| 529 | if hasattr(obj, attr): |
| 530 | return lambda func: func |
| 531 | return unittest.skip("{0!r} doesn't have {1!r}".format(obj, attr)) |
| 532 | |
| 533 | The following decorators implement test skipping and expected failures: |
| 534 | |
| 535 | .. function:: skip(reason) |
| 536 | |
| 537 | Unconditionally skip the decorated test. *reason* should describe why the |
| 538 | test is being skipped. |
| 539 | |
| 540 | .. function:: skipIf(condition, reason) |
| 541 | |
| 542 | Skip the decorated test if *condition* is true. |
| 543 | |
| 544 | .. function:: skipUnless(condition, reason) |
| 545 | |
Georg Brandl | 4b05466 | 2010-10-06 08:56:53 +0000 | [diff] [blame] | 546 | Skip the decorated test unless *condition* is true. |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 547 | |
| 548 | .. function:: expectedFailure |
| 549 | |
| 550 | Mark the test as an expected failure. If the test fails when run, the test |
| 551 | is not counted as a failure. |
| 552 | |
| 553 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 554 | .. _unittest-contents: |
| 555 | |
| 556 | Classes and functions |
| 557 | --------------------- |
| 558 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 559 | This section describes in depth the API of :mod:`unittest`. |
| 560 | |
| 561 | |
| 562 | .. _testcase-objects: |
| 563 | |
| 564 | Test cases |
| 565 | ~~~~~~~~~~ |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 566 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 567 | .. class:: TestCase(methodName='runTest') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 568 | |
| 569 | Instances of the :class:`TestCase` class represent the smallest testable units |
| 570 | in the :mod:`unittest` universe. This class is intended to be used as a base |
| 571 | class, with specific tests being implemented by concrete subclasses. This class |
| 572 | implements the interface needed by the test runner to allow it to drive the |
| 573 | test, and methods that the test code can use to check for and report various |
| 574 | kinds of failure. |
| 575 | |
| 576 | Each instance of :class:`TestCase` will run a single test method: the method |
| 577 | named *methodName*. If you remember, we had an earlier example that went |
| 578 | something like this:: |
| 579 | |
| 580 | def suite(): |
| 581 | suite = unittest.TestSuite() |
Ezio Melotti | c08cae9 | 2010-02-28 03:48:50 +0000 | [diff] [blame] | 582 | suite.addTest(WidgetTestCase('test_default_size')) |
| 583 | suite.addTest(WidgetTestCase('test_resize')) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 584 | return suite |
| 585 | |
| 586 | Here, we create two instances of :class:`WidgetTestCase`, each of which runs a |
| 587 | single test. |
| 588 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 589 | *methodName* defaults to :meth:`runTest`. |
| 590 | |
| 591 | :class:`TestCase` instances provide three groups of methods: one group used |
| 592 | to run the test, another used by the test implementation to check conditions |
| 593 | and report failures, and some inquiry methods allowing information about the |
| 594 | test itself to be gathered. |
| 595 | |
| 596 | Methods in the first group (running the test) are: |
| 597 | |
| 598 | |
| 599 | .. method:: setUp() |
| 600 | |
| 601 | Method called to prepare the test fixture. This is called immediately |
| 602 | before calling the test method; any exception raised by this method will |
| 603 | be considered an error rather than a test failure. The default |
| 604 | implementation does nothing. |
| 605 | |
| 606 | |
| 607 | .. method:: tearDown() |
| 608 | |
| 609 | Method called immediately after the test method has been called and the |
| 610 | result recorded. This is called even if the test method raised an |
| 611 | exception, so the implementation in subclasses may need to be particularly |
| 612 | careful about checking internal state. Any exception raised by this |
| 613 | method will be considered an error rather than a test failure. This |
| 614 | method will only be called if the :meth:`setUp` succeeds, regardless of |
| 615 | the outcome of the test method. The default implementation does nothing. |
| 616 | |
| 617 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 618 | .. method:: run(result=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 619 | |
| 620 | Run the test, collecting the result into the test result object passed as |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 621 | *result*. If *result* is omitted or ``None``, a temporary result |
Georg Brandl | bcc484e | 2009-08-13 11:51:54 +0000 | [diff] [blame] | 622 | object is created (by calling the :meth:`defaultTestResult` method) and |
| 623 | used. The result object is not returned to :meth:`run`'s caller. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 624 | |
| 625 | The same effect may be had by simply calling the :class:`TestCase` |
| 626 | instance. |
| 627 | |
| 628 | |
Benjamin Peterson | e549ead | 2009-03-28 21:42:05 +0000 | [diff] [blame] | 629 | .. method:: skipTest(reason) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 630 | |
Stefan Krah | 671b016 | 2010-05-19 16:11:36 +0000 | [diff] [blame] | 631 | Calling this during a test method or :meth:`setUp` skips the current |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 632 | test. See :ref:`unittest-skipping` for more information. |
| 633 | |
Ezio Melotti | 8d5c16a | 2010-04-20 09:36:13 +0000 | [diff] [blame] | 634 | .. versionadded:: 3.1 |
| 635 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 636 | |
| 637 | .. method:: debug() |
| 638 | |
| 639 | Run the test without collecting the result. This allows exceptions raised |
| 640 | by the test to be propagated to the caller, and can be used to support |
| 641 | running tests under a debugger. |
| 642 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 643 | .. _assert-methods: |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 644 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 645 | The :class:`TestCase` class provides a number of methods to check for and |
| 646 | report failures, such as: |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 647 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 648 | +-----------------------------------------+-----------------------------+---------------+ |
| 649 | | Method | Checks that | New in | |
| 650 | +=========================================+=============================+===============+ |
| 651 | | :meth:`assertEqual(a, b) | ``a == b`` | | |
| 652 | | <TestCase.assertEqual>` | | | |
| 653 | +-----------------------------------------+-----------------------------+---------------+ |
| 654 | | :meth:`assertNotEqual(a, b) | ``a != b`` | | |
| 655 | | <TestCase.assertNotEqual>` | | | |
| 656 | +-----------------------------------------+-----------------------------+---------------+ |
| 657 | | :meth:`assertTrue(x) | ``bool(x) is True`` | | |
| 658 | | <TestCase.assertTrue>` | | | |
| 659 | +-----------------------------------------+-----------------------------+---------------+ |
| 660 | | :meth:`assertFalse(x) | ``bool(x) is False`` | | |
| 661 | | <TestCase.assertFalse>` | | | |
| 662 | +-----------------------------------------+-----------------------------+---------------+ |
| 663 | | :meth:`assertIs(a, b) | ``a is b`` | 3.1 | |
| 664 | | <TestCase.assertIs>` | | | |
| 665 | +-----------------------------------------+-----------------------------+---------------+ |
| 666 | | :meth:`assertIsNot(a, b) | ``a is not b`` | 3.1 | |
| 667 | | <TestCase.assertIsNot>` | | | |
| 668 | +-----------------------------------------+-----------------------------+---------------+ |
| 669 | | :meth:`assertIsNone(x) | ``x is None`` | 3.1 | |
| 670 | | <TestCase.assertIsNone>` | | | |
| 671 | +-----------------------------------------+-----------------------------+---------------+ |
| 672 | | :meth:`assertIsNotNone(x) | ``x is not None`` | 3.1 | |
| 673 | | <TestCase.assertIsNotNone>` | | | |
| 674 | +-----------------------------------------+-----------------------------+---------------+ |
| 675 | | :meth:`assertIn(a, b) | ``a in b`` | 3.1 | |
| 676 | | <TestCase.assertIn>` | | | |
| 677 | +-----------------------------------------+-----------------------------+---------------+ |
| 678 | | :meth:`assertNotIn(a, b) | ``a not in b`` | 3.1 | |
| 679 | | <TestCase.assertNotIn>` | | | |
| 680 | +-----------------------------------------+-----------------------------+---------------+ |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 681 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 682 | All the assert methods (except :meth:`assertRaises`, |
| 683 | :meth:`assertRaisesRegexp`, :meth:`assertWarns`, :meth:`assertWarnsRegexp`) |
| 684 | accept a *msg* argument that, if specified, is used as the error message on |
| 685 | failure (see also :data:`longMessage`). |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 686 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 687 | .. method:: assertEqual(first, second, msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 688 | |
| 689 | Test that *first* and *second* are equal. If the values do not compare |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 690 | equal, the test will fail. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 691 | |
| 692 | In addition, if *first* and *second* are the exact same type and one of |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 693 | list, tuple, dict, set, frozenset or str or any type that a subclass |
| 694 | registers with :meth:`addTypeEqualityFunc` the type specific equality |
| 695 | function will be called in order to generate a more useful default |
| 696 | error message (see also the :ref:`list of type-specific methods |
| 697 | <type-specific-methods>`). |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 698 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 699 | .. versionchanged:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 700 | Added the automatic calling of type specific equality function. |
| 701 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 702 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 703 | .. method:: assertNotEqual(first, second, msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 704 | |
| 705 | Test that *first* and *second* are not equal. If the values do compare |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 706 | equal, the test will fail. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 707 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 708 | .. method:: assertTrue(expr, msg=None) |
| 709 | assertFalse(expr, msg=None) |
| 710 | |
| 711 | Test that *expr* is true (or false). |
| 712 | |
| 713 | Note that this is equivalent to ``bool(expr) is True`` and not to ``expr |
| 714 | is True`` (use ``assertIs(expr, True)`` for the latter). This method |
| 715 | should also be avoided when more specific methods are available (e.g. |
| 716 | ``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they |
| 717 | provide a better error message in case of failure. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 718 | |
Benjamin Peterson | 70e32c8 | 2009-03-24 01:00:11 +0000 | [diff] [blame] | 719 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 720 | .. method:: assertIs(first, second, msg=None) |
| 721 | assertIsNot(first, second, msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 722 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 723 | Test that *first* and *second* evaluate (or don't evaluate) to the same object. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 724 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 725 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 726 | |
| 727 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 728 | .. method:: assertIsNone(expr, msg=None) |
| 729 | assertIsNotNone(expr, msg=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 730 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 731 | Test that *expr* is (or is not) None. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 732 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 733 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 734 | |
| 735 | |
| 736 | .. method:: assertIn(first, second, msg=None) |
| 737 | assertNotIn(first, second, msg=None) |
| 738 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 739 | Test that *first* is (or is not) in *second*. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 740 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 741 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 742 | |
| 743 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 744 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 745 | It is also possible to check that exceptions and warnings are raised using |
| 746 | the following methods: |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 747 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 748 | +---------------------------------------------------------+--------------------------------------+------------+ |
| 749 | | Method | Checks that | New in | |
| 750 | +=========================================================+======================================+============+ |
| 751 | | :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises `exc` | | |
| 752 | | <TestCase.assertRaises>` | | | |
| 753 | +---------------------------------------------------------+--------------------------------------+------------+ |
| 754 | | :meth:`assertRaisesRegexp(exc, re, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises `exc` | 3.1 | |
| 755 | | <TestCase.assertRaisesRegexp>` | and the message matches `re` | | |
| 756 | +---------------------------------------------------------+--------------------------------------+------------+ |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 757 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 758 | .. method:: assertRaises(exception, callable, *args, **kwds) |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 759 | assertRaises(exception) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 760 | |
| 761 | Test that an exception is raised when *callable* is called with any |
| 762 | positional or keyword arguments that are also passed to |
| 763 | :meth:`assertRaises`. The test passes if *exception* is raised, is an |
| 764 | error if another exception is raised, or fails if no exception is raised. |
| 765 | To catch any of a group of exceptions, a tuple containing the exception |
| 766 | classes may be passed as *exception*. |
| 767 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 768 | If only the *exception* argument is given, returns a context manager so |
| 769 | that the code under test can be written inline rather than as a function:: |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 770 | |
Ezio Melotti | 0f36573 | 2010-02-08 22:07:38 +0000 | [diff] [blame] | 771 | with self.assertRaises(SomeException): |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 772 | do_something() |
| 773 | |
Benjamin Peterson | 70e32c8 | 2009-03-24 01:00:11 +0000 | [diff] [blame] | 774 | .. versionchanged:: 3.1 |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 775 | Added the ability to use :meth:`assertRaises` as a context manager. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 776 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 777 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 778 | .. method:: assertRaisesRegexp(exception, regexp, callable, *args, **kwds) |
| 779 | assertRaisesRegexp(exception, regexp) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 780 | |
| 781 | Like :meth:`assertRaises` but also tests that *regexp* matches |
| 782 | on the string representation of the raised exception. *regexp* may be |
| 783 | a regular expression object or a string containing a regular expression |
| 784 | suitable for use by :func:`re.search`. Examples:: |
| 785 | |
| 786 | self.assertRaisesRegexp(ValueError, 'invalid literal for.*XYZ$', |
| 787 | int, 'XYZ') |
| 788 | |
| 789 | or:: |
| 790 | |
| 791 | with self.assertRaisesRegexp(ValueError, 'literal'): |
| 792 | int('XYZ') |
| 793 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 794 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 795 | |
| 796 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 797 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 798 | There are also other methods used to perform more specific checks, such as: |
| 799 | |
| 800 | +---------------------------------------+--------------------------------+--------------+ |
| 801 | | Method | Checks that | New in | |
| 802 | +=======================================+================================+==============+ |
| 803 | | :meth:`assertAlmostEqual(a, b) | ``round(a-b, 7) == 0`` | | |
| 804 | | <TestCase.assertAlmostEqual>` | | | |
| 805 | +---------------------------------------+--------------------------------+--------------+ |
| 806 | | :meth:`assertNotAlmostEqual(a, b) | ``round(a-b, 7) != 0`` | | |
| 807 | | <TestCase.assertNotAlmostEqual>` | | | |
| 808 | +---------------------------------------+--------------------------------+--------------+ |
| 809 | | :meth:`assertGreater(a, b) | ``a > b`` | 3.1 | |
| 810 | | <TestCase.assertGreater>` | | | |
| 811 | +---------------------------------------+--------------------------------+--------------+ |
| 812 | | :meth:`assertGreaterEqual(a, b) | ``a >= b`` | 3.1 | |
| 813 | | <TestCase.assertGreaterEqual>` | | | |
| 814 | +---------------------------------------+--------------------------------+--------------+ |
| 815 | | :meth:`assertLess(a, b) | ``a < b`` | 3.1 | |
| 816 | | <TestCase.assertLess>` | | | |
| 817 | +---------------------------------------+--------------------------------+--------------+ |
| 818 | | :meth:`assertLessEqual(a, b) | ``a <= b`` | 3.1 | |
| 819 | | <TestCase.assertLessEqual>` | | | |
| 820 | +---------------------------------------+--------------------------------+--------------+ |
| 821 | | :meth:`assertRegexpMatches(s, re) | ``regex.search(s)`` | 3.1 | |
| 822 | | <TestCase.assertRegexpMatches>` | | | |
| 823 | +---------------------------------------+--------------------------------+--------------+ |
| 824 | | :meth:`assertDictContainsSubset(a, b) | all the key/value pairs | 3.1 | |
| 825 | | <TestCase.assertDictContainsSubset>` | in `a` exist in `b` | | |
| 826 | +---------------------------------------+--------------------------------+--------------+ |
| 827 | |
| 828 | |
| 829 | .. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None) |
| 830 | assertNotAlmostEqual(first, second, places=7, msg=None, delta=None) |
| 831 | |
| 832 | Test that *first* and *second* are approximately (or not approximately) |
| 833 | equal by computing the difference, rounding to the given number of |
| 834 | decimal *places* (default 7), and comparing to zero. Note that these |
| 835 | methods round the values to the given number of *decimal places* (i.e. |
| 836 | like the :func:`round` function) and not *significant digits*. |
| 837 | |
| 838 | If *delta* is supplied instead of *places* then the difference |
| 839 | between *first* and *second* must be less (or more) than *delta*. |
| 840 | |
| 841 | Supplying both *delta* and *places* raises a ``TypeError``. |
| 842 | |
| 843 | |
| 844 | .. method:: assertGreater(first, second, msg=None) |
| 845 | assertGreaterEqual(first, second, msg=None) |
| 846 | assertLess(first, second, msg=None) |
| 847 | assertLessEqual(first, second, msg=None) |
| 848 | |
| 849 | Test that *first* is respectively >, >=, < or <= than *second* depending |
| 850 | on the method name. If not, the test will fail:: |
| 851 | |
| 852 | >>> self.assertGreaterEqual(3, 4) |
| 853 | AssertionError: "3" unexpectedly not greater than or equal to "4" |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 854 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 855 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 856 | |
| 857 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 858 | .. method:: assertRegexpMatches(text, regexp, msg=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 859 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 860 | Test that a *regexp* search matches *text*. In case |
| 861 | of failure, the error message will include the pattern and the *text* (or |
| 862 | the pattern and the part of *text* that unexpectedly matched). *regexp* |
| 863 | may be a regular expression object or a string containing a regular |
| 864 | expression suitable for use by :func:`re.search`. |
| 865 | |
| 866 | .. versionadded:: 3.1 :meth:`~TestCase.assertRegexpMatches` |
| 867 | |
| 868 | |
| 869 | .. method:: assertDictContainsSubset(expected, actual, msg=None) |
| 870 | |
| 871 | Tests whether the key/value pairs in dictionary *actual* are a |
| 872 | superset of those in *expected*. If not, an error message listing |
| 873 | the missing keys and mismatched values is generated. |
| 874 | |
| 875 | .. versionadded:: 3.1 |
| 876 | .. deprecated:: 3.2 |
| 877 | |
| 878 | |
| 879 | .. method:: assertSameElements(actual, expected, msg=None) |
| 880 | |
| 881 | Test that sequence *expected* contains the same elements as *actual*, |
| 882 | regardless of their order. When they don't, an error message listing |
| 883 | the differences between the sequences will be generated. |
| 884 | |
| 885 | Duplicate elements are ignored when comparing *actual* and *expected*. |
| 886 | It is the equivalent of ``assertEqual(set(expected), set(actual))`` |
| 887 | but it works with sequences of unhashable objects as well. Because |
| 888 | duplicates are ignored, this method has been deprecated in favour of |
| 889 | :meth:`assertItemsEqual`. |
| 890 | |
| 891 | .. versionadded:: 3.1 |
| 892 | .. deprecated:: 3.2 |
| 893 | |
| 894 | |
| 895 | .. _type-specific-methods: |
| 896 | |
| 897 | The :meth:`assertEqual` method dispatches the equality check for objects of |
| 898 | the same type to different type-specific methods. These methods are already |
| 899 | implemented for most of the built-in types, but it's also possible to |
| 900 | register new methods using :meth:`addTypeEqualityFunc`: |
| 901 | |
| 902 | .. method:: addTypeEqualityFunc(typeobj, function) |
| 903 | |
| 904 | Registers a type-specific method called by :meth:`assertEqual` to check |
| 905 | if two objects of exactly the same *typeobj* (not subclasses) compare |
| 906 | equal. *function* must take two positional arguments and a third msg=None |
| 907 | keyword argument just as :meth:`assertEqual` does. It must raise |
| 908 | :data:`self.failureException(msg) <failureException>` when inequality |
| 909 | between the first two parameters is detected -- possibly providing useful |
| 910 | information and explaining the inequalities in details in the error |
| 911 | message. |
| 912 | |
| 913 | .. versionadded:: 3.1 |
| 914 | |
| 915 | The list of type-specific methods automatically used by |
| 916 | :meth:`~TestCase.assertEqual` are summarized in the following table. Note |
| 917 | that it's usually not necessary to invoke these methods directly. |
| 918 | |
| 919 | +-----------------------------------------+-----------------------------+--------------+ |
| 920 | | Method | Used to compare | New in | |
| 921 | +=========================================+=============================+==============+ |
| 922 | | :meth:`assertMultiLineEqual(a, b) | strings | 3.1 | |
| 923 | | <TestCase.assertMultiLineEqual>` | | | |
| 924 | +-----------------------------------------+-----------------------------+--------------+ |
| 925 | | :meth:`assertSequenceEqual(a, b) | sequences | 3.1 | |
| 926 | | <TestCase.assertSequenceEqual>` | | | |
| 927 | +-----------------------------------------+-----------------------------+--------------+ |
| 928 | | :meth:`assertListEqual(a, b) | lists | 3.1 | |
| 929 | | <TestCase.assertListEqual>` | | | |
| 930 | +-----------------------------------------+-----------------------------+--------------+ |
| 931 | | :meth:`assertTupleEqual(a, b) | tuples | 3.1 | |
| 932 | | <TestCase.assertTupleEqual>` | | | |
| 933 | +-----------------------------------------+-----------------------------+--------------+ |
| 934 | | :meth:`assertSetEqual(a, b) | sets or frozensets | 3.1 | |
| 935 | | <TestCase.assertSetEqual>` | | | |
| 936 | +-----------------------------------------+-----------------------------+--------------+ |
| 937 | | :meth:`assertDictEqual(a, b) | dicts | 3.1 | |
| 938 | | <TestCase.assertDictEqual>` | | | |
| 939 | +-----------------------------------------+-----------------------------+--------------+ |
| 940 | |
| 941 | |
| 942 | |
| 943 | .. method:: assertMultiLineEqual(first, second, msg=None) |
| 944 | |
| 945 | Test that the multiline string *first* is equal to the string *second*. |
| 946 | When not equal a diff of the two strings highlighting the differences |
| 947 | will be included in the error message. This method is used by default |
| 948 | when comparing strings with :meth:`assertEqual`. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 949 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 950 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 951 | |
| 952 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 953 | .. method:: assertSequenceEqual(seq1, seq2, msg=None, seq_type=None) |
Benjamin Peterson | f47ed4a | 2009-04-11 20:45:40 +0000 | [diff] [blame] | 954 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 955 | Tests that two sequences are equal. If a *seq_type* is supplied, both |
| 956 | *seq1* and *seq2* must be instances of *seq_type* or a failure will |
| 957 | be raised. If the sequences are different an error message is |
| 958 | constructed that shows the difference between the two. |
| 959 | |
| 960 | This method is not called directly by :meth:`assertEqual`, but |
| 961 | it's used to implement :meth:`assertListEqual` and |
| 962 | :meth:`assertTupleEqual`. |
Benjamin Peterson | f47ed4a | 2009-04-11 20:45:40 +0000 | [diff] [blame] | 963 | |
Georg Brandl | 705d9d5 | 2009-05-05 09:29:50 +0000 | [diff] [blame] | 964 | .. versionadded:: 3.1 |
Benjamin Peterson | f47ed4a | 2009-04-11 20:45:40 +0000 | [diff] [blame] | 965 | |
| 966 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 967 | .. method:: assertListEqual(list1, list2, msg=None) |
| 968 | assertTupleEqual(tuple1, tuple2, msg=None) |
Benjamin Peterson | f47ed4a | 2009-04-11 20:45:40 +0000 | [diff] [blame] | 969 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 970 | Tests that two lists or tuples are equal. If not an error message is |
| 971 | constructed that shows only the differences between the two. An error |
| 972 | is also raised if either of the parameters are of the wrong type. |
| 973 | These methods are used by default when comparing lists or tuples with |
| 974 | :meth:`assertEqual`. |
Benjamin Peterson | f47ed4a | 2009-04-11 20:45:40 +0000 | [diff] [blame] | 975 | |
Georg Brandl | 705d9d5 | 2009-05-05 09:29:50 +0000 | [diff] [blame] | 976 | .. versionadded:: 3.1 |
Benjamin Peterson | f47ed4a | 2009-04-11 20:45:40 +0000 | [diff] [blame] | 977 | |
| 978 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 979 | .. method:: assertSetEqual(set1, set2, msg=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 980 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 981 | Tests that two sets are equal. If not, an error message is constructed |
| 982 | that lists the differences between the sets. This method is used by |
| 983 | default when comparing sets or frozensets with :meth:`assertEqual`. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 984 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 985 | Fails if either of *set1* or *set2* does not have a :meth:`set.difference` |
| 986 | method. |
| 987 | |
| 988 | .. versionadded:: 3.1 |
| 989 | |
| 990 | |
| 991 | .. method:: assertDictEqual(expected, actual, msg=None) |
| 992 | |
| 993 | Test that two dictionaries are equal. If not, an error message is |
| 994 | constructed that shows the differences in the dictionaries. This |
| 995 | method will be used by default to compare dictionaries in |
| 996 | calls to :meth:`assertEqual`. |
| 997 | |
| 998 | .. versionadded:: 3.1 |
| 999 | |
| 1000 | |
| 1001 | |
| 1002 | .. _other-methods-and-attrs: |
| 1003 | |
| 1004 | Finally the :class:`TestCase` provides the following methods and attributes: |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1005 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1006 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1007 | .. method:: fail(msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1008 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1009 | Signals a test failure unconditionally, with *msg* or ``None`` for |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1010 | the error message. |
| 1011 | |
| 1012 | |
| 1013 | .. attribute:: failureException |
| 1014 | |
| 1015 | This class attribute gives the exception raised by the test method. If a |
| 1016 | test framework needs to use a specialized exception, possibly to carry |
| 1017 | additional information, it must subclass this exception in order to "play |
| 1018 | fair" with the framework. The initial value of this attribute is |
| 1019 | :exc:`AssertionError`. |
| 1020 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1021 | |
| 1022 | .. attribute:: longMessage |
| 1023 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1024 | If set to ``True`` then any explicit failure message you pass in to the |
| 1025 | :ref:`assert methods <assert-methods>` will be appended to the end of the |
| 1026 | normal failure message. The normal messages contain useful information |
| 1027 | about the objects involved, for example the message from assertEqual |
| 1028 | shows you the repr of the two unequal objects. Setting this attribute |
| 1029 | to ``True`` allows you to have a custom error message in addition to the |
| 1030 | normal one. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1031 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1032 | This attribute defaults to ``False``, meaning that a custom message passed |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1033 | to an assert method will silence the normal message. |
| 1034 | |
| 1035 | The class setting can be overridden in individual tests by assigning an |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1036 | instance attribute to ``True`` or ``False`` before calling the assert methods. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1037 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 1038 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1039 | |
| 1040 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1041 | Testing frameworks can use the following methods to collect information on |
| 1042 | the test: |
| 1043 | |
| 1044 | |
| 1045 | .. method:: countTestCases() |
| 1046 | |
| 1047 | Return the number of tests represented by this test object. For |
| 1048 | :class:`TestCase` instances, this will always be ``1``. |
| 1049 | |
| 1050 | |
| 1051 | .. method:: defaultTestResult() |
| 1052 | |
| 1053 | Return an instance of the test result class that should be used for this |
| 1054 | test case class (if no other result instance is provided to the |
| 1055 | :meth:`run` method). |
| 1056 | |
| 1057 | For :class:`TestCase` instances, this will always be an instance of |
| 1058 | :class:`TestResult`; subclasses of :class:`TestCase` should override this |
| 1059 | as necessary. |
| 1060 | |
| 1061 | |
| 1062 | .. method:: id() |
| 1063 | |
| 1064 | Return a string identifying the specific test case. This is usually the |
| 1065 | full name of the test method, including the module and class name. |
| 1066 | |
| 1067 | |
| 1068 | .. method:: shortDescription() |
| 1069 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1070 | Returns a description of the test, or ``None`` if no description |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1071 | has been provided. The default implementation of this method |
| 1072 | returns the first line of the test method's docstring, if available, |
| 1073 | along with the method name. |
| 1074 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 1075 | .. versionchanged:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1076 | In earlier versions this only returned the first line of the test |
| 1077 | method's docstring, if available or the :const:`None`. That led to |
| 1078 | undesirable behavior of not printing the test name when someone was |
| 1079 | thoughtful enough to write a docstring. |
| 1080 | |
| 1081 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1082 | .. method:: addCleanup(function, *args, **kwargs) |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1083 | |
| 1084 | Add a function to be called after :meth:`tearDown` to cleanup resources |
| 1085 | used during the test. Functions will be called in reverse order to the |
| 1086 | order they are added (LIFO). They are called with any arguments and |
| 1087 | keyword arguments passed into :meth:`addCleanup` when they are |
| 1088 | added. |
| 1089 | |
| 1090 | If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called, |
| 1091 | then any cleanup functions added will still be called. |
| 1092 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1093 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1094 | |
| 1095 | |
| 1096 | .. method:: doCleanups() |
| 1097 | |
Barry Warsaw | 14e0a43 | 2010-04-12 14:55:03 +0000 | [diff] [blame] | 1098 | This method is called unconditionally after :meth:`tearDown`, or |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1099 | after :meth:`setUp` if :meth:`setUp` raises an exception. |
| 1100 | |
| 1101 | It is responsible for calling all the cleanup functions added by |
| 1102 | :meth:`addCleanup`. If you need cleanup functions to be called |
| 1103 | *prior* to :meth:`tearDown` then you can call :meth:`doCleanups` |
| 1104 | yourself. |
| 1105 | |
| 1106 | :meth:`doCleanups` pops methods off the stack of cleanup |
| 1107 | functions one at a time, so it can be called at any time. |
| 1108 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1109 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1110 | |
| 1111 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1112 | .. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1113 | |
| 1114 | This class implements the portion of the :class:`TestCase` interface which |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1115 | allows the test runner to drive the test, but does not provide the methods |
| 1116 | which test code can use to check and report errors. This is used to create |
| 1117 | test cases using legacy test code, allowing it to be integrated into a |
| 1118 | :mod:`unittest`-based test framework. |
| 1119 | |
| 1120 | |
| 1121 | Deprecated aliases |
| 1122 | ################## |
| 1123 | |
| 1124 | For historical reasons, some of the :class:`TestCase` methods had one or more |
| 1125 | aliases that are now deprecated. The following table lists the correct names |
| 1126 | along with their deprecated aliases: |
| 1127 | |
| 1128 | ============================== =============================== |
| 1129 | Method Name Deprecated alias(es) |
| 1130 | ============================== =============================== |
| 1131 | :meth:`.assertEqual` failUnlessEqual, assertEquals |
| 1132 | :meth:`.assertNotEqual` failIfEqual |
| 1133 | :meth:`.assertTrue` failUnless, assert\_ |
| 1134 | :meth:`.assertFalse` failIf |
| 1135 | :meth:`.assertRaises` failUnlessRaises |
| 1136 | :meth:`.assertAlmostEqual` failUnlessAlmostEqual |
| 1137 | :meth:`.assertNotAlmostEqual` failIfAlmostEqual |
| 1138 | ============================== =============================== |
| 1139 | |
| 1140 | .. deprecated:: 3.1 |
| 1141 | the aliases listed in the second column |
| 1142 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1143 | |
| 1144 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1145 | .. _testsuite-objects: |
| 1146 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1147 | Grouping tests |
| 1148 | ~~~~~~~~~~~~~~ |
| 1149 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1150 | .. class:: TestSuite(tests=()) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1151 | |
| 1152 | This class represents an aggregation of individual tests cases and test suites. |
| 1153 | The class presents the interface needed by the test runner to allow it to be run |
| 1154 | as any other test case. Running a :class:`TestSuite` instance is the same as |
| 1155 | iterating over the suite, running each test individually. |
| 1156 | |
| 1157 | If *tests* is given, it must be an iterable of individual test cases or other |
| 1158 | test suites that will be used to build the suite initially. Additional methods |
| 1159 | are provided to add test cases and suites to the collection later on. |
| 1160 | |
Benjamin Peterson | 14a3dd7 | 2009-05-25 00:51:58 +0000 | [diff] [blame] | 1161 | :class:`TestSuite` objects behave much like :class:`TestCase` objects, except |
| 1162 | they do not actually implement a test. Instead, they are used to aggregate |
| 1163 | tests into groups of tests that should be run together. Some additional |
| 1164 | methods are available to add tests to :class:`TestSuite` instances: |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1165 | |
| 1166 | |
| 1167 | .. method:: TestSuite.addTest(test) |
| 1168 | |
| 1169 | Add a :class:`TestCase` or :class:`TestSuite` to the suite. |
| 1170 | |
| 1171 | |
| 1172 | .. method:: TestSuite.addTests(tests) |
| 1173 | |
| 1174 | Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite` |
| 1175 | instances to this test suite. |
| 1176 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1177 | This is equivalent to iterating over *tests*, calling :meth:`addTest` for |
| 1178 | each element. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1179 | |
| 1180 | :class:`TestSuite` shares the following methods with :class:`TestCase`: |
| 1181 | |
| 1182 | |
| 1183 | .. method:: run(result) |
| 1184 | |
| 1185 | Run the tests associated with this suite, collecting the result into the |
| 1186 | test result object passed as *result*. Note that unlike |
| 1187 | :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to |
| 1188 | be passed in. |
| 1189 | |
| 1190 | |
| 1191 | .. method:: debug() |
| 1192 | |
| 1193 | Run the tests associated with this suite without collecting the |
| 1194 | result. This allows exceptions raised by the test to be propagated to the |
| 1195 | caller and can be used to support running tests under a debugger. |
| 1196 | |
| 1197 | |
| 1198 | .. method:: countTestCases() |
| 1199 | |
| 1200 | Return the number of tests represented by this test object, including all |
| 1201 | individual tests and sub-suites. |
| 1202 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1203 | |
| 1204 | .. method:: __iter__() |
| 1205 | |
| 1206 | Tests grouped by a :class:`TestSuite` are always accessed by iteration. |
| 1207 | Subclasses can lazily provide tests by overriding :meth:`__iter__`. Note |
| 1208 | that this method maybe called several times on a single suite |
| 1209 | (for example when counting tests or comparing for equality) |
| 1210 | so the tests returned must be the same for repeated iterations. |
| 1211 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1212 | In the typical usage of a :class:`TestSuite` object, the :meth:`run` method |
| 1213 | is invoked by a :class:`TestRunner` rather than by the end-user test harness. |
| 1214 | |
| 1215 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1216 | Loading and running tests |
| 1217 | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1218 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1219 | .. class:: TestLoader() |
| 1220 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1221 | The :class:`TestLoader` class is used to create test suites from classes and |
| 1222 | modules. Normally, there is no need to create an instance of this class; the |
| 1223 | :mod:`unittest` module provides an instance that can be shared as |
| 1224 | ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows |
| 1225 | customization of some configurable properties. |
| 1226 | |
| 1227 | :class:`TestLoader` objects have the following methods: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1228 | |
| 1229 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1230 | .. method:: loadTestsFromTestCase(testCaseClass) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1231 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1232 | Return a suite of all tests cases contained in the :class:`TestCase`\ -derived |
| 1233 | :class:`testCaseClass`. |
| 1234 | |
| 1235 | |
| 1236 | .. method:: loadTestsFromModule(module) |
| 1237 | |
| 1238 | Return a suite of all tests cases contained in the given module. This |
| 1239 | method searches *module* for classes derived from :class:`TestCase` and |
| 1240 | creates an instance of the class for each test method defined for the |
| 1241 | class. |
| 1242 | |
Georg Brandl | e720c0a | 2009-04-27 16:20:50 +0000 | [diff] [blame] | 1243 | .. note:: |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1244 | |
| 1245 | While using a hierarchy of :class:`TestCase`\ -derived classes can be |
| 1246 | convenient in sharing fixtures and helper functions, defining test |
| 1247 | methods on base classes that are not intended to be instantiated |
| 1248 | directly does not play well with this method. Doing so, however, can |
| 1249 | be useful when the fixtures are different and defined in subclasses. |
| 1250 | |
| 1251 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1252 | .. method:: loadTestsFromName(name, module=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1253 | |
| 1254 | Return a suite of all tests cases given a string specifier. |
| 1255 | |
| 1256 | The specifier *name* is a "dotted name" that may resolve either to a |
| 1257 | module, a test case class, a test method within a test case class, a |
| 1258 | :class:`TestSuite` instance, or a callable object which returns a |
| 1259 | :class:`TestCase` or :class:`TestSuite` instance. These checks are |
| 1260 | applied in the order listed here; that is, a method on a possible test |
| 1261 | case class will be picked up as "a test method within a test case class", |
| 1262 | rather than "a callable object". |
| 1263 | |
| 1264 | For example, if you have a module :mod:`SampleTests` containing a |
| 1265 | :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test |
| 1266 | methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1267 | specifier ``'SampleTests.SampleTestCase'`` would cause this method to |
| 1268 | return a suite which will run all three test methods. Using the specifier |
| 1269 | ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test |
| 1270 | suite which will run only the :meth:`test_two` test method. The specifier |
| 1271 | can refer to modules and packages which have not been imported; they will |
| 1272 | be imported as a side-effect. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1273 | |
| 1274 | The method optionally resolves *name* relative to the given *module*. |
| 1275 | |
| 1276 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1277 | .. method:: loadTestsFromNames(names, module=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1278 | |
| 1279 | Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather |
| 1280 | than a single name. The return value is a test suite which supports all |
| 1281 | the tests defined for each name. |
| 1282 | |
| 1283 | |
| 1284 | .. method:: getTestCaseNames(testCaseClass) |
| 1285 | |
| 1286 | Return a sorted sequence of method names found within *testCaseClass*; |
| 1287 | this should be a subclass of :class:`TestCase`. |
| 1288 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1289 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1290 | The following attributes of a :class:`TestLoader` can be configured either by |
| 1291 | subclassing or assignment on an instance: |
| 1292 | |
| 1293 | |
| 1294 | .. attribute:: testMethodPrefix |
| 1295 | |
| 1296 | String giving the prefix of method names which will be interpreted as test |
| 1297 | methods. The default value is ``'test'``. |
| 1298 | |
| 1299 | This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` |
| 1300 | methods. |
| 1301 | |
| 1302 | |
| 1303 | .. attribute:: sortTestMethodsUsing |
| 1304 | |
| 1305 | Function to be used to compare method names when sorting them in |
| 1306 | :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. |
| 1307 | |
| 1308 | |
| 1309 | .. attribute:: suiteClass |
| 1310 | |
| 1311 | Callable object that constructs a test suite from a list of tests. No |
| 1312 | methods on the resulting object are needed. The default value is the |
| 1313 | :class:`TestSuite` class. |
| 1314 | |
| 1315 | This affects all the :meth:`loadTestsFrom\*` methods. |
| 1316 | |
| 1317 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1318 | .. class:: TestResult |
| 1319 | |
| 1320 | This class is used to compile information about which tests have succeeded |
| 1321 | and which have failed. |
| 1322 | |
| 1323 | A :class:`TestResult` object stores the results of a set of tests. The |
| 1324 | :class:`TestCase` and :class:`TestSuite` classes ensure that results are |
| 1325 | properly recorded; test authors do not need to worry about recording the |
| 1326 | outcome of tests. |
| 1327 | |
| 1328 | Testing frameworks built on top of :mod:`unittest` may want access to the |
| 1329 | :class:`TestResult` object generated by running a set of tests for reporting |
| 1330 | purposes; a :class:`TestResult` instance is returned by the |
| 1331 | :meth:`TestRunner.run` method for this purpose. |
| 1332 | |
| 1333 | :class:`TestResult` instances have the following attributes that will be of |
| 1334 | interest when inspecting the results of running a set of tests: |
| 1335 | |
| 1336 | |
| 1337 | .. attribute:: errors |
| 1338 | |
| 1339 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1340 | holding formatted tracebacks. Each tuple represents a test which raised an |
| 1341 | unexpected exception. |
| 1342 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1343 | .. attribute:: failures |
| 1344 | |
| 1345 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1346 | holding formatted tracebacks. Each tuple represents a test where a failure |
| 1347 | was explicitly signalled using the :meth:`TestCase.fail\*` or |
| 1348 | :meth:`TestCase.assert\*` methods. |
| 1349 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1350 | .. attribute:: skipped |
| 1351 | |
| 1352 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1353 | holding the reason for skipping the test. |
| 1354 | |
Benjamin Peterson | 70e32c8 | 2009-03-24 01:00:11 +0000 | [diff] [blame] | 1355 | .. versionadded:: 3.1 |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1356 | |
| 1357 | .. attribute:: expectedFailures |
| 1358 | |
Georg Brandl | 4b05466 | 2010-10-06 08:56:53 +0000 | [diff] [blame] | 1359 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1360 | holding formatted tracebacks. Each tuple represents an expected failure |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1361 | of the test case. |
| 1362 | |
| 1363 | .. attribute:: unexpectedSuccesses |
| 1364 | |
| 1365 | A list containing :class:`TestCase` instances that were marked as expected |
| 1366 | failures, but succeeded. |
| 1367 | |
| 1368 | .. attribute:: shouldStop |
| 1369 | |
| 1370 | Set to ``True`` when the execution of tests should stop by :meth:`stop`. |
| 1371 | |
| 1372 | |
| 1373 | .. attribute:: testsRun |
| 1374 | |
| 1375 | The total number of tests run so far. |
| 1376 | |
| 1377 | |
| 1378 | .. method:: wasSuccessful() |
| 1379 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1380 | Return ``True`` if all tests run so far have passed, otherwise returns |
| 1381 | ``False``. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1382 | |
| 1383 | |
| 1384 | .. method:: stop() |
| 1385 | |
| 1386 | This method can be called to signal that the set of tests being run should |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1387 | be aborted by setting the :attr:`shouldStop` attribute to ``True``. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1388 | :class:`TestRunner` objects should respect this flag and return without |
| 1389 | running any additional tests. |
| 1390 | |
| 1391 | For example, this feature is used by the :class:`TextTestRunner` class to |
| 1392 | stop the test framework when the user signals an interrupt from the |
| 1393 | keyboard. Interactive tools which provide :class:`TestRunner` |
| 1394 | implementations can use this in a similar manner. |
| 1395 | |
| 1396 | The following methods of the :class:`TestResult` class are used to maintain |
| 1397 | the internal data structures, and may be extended in subclasses to support |
| 1398 | additional reporting requirements. This is particularly useful in building |
| 1399 | tools which support interactive reporting while tests are being run. |
| 1400 | |
| 1401 | |
| 1402 | .. method:: startTest(test) |
| 1403 | |
| 1404 | Called when the test case *test* is about to be run. |
| 1405 | |
| 1406 | The default implementation simply increments the instance's :attr:`testsRun` |
| 1407 | counter. |
| 1408 | |
| 1409 | |
| 1410 | .. method:: stopTest(test) |
| 1411 | |
| 1412 | Called after the test case *test* has been executed, regardless of the |
| 1413 | outcome. |
| 1414 | |
| 1415 | The default implementation does nothing. |
| 1416 | |
| 1417 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1418 | .. method:: startTestRun(test) |
| 1419 | |
| 1420 | Called once before any tests are executed. |
| 1421 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1422 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1423 | |
| 1424 | |
| 1425 | .. method:: stopTestRun(test) |
| 1426 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1427 | Called once after all tests are executed. |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1428 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1429 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1430 | |
| 1431 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1432 | .. method:: addError(test, err) |
| 1433 | |
| 1434 | Called when the test case *test* raises an unexpected exception *err* is a |
| 1435 | tuple of the form returned by :func:`sys.exc_info`: ``(type, value, |
| 1436 | traceback)``. |
| 1437 | |
| 1438 | The default implementation appends a tuple ``(test, formatted_err)`` to |
| 1439 | the instance's :attr:`errors` attribute, where *formatted_err* is a |
| 1440 | formatted traceback derived from *err*. |
| 1441 | |
| 1442 | |
| 1443 | .. method:: addFailure(test, err) |
| 1444 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1445 | Called when the test case *test* signals a failure. *err* is a tuple of |
| 1446 | the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1447 | |
| 1448 | The default implementation appends a tuple ``(test, formatted_err)`` to |
| 1449 | the instance's :attr:`failures` attribute, where *formatted_err* is a |
| 1450 | formatted traceback derived from *err*. |
| 1451 | |
| 1452 | |
| 1453 | .. method:: addSuccess(test) |
| 1454 | |
| 1455 | Called when the test case *test* succeeds. |
| 1456 | |
| 1457 | The default implementation does nothing. |
| 1458 | |
| 1459 | |
| 1460 | .. method:: addSkip(test, reason) |
| 1461 | |
| 1462 | Called when the test case *test* is skipped. *reason* is the reason the |
| 1463 | test gave for skipping. |
| 1464 | |
| 1465 | The default implementation appends a tuple ``(test, reason)`` to the |
| 1466 | instance's :attr:`skipped` attribute. |
| 1467 | |
| 1468 | |
| 1469 | .. method:: addExpectedFailure(test, err) |
| 1470 | |
| 1471 | Called when the test case *test* fails, but was marked with the |
| 1472 | :func:`expectedFailure` decorator. |
| 1473 | |
| 1474 | The default implementation appends a tuple ``(test, formatted_err)`` to |
| 1475 | the instance's :attr:`expectedFailures` attribute, where *formatted_err* |
| 1476 | is a formatted traceback derived from *err*. |
| 1477 | |
| 1478 | |
| 1479 | .. method:: addUnexpectedSuccess(test) |
| 1480 | |
| 1481 | Called when the test case *test* was marked with the |
| 1482 | :func:`expectedFailure` decorator, but succeeded. |
| 1483 | |
| 1484 | The default implementation appends the test to the instance's |
| 1485 | :attr:`unexpectedSuccesses` attribute. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1486 | |
| 1487 | |
| 1488 | .. data:: defaultTestLoader |
| 1489 | |
| 1490 | Instance of the :class:`TestLoader` class intended to be shared. If no |
| 1491 | customization of the :class:`TestLoader` is needed, this instance can be used |
| 1492 | instead of repeatedly creating new instances. |
| 1493 | |
| 1494 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1495 | .. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1496 | |
| 1497 | A basic test runner implementation which prints results on standard error. It |
| 1498 | has a few configurable parameters, but is essentially very simple. Graphical |
| 1499 | applications which run test suites should provide alternate implementations. |
| 1500 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1501 | .. method:: _makeResult() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1502 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1503 | This method returns the instance of ``TestResult`` used by :meth:`run`. |
| 1504 | It is not intended to be called directly, but can be overridden in |
| 1505 | subclasses to provide a custom ``TestResult``. |
| 1506 | |
| 1507 | |
Georg Brandl | b044b2a | 2009-09-16 16:05:59 +0000 | [diff] [blame] | 1508 | .. function:: main(module='__main__', defaultTest=None, argv=None, testRunner=TextTestRunner, testLoader=unittest.defaultTestLoader, exit=True) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1509 | |
| 1510 | A command-line program that runs a set of tests; this is primarily for making |
| 1511 | test modules conveniently executable. The simplest use for this function is to |
| 1512 | include the following line at the end of a test script:: |
| 1513 | |
| 1514 | if __name__ == '__main__': |
| 1515 | unittest.main() |
| 1516 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame^] | 1517 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1518 | The *testRunner* argument can either be a test runner class or an already |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1519 | created instance of it. By default ``main`` calls :func:`sys.exit` with |
| 1520 | an exit code indicating success or failure of the tests run. |
| 1521 | |
| 1522 | ``main`` supports being used from the interactive interpreter by passing in the |
| 1523 | argument ``exit=False``. This displays the result on standard output without |
| 1524 | calling :func:`sys.exit`:: |
| 1525 | |
| 1526 | >>> from unittest import main |
| 1527 | >>> main(module='test_module', exit=False) |
| 1528 | |
| 1529 | Calling ``main`` actually returns an instance of the ``TestProgram`` class. |
| 1530 | This stores the result of the tests run as the ``result`` attribute. |
| 1531 | |
Georg Brandl | c62efa8 | 2010-07-11 10:41:07 +0000 | [diff] [blame] | 1532 | .. versionchanged:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1533 | The ``exit`` parameter was added. |