blob: 30df44e7fb714bbfcaf640ba48f4bc9d1facc603 [file] [log] [blame]
Georg Brandl8ec7f652007-08-15 14:28:01 +00001:mod:`unittest` --- Unit testing framework
2==========================================
3
4.. module:: unittest
5 :synopsis: Unit testing framework for Python.
6.. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
7.. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
8.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
9.. sectionauthor:: Raymond Hettinger <python@rcn.com>
10
Georg Brandl8ec7f652007-08-15 14:28:01 +000011.. versionadded:: 2.1
Benjamin Peterson692428e2009-03-23 21:50:21 +000012
Ezio Melottidd7c5932011-03-10 23:00:48 +020013(If you are already familiar with the basic concepts of testing, you might want
14to skip to :ref:`the list of assert methods <assert-methods>`.)
15
Georg Brandl8ec7f652007-08-15 14:28:01 +000016The Python unit testing framework, sometimes referred to as "PyUnit," is a
17Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
18turn, a Java version of Kent's Smalltalk testing framework. Each is the de
19facto standard unit testing framework for its respective language.
20
21:mod:`unittest` supports test automation, sharing of setup and shutdown code for
22tests, aggregation of tests into collections, and independence of the tests from
23the reporting framework. The :mod:`unittest` module provides classes that make
24it easy to support these qualities for a set of tests.
25
26To achieve this, :mod:`unittest` supports some important concepts:
27
28test fixture
29 A :dfn:`test fixture` represents the preparation needed to perform one or more
30 tests, and any associate cleanup actions. This may involve, for example,
31 creating temporary or proxy databases, directories, or starting a server
32 process.
33
34test case
35 A :dfn:`test case` is the smallest unit of testing. It checks for a specific
36 response to a particular set of inputs. :mod:`unittest` provides a base class,
37 :class:`TestCase`, which may be used to create new test cases.
38
39test suite
40 A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
41 used to aggregate tests that should be executed together.
42
43test runner
44 A :dfn:`test runner` is a component which orchestrates the execution of tests
45 and provides the outcome to the user. The runner may use a graphical interface,
46 a textual interface, or return a special value to indicate the results of
47 executing the tests.
48
49The test case and test fixture concepts are supported through the
50:class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
51used when creating new tests, and the latter can be used when integrating
52existing test code with a :mod:`unittest`\ -driven framework. When building test
Benjamin Peterson99721e02009-03-23 23:10:14 +000053fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and
54:meth:`~TestCase.tearDown` methods can be overridden to provide initialization
55and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions
56can be passed to the constructor for these purposes. When the test is run, the
57fixture initialization is run first; if it succeeds, the cleanup method is run
58after the test has been executed, regardless of the outcome of the test. Each
59instance of the :class:`TestCase` will only be used to run a single test method,
60so a new fixture is created for each test.
Georg Brandl8ec7f652007-08-15 14:28:01 +000061
62Test suites are implemented by the :class:`TestSuite` class. This class allows
63individual tests and test suites to be aggregated; when the suite is executed,
Benjamin Peterson176a56c2009-05-25 00:48:58 +000064all tests added directly to the suite and in "child" test suites are run.
Georg Brandl8ec7f652007-08-15 14:28:01 +000065
Benjamin Peterson99721e02009-03-23 23:10:14 +000066A test runner is an object that provides a single method,
67:meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite`
68object as a parameter, and returns a result object. The class
69:class:`TestResult` is provided for use as the result object. :mod:`unittest`
70provides the :class:`TextTestRunner` as an example test runner which reports
71test results on the standard error stream by default. Alternate runners can be
72implemented for other environments (such as graphical environments) without any
73need to derive from a specific class.
Georg Brandl8ec7f652007-08-15 14:28:01 +000074
75
76.. seealso::
77
78 Module :mod:`doctest`
79 Another test-support module with a very different flavor.
80
Michael Foordba097ec2010-04-03 17:03:11 +000081 `unittest2: A backport of new unittest features for Python 2.4-2.6 <http://pypi.python.org/pypi/unittest2>`_
82 Many new features were added to unittest in Python 2.7, including test
83 discovery. unittest2 allows you to use these features with earlier
84 versions of Python.
85
Georg Brandld198b762009-05-31 14:15:25 +000086 `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_
Georg Brandl2fcd1732009-05-30 10:45:40 +000087 Kent Beck's original paper on testing frameworks using the pattern shared
88 by :mod:`unittest`.
Georg Brandl8ec7f652007-08-15 14:28:01 +000089
Georg Brandld198b762009-05-31 14:15:25 +000090 `Nose <http://code.google.com/p/python-nose/>`_ and `py.test <http://pytest.org>`_
Georg Brandl2fcd1732009-05-30 10:45:40 +000091 Third-party unittest frameworks with a lighter-weight syntax for writing
92 tests. For example, ``assert func(10) == 42``.
Raymond Hettinger21b617b2009-03-24 00:17:11 +000093
Michael Foordba097ec2010-04-03 17:03:11 +000094 `The Python Testing Tools Taxonomy <http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy>`_
95 An extensive list of Python testing tools including functional testing
96 frameworks and mock object libraries.
Georg Brandl8ec7f652007-08-15 14:28:01 +000097
Michael Foordba097ec2010-04-03 17:03:11 +000098 `Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_
99 A special-interest-group for discussion of testing, and testing tools,
100 in Python.
Michael Foordb4a81c82009-05-29 20:33:46 +0000101
Michael Foord5d31e052009-05-11 17:59:43 +0000102
Georg Brandl8ec7f652007-08-15 14:28:01 +0000103.. _unittest-minimal-example:
104
105Basic example
106-------------
107
108The :mod:`unittest` module provides a rich set of tools for constructing and
109running tests. This section demonstrates that a small subset of the tools
110suffice to meet the needs of most users.
111
112Here is a short script to test three functions from the :mod:`random` module::
113
114 import random
115 import unittest
116
117 class TestSequenceFunctions(unittest.TestCase):
118
119 def setUp(self):
120 self.seq = range(10)
121
Benjamin Peterson99721e02009-03-23 23:10:14 +0000122 def test_shuffle(self):
Georg Brandl8ec7f652007-08-15 14:28:01 +0000123 # make sure the shuffled sequence does not lose any elements
124 random.shuffle(self.seq)
125 self.seq.sort()
126 self.assertEqual(self.seq, range(10))
127
Raymond Hettinger08090bf2010-03-09 08:44:18 +0000128 # should raise an exception for an immutable sequence
129 self.assertRaises(TypeError, random.shuffle, (1,2,3))
130
Benjamin Peterson99721e02009-03-23 23:10:14 +0000131 def test_choice(self):
Georg Brandl8ec7f652007-08-15 14:28:01 +0000132 element = random.choice(self.seq)
Ezio Melotti3cbb66b2011-03-10 23:35:39 +0200133 self.assertTrue(element in self.seq)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000134
Benjamin Peterson99721e02009-03-23 23:10:14 +0000135 def test_sample(self):
Raymond Hettinger08090bf2010-03-09 08:44:18 +0000136 with self.assertRaises(ValueError):
137 random.sample(self.seq, 20)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000138 for element in random.sample(self.seq, 5):
Ezio Melotti3cbb66b2011-03-10 23:35:39 +0200139 self.assertTrue(element in self.seq)
Georg Brandl8ec7f652007-08-15 14:28:01 +0000140
141 if __name__ == '__main__':
142 unittest.main()
143
Benjamin Peterson99721e02009-03-23 23:10:14 +0000144A testcase is created by subclassing :class:`unittest.TestCase`. The three
Georg Brandl8ec7f652007-08-15 14:28:01 +0000145individual tests are defined with methods whose names start with the letters
146``test``. This naming convention informs the test runner about which methods
147represent tests.
148
Benjamin Peterson99721e02009-03-23 23:10:14 +0000149The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
Michael Foorddb43b5a2010-02-10 14:25:12 +0000150expected result; :meth:`~TestCase.assertTrue` to verify a condition; or
Benjamin Peterson99721e02009-03-23 23:10:14 +0000151:meth:`~TestCase.assertRaises` to verify that an expected exception gets raised.
152These methods are used instead of the :keyword:`assert` statement so the test
153runner can accumulate all test results and produce a report.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000154
Benjamin Peterson99721e02009-03-23 23:10:14 +0000155When a :meth:`~TestCase.setUp` method is defined, the test runner will run that
156method prior to each test. Likewise, if a :meth:`~TestCase.tearDown` method is
157defined, the test runner will invoke that method after each test. In the
158example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each
159test.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000160
161The final block shows a simple way to run the tests. :func:`unittest.main`
Éric Araujoa8132ec2010-12-16 03:53:53 +0000162provides a command-line interface to the test script. When run from the command
Georg Brandl8ec7f652007-08-15 14:28:01 +0000163line, the above script produces an output that looks like this::
164
165 ...
166 ----------------------------------------------------------------------
167 Ran 3 tests in 0.000s
168
169 OK
170
171Instead of :func:`unittest.main`, there are other ways to run the tests with a
172finer level of control, less terse output, and no requirement to be run from the
173command line. For example, the last two lines may be replaced with::
174
175 suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
176 unittest.TextTestRunner(verbosity=2).run(suite)
177
178Running the revised script from the interpreter or another script produces the
179following output::
180
Ezio Melotti68beef62010-02-28 03:11:07 +0000181 test_choice (__main__.TestSequenceFunctions) ... ok
182 test_sample (__main__.TestSequenceFunctions) ... ok
183 test_shuffle (__main__.TestSequenceFunctions) ... ok
Georg Brandl8ec7f652007-08-15 14:28:01 +0000184
185 ----------------------------------------------------------------------
186 Ran 3 tests in 0.110s
187
188 OK
189
190The above examples show the most commonly used :mod:`unittest` features which
191are sufficient to meet many everyday testing needs. The remainder of the
192documentation explores the full feature set from first principles.
193
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000194
195.. _unittest-command-line-interface:
196
Ezio Melottic3ab30b2011-03-12 22:21:37 +0200197Command-Line Interface
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000198----------------------
199
200The unittest module can be used from the command line to run tests from
201modules, classes or even individual test methods::
202
203 python -m unittest test_module1 test_module2
204 python -m unittest test_module.TestClass
205 python -m unittest test_module.TestClass.test_method
206
207You can pass in a list with any combination of module names, and fully
208qualified class or method names.
209
210You can run tests with more detail (higher verbosity) by passing in the -v flag::
211
Ezio Melotti062d2b52009-12-19 22:41:49 +0000212 python -m unittest -v test_module
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000213
Éric Araujoa8132ec2010-12-16 03:53:53 +0000214For a list of all the command-line options::
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000215
216 python -m unittest -h
217
218.. versionchanged:: 2.7
219 In earlier versions it was only possible to run individual test methods and
220 not modules or classes.
221
Michael Foordba097ec2010-04-03 17:03:11 +0000222
Éric Araujoa8132ec2010-12-16 03:53:53 +0000223Command-line options
224~~~~~~~~~~~~~~~~~~~~
Michael Foordba097ec2010-04-03 17:03:11 +0000225
Éric Araujoa8132ec2010-12-16 03:53:53 +0000226:program:`unittest` supports these command-line options:
Michael Foordba097ec2010-04-03 17:03:11 +0000227
Éric Araujoa8132ec2010-12-16 03:53:53 +0000228.. program:: unittest
Michael Foordba097ec2010-04-03 17:03:11 +0000229
Éric Araujoa8132ec2010-12-16 03:53:53 +0000230.. cmdoption:: -b, --buffer
Michael Foordba097ec2010-04-03 17:03:11 +0000231
Éric Araujoa8132ec2010-12-16 03:53:53 +0000232 The standard output and standard error streams are buffered during the test
233 run. Output during a passing test is discarded. Output is echoed normally
234 on test fail or error and is added to the failure messages.
Andrew M. Kuchlingfb759a22010-04-29 01:44:30 +0000235
Éric Araujoa8132ec2010-12-16 03:53:53 +0000236.. cmdoption:: -c, --catch
Andrew M. Kuchlingfb759a22010-04-29 01:44:30 +0000237
Éric Araujoa8132ec2010-12-16 03:53:53 +0000238 Control-C during the test run waits for the current test to end and then
239 reports all the results so far. A second control-C raises the normal
240 :exc:`KeyboardInterrupt` exception.
Andrew M. Kuchlingfb759a22010-04-29 01:44:30 +0000241
Éric Araujoa8132ec2010-12-16 03:53:53 +0000242 See `Signal Handling`_ for the functions that provide this functionality.
Andrew M. Kuchlingfb759a22010-04-29 01:44:30 +0000243
Éric Araujoa8132ec2010-12-16 03:53:53 +0000244.. cmdoption:: -f, --failfast
Andrew M. Kuchlingfb759a22010-04-29 01:44:30 +0000245
Éric Araujoa8132ec2010-12-16 03:53:53 +0000246 Stop the test run on the first error or failure.
247
248.. versionadded:: 2.7
249 The command-line options ``-b``, ``-c`` and ``-f`` were added.
Michael Foordba097ec2010-04-03 17:03:11 +0000250
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000251The command line can also be used for test discovery, for running all of the
252tests in a project or just a subset.
253
254
255.. _unittest-test-discovery:
256
257Test Discovery
258--------------
259
260.. versionadded:: 2.7
261
Ezio Melotti9e1ed472011-03-08 17:08:25 +0200262Unittest supports simple test discovery. In order to be compatible with test
263discovery, all of the test files must be :ref:`modules <tut-modules>` or
264:ref:`packages <tut-packages>` importable from the top-level directory of
265the project (this means that their filenames must be valid
266:ref:`identifiers <identifiers>`).
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000267
268Test discovery is implemented in :meth:`TestLoader.discover`, but can also be
Éric Araujoa8132ec2010-12-16 03:53:53 +0000269used from the command line. The basic command-line usage is::
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000270
271 cd project_directory
272 python -m unittest discover
273
274The ``discover`` sub-command has the following options:
275
Éric Araujoa8132ec2010-12-16 03:53:53 +0000276.. program:: unittest discover
277
278.. cmdoption:: -v, --verbose
279
280 Verbose output
281
Chris Jerdonek13cee162013-02-21 18:52:12 -0800282.. cmdoption:: -s, --start-directory directory
Éric Araujoa8132ec2010-12-16 03:53:53 +0000283
Chris Jerdonek13cee162013-02-21 18:52:12 -0800284 Directory to start discovery (``.`` default)
Éric Araujoa8132ec2010-12-16 03:53:53 +0000285
Chris Jerdonek13cee162013-02-21 18:52:12 -0800286.. cmdoption:: -p, --pattern pattern
Éric Araujoa8132ec2010-12-16 03:53:53 +0000287
Chris Jerdonek13cee162013-02-21 18:52:12 -0800288 Pattern to match test files (``test*.py`` default)
Éric Araujoa8132ec2010-12-16 03:53:53 +0000289
Chris Jerdonek13cee162013-02-21 18:52:12 -0800290.. cmdoption:: -t, --top-level-directory directory
Éric Araujoa8132ec2010-12-16 03:53:53 +0000291
292 Top level directory of project (defaults to start directory)
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000293
Andrew M. Kuchling60383182010-04-30 01:32:47 +0000294The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in
295as positional arguments in that order. The following two command lines
296are equivalent::
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000297
Ezio Melotti7b4e02c2010-01-27 20:25:11 +0000298 python -m unittest discover -s project_directory -p '*_test.py'
299 python -m unittest discover project_directory '*_test.py'
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000300
Michael Foord8851b712010-05-08 15:09:37 +0000301As well as being a path it is possible to pass a package name, for example
302``myproject.subpackage.test``, as the start directory. The package name you
303supply will then be imported and its location on the filesystem will be used
304as the start directory.
305
306.. caution::
307
308 Test discovery loads tests by importing them. Once test discovery has
309 found all the test files from the start directory you specify it turns the
Éric Araujoa7cbe282011-09-01 19:49:31 +0200310 paths into package names to import. For example :file:`foo/bar/baz.py` will be
Michael Foord8851b712010-05-08 15:09:37 +0000311 imported as ``foo.bar.baz``.
312
313 If you have a package installed globally and attempt test discovery on
314 a different copy of the package then the import *could* happen from the
315 wrong place. If this happens test discovery will warn you and exit.
316
317 If you supply the start directory as a package name rather than a
318 path to a directory then discover assumes that whichever location it
319 imports from is the location you intended, so you will not get the
320 warning.
321
Raymond Hettingerb09f1982009-05-29 21:20:41 +0000322Test modules and packages can customize test loading and discovery by through
323the `load_tests protocol`_.
324
325
Georg Brandl8ec7f652007-08-15 14:28:01 +0000326.. _organizing-tests:
327
328Organizing test code
329--------------------
330
331The basic building blocks of unit testing are :dfn:`test cases` --- single
332scenarios that must be set up and checked for correctness. In :mod:`unittest`,
333test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`
334class. To make your own test cases you must write subclasses of
335:class:`TestCase`, or use :class:`FunctionTestCase`.
336
337An instance of a :class:`TestCase`\ -derived class is an object that can
338completely run a single test method, together with optional set-up and tidy-up
339code.
340
341The testing code of a :class:`TestCase` instance should be entirely self
342contained, such that it can be run either in isolation or in arbitrary
343combination with any number of other test cases.
344
Benjamin Peterson99721e02009-03-23 23:10:14 +0000345The simplest :class:`TestCase` subclass will simply override the
346:meth:`~TestCase.runTest` method in order to perform specific testing code::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000347
348 import unittest
349
350 class DefaultWidgetSizeTestCase(unittest.TestCase):
351 def runTest(self):
352 widget = Widget('The widget')
353 self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
354
Sandro Tosi2d71e5c2012-01-21 10:59:12 +0100355Note that in order to test something, we use one of the :meth:`assert\*`
Georg Brandl2fcd1732009-05-30 10:45:40 +0000356methods provided by the :class:`TestCase` base class. If the test fails, an
357exception will be raised, and :mod:`unittest` will identify the test case as a
358:dfn:`failure`. Any other exceptions will be treated as :dfn:`errors`. This
359helps you identify where the problem is: :dfn:`failures` are caused by incorrect
360results - a 5 where you expected a 6. :dfn:`Errors` are caused by incorrect
361code - e.g., a :exc:`TypeError` caused by an incorrect function call.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000362
363The way to run a test case will be described later. For now, note that to
364construct an instance of such a test case, we call its constructor without
365arguments::
366
367 testCase = DefaultWidgetSizeTestCase()
368
369Now, such test cases can be numerous, and their set-up can be repetitive. In
370the above case, constructing a :class:`Widget` in each of 100 Widget test case
371subclasses would mean unsightly duplication.
372
373Luckily, we can factor out such set-up code by implementing a method called
Benjamin Peterson99721e02009-03-23 23:10:14 +0000374:meth:`~TestCase.setUp`, which the testing framework will automatically call for
375us when we run the test::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000376
377 import unittest
378
379 class SimpleWidgetTestCase(unittest.TestCase):
380 def setUp(self):
381 self.widget = Widget('The widget')
382
383 class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
384 def runTest(self):
Ezio Melotti85ee3e12010-02-04 20:06:38 +0000385 self.assertEqual(self.widget.size(), (50,50),
386 'incorrect default size')
Georg Brandl8ec7f652007-08-15 14:28:01 +0000387
388 class WidgetResizeTestCase(SimpleWidgetTestCase):
389 def runTest(self):
390 self.widget.resize(100,150)
Ezio Melotti85ee3e12010-02-04 20:06:38 +0000391 self.assertEqual(self.widget.size(), (100,150),
392 'wrong size after resize')
Georg Brandl8ec7f652007-08-15 14:28:01 +0000393
Benjamin Peterson99721e02009-03-23 23:10:14 +0000394If the :meth:`~TestCase.setUp` method raises an exception while the test is
395running, the framework will consider the test to have suffered an error, and the
396:meth:`~TestCase.runTest` method will not be executed.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000397
Benjamin Peterson99721e02009-03-23 23:10:14 +0000398Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
399after the :meth:`~TestCase.runTest` method has been run::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000400
401 import unittest
402
403 class SimpleWidgetTestCase(unittest.TestCase):
404 def setUp(self):
405 self.widget = Widget('The widget')
406
407 def tearDown(self):
408 self.widget.dispose()
409 self.widget = None
410
Benjamin Peterson99721e02009-03-23 23:10:14 +0000411If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will
412be run whether :meth:`~TestCase.runTest` succeeded or not.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000413
414Such a working environment for the testing code is called a :dfn:`fixture`.
415
416Often, many small test cases will use the same fixture. In this case, we would
417end up subclassing :class:`SimpleWidgetTestCase` into many small one-method
418classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and
Georg Brandl8ec7f652007-08-15 14:28:01 +0000419discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler
420mechanism::
421
422 import unittest
423
424 class WidgetTestCase(unittest.TestCase):
425 def setUp(self):
426 self.widget = Widget('The widget')
427
428 def tearDown(self):
429 self.widget.dispose()
430 self.widget = None
431
Ezio Melotti68beef62010-02-28 03:11:07 +0000432 def test_default_size(self):
Ezio Melotti85ee3e12010-02-04 20:06:38 +0000433 self.assertEqual(self.widget.size(), (50,50),
434 'incorrect default size')
Georg Brandl8ec7f652007-08-15 14:28:01 +0000435
Ezio Melotti68beef62010-02-28 03:11:07 +0000436 def test_resize(self):
Georg Brandl8ec7f652007-08-15 14:28:01 +0000437 self.widget.resize(100,150)
Ezio Melotti85ee3e12010-02-04 20:06:38 +0000438 self.assertEqual(self.widget.size(), (100,150),
439 'wrong size after resize')
Georg Brandl8ec7f652007-08-15 14:28:01 +0000440
Benjamin Peterson99721e02009-03-23 23:10:14 +0000441Here we have not provided a :meth:`~TestCase.runTest` method, but have instead
442provided two different test methods. Class instances will now each run one of
Ezio Melotti68beef62010-02-28 03:11:07 +0000443the :meth:`test_\*` methods, with ``self.widget`` created and destroyed
Benjamin Peterson99721e02009-03-23 23:10:14 +0000444separately for each instance. When creating an instance we must specify the
445test method it is to run. We do this by passing the method name in the
446constructor::
Georg Brandl8ec7f652007-08-15 14:28:01 +0000447
Ezio Melotti68beef62010-02-28 03:11:07 +0000448 defaultSizeTestCase = WidgetTestCase('test_default_size')
449 resizeTestCase = WidgetTestCase('test_resize')
Georg Brandl8ec7f652007-08-15 14:28:01 +0000450
451Test case instances are grouped together according to the features they test.
452:mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,
453represented by :mod:`unittest`'s :class:`TestSuite` class::
454
455 widgetTestSuite = unittest.TestSuite()
Ezio Melotti68beef62010-02-28 03:11:07 +0000456 widgetTestSuite.addTest(WidgetTestCase('test_default_size'))
457 widgetTestSuite.addTest(WidgetTestCase('test_resize'))
Georg Brandl8ec7f652007-08-15 14:28:01 +0000458
459For the ease of running tests, as we will see later, it is a good idea to
460provide in each test module a callable object that returns a pre-built test
461suite::
462
463 def suite():
464 suite = unittest.TestSuite()
Ezio Melotti68beef62010-02-28 03:11:07 +0000465 suite.addTest(WidgetTestCase('test_default_size'))
466 suite.addTest(WidgetTestCase('test_resize'))
Georg Brandl8ec7f652007-08-15 14:28:01 +0000467 return suite
468
469or even::
470
471 def suite():
Ezio Melotti68beef62010-02-28 03:11:07 +0000472 tests = ['test_default_size', 'test_resize']
Georg Brandl8ec7f652007-08-15 14:28:01 +0000473
474 return unittest.TestSuite(map(WidgetTestCase, tests))
475
476Since it is a common pattern to create a :class:`TestCase` subclass with many
477similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`
478class that can be used to automate the process of creating a test suite and
479populating it with individual tests. For example, ::
480
481 suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
482
Ezio Melotti68beef62010-02-28 03:11:07 +0000483will create a test suite that will run ``WidgetTestCase.test_default_size()`` and
484``WidgetTestCase.test_resize``. :class:`TestLoader` uses the ``'test'`` method
Georg Brandl8ec7f652007-08-15 14:28:01 +0000485name prefix to identify test methods automatically.
486
Ezio Melottidd7c5932011-03-10 23:00:48 +0200487Note that the order in which the various test cases will be run is
488determined by sorting the test function names with respect to the
489built-in ordering for strings.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000490
491Often it is desirable to group suites of test cases together, so as to run tests
492for the whole system at once. This is easy, since :class:`TestSuite` instances
493can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be
494added to a :class:`TestSuite`::
495
496 suite1 = module1.TheTestSuite()
497 suite2 = module2.TheTestSuite()
498 alltests = unittest.TestSuite([suite1, suite2])
499
500You can place the definitions of test cases and test suites in the same modules
501as the code they are to test (such as :file:`widget.py`), but there are several
502advantages to placing the test code in a separate module, such as
503:file:`test_widget.py`:
504
505* The test module can be run standalone from the command line.
506
507* The test code can more easily be separated from shipped code.
508
509* There is less temptation to change test code to fit the code it tests without
510 a good reason.
511
512* Test code should be modified much less frequently than the code it tests.
513
514* Tested code can be refactored more easily.
515
516* Tests for modules written in C must be in separate modules anyway, so why not
517 be consistent?
518
519* If the testing strategy changes, there is no need to change the source code.
520
521
522.. _legacy-unit-tests:
523
524Re-using old test code
525----------------------
526
527Some users will find that they have existing test code that they would like to
528run from :mod:`unittest`, without converting every old test function to a
529:class:`TestCase` subclass.
530
531For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
532This subclass of :class:`TestCase` can be used to wrap an existing test
533function. Set-up and tear-down functions can also be provided.
534
535Given the following test function::
536
537 def testSomething():
538 something = makeSomething()
539 assert something.name is not None
540 # ...
541
542one can create an equivalent test case instance as follows::
543
544 testcase = unittest.FunctionTestCase(testSomething)
545
546If there are additional set-up and tear-down methods that should be called as
547part of the test case's operation, they can also be provided like so::
548
549 testcase = unittest.FunctionTestCase(testSomething,
550 setUp=makeSomethingDB,
551 tearDown=deleteSomethingDB)
552
553To make migrating existing test suites easier, :mod:`unittest` supports tests
554raising :exc:`AssertionError` to indicate test failure. However, it is
555recommended that you use the explicit :meth:`TestCase.fail\*` and
556:meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`
557may treat :exc:`AssertionError` differently.
558
559.. note::
560
Georg Brandl2fcd1732009-05-30 10:45:40 +0000561 Even though :class:`FunctionTestCase` can be used to quickly convert an
562 existing test base over to a :mod:`unittest`\ -based system, this approach is
563 not recommended. Taking the time to set up proper :class:`TestCase`
564 subclasses will make future test refactorings infinitely easier.
Georg Brandl8ec7f652007-08-15 14:28:01 +0000565
Benjamin Peterson99721e02009-03-23 23:10:14 +0000566In some cases, the existing tests may have been written using the :mod:`doctest`
567module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
568automatically build :class:`unittest.TestSuite` instances from the existing
569:mod:`doctest`\ -based tests.
570
Georg Brandl8ec7f652007-08-15 14:28:01 +0000571
Benjamin Peterson692428e2009-03-23 21:50:21 +0000572.. _unittest-skipping:
573
574Skipping tests and expected failures
575------------------------------------
576
Michael Foordfb0844b2010-02-05 21:45:12 +0000577.. versionadded:: 2.7
578
Benjamin Peterson692428e2009-03-23 21:50:21 +0000579Unittest supports skipping individual test methods and even whole classes of
580tests. In addition, it supports marking a test as a "expected failure," a test
581that is broken and will fail, but shouldn't be counted as a failure on a
582:class:`TestResult`.
583
584Skipping a test is simply a matter of using the :func:`skip` :term:`decorator`
585or one of its conditional variants.
586
Ezio Melottiff0deb02013-03-01 21:26:04 +0200587Basic skipping looks like this::
Benjamin Peterson692428e2009-03-23 21:50:21 +0000588
589 class MyTestCase(unittest.TestCase):
590
591 @unittest.skip("demonstrating skipping")
592 def test_nothing(self):
593 self.fail("shouldn't happen")
594
Georg Brandl2fcd1732009-05-30 10:45:40 +0000595 @unittest.skipIf(mylib.__version__ < (1, 3),
596 "not supported in this library version")
Benjamin Petersonbe76d4c2009-03-29 03:16:57 +0000597 def test_format(self):
Benjamin Peterson097aafd2009-03-29 03:39:58 +0000598 # Tests that work for only a certain version of the library.
Benjamin Petersonbe76d4c2009-03-29 03:16:57 +0000599 pass
600
601 @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
602 def test_windows_support(self):
603 # windows specific testing code
604 pass
605
Ezio Melottiff0deb02013-03-01 21:26:04 +0200606This is the output of running the example above in verbose mode::
Benjamin Peterson692428e2009-03-23 21:50:21 +0000607
Benjamin Peterson097aafd2009-03-29 03:39:58 +0000608 test_format (__main__.MyTestCase) ... skipped 'not supported in this library version'
Benjamin Peterson692428e2009-03-23 21:50:21 +0000609 test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping'
Benjamin Petersonbe76d4c2009-03-29 03:16:57 +0000610 test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows'
Benjamin Peterson692428e2009-03-23 21:50:21 +0000611
612 ----------------------------------------------------------------------
Benjamin Petersonbe76d4c2009-03-29 03:16:57 +0000613 Ran 3 tests in 0.005s
614
615 OK (skipped=3)
Benjamin Peterson692428e2009-03-23 21:50:21 +0000616
Ezio Melottiff0deb02013-03-01 21:26:04 +0200617Classes can be skipped just like methods::
Benjamin Peterson692428e2009-03-23 21:50:21 +0000618
Sandro Tosi6ca845c2012-03-31 18:34:42 +0200619 @unittest.skip("showing class skipping")
Benjamin Peterson692428e2009-03-23 21:50:21 +0000620 class MySkippedTestCase(unittest.TestCase):
621 def test_not_run(self):
622 pass
623
Benjamin Peterson31b78062009-03-23 23:13:36 +0000624:meth:`TestCase.setUp` can also skip the test. This is useful when a resource
625that needs to be set up is not available.
626
Benjamin Peterson692428e2009-03-23 21:50:21 +0000627Expected failures use the :func:`expectedFailure` decorator. ::
628
629 class ExpectedFailureTestCase(unittest.TestCase):
630 @unittest.expectedFailure
631 def test_fail(self):
632 self.assertEqual(1, 0, "broken")
633
634It's easy to roll your own skipping decorators by making a decorator that calls
635:func:`skip` on the test when it wants it to be skipped. This decorator skips
Ezio Melottiff0deb02013-03-01 21:26:04 +0200636the test unless the passed object has a certain attribute::
Benjamin Peterson692428e2009-03-23 21:50:21 +0000637
638 def skipUnlessHasattr(obj, attr):
639 if hasattr(obj, attr):
640 return lambda func: func
Ezio Melotti352def02013-03-27 20:11:55 +0200641 return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))
Benjamin Peterson692428e2009-03-23 21:50:21 +0000642
643The following decorators implement test skipping and expected failures:
644
645.. function:: skip(reason)
646
647 Unconditionally skip the decorated test. *reason* should describe why the
648 test is being skipped.
649
650.. function:: skipIf(condition, reason)
651
652 Skip the decorated test if *condition* is true.
653
654.. function:: skipUnless(condition, reason)
655
Georg Brandl09302282010-10-06 09:32:48 +0000656 Skip the decorated test unless *condition* is true.
Benjamin Peterson692428e2009-03-23 21:50:21 +0000657
658.. function:: expectedFailure
659
660 Mark the test as an expected failure. If the test fails when run, the test
661 is not counted as a failure.
662
Ezio Melotti352def02013-03-27 20:11:55 +0200663.. exception:: SkipTest(reason)
664
665 This exception is raised to skip a test.
666
667 Usually you can use :meth:`TestCase.skipTest` or one of the skipping
668 decorators instead of raising this directly.
669
Michael Foord09e29802010-04-04 22:41:54 +0000670Skipped tests will not have :meth:`setUp` or :meth:`tearDown` run around them.
671Skipped classes will not have :meth:`setUpClass` or :meth:`tearDownClass` run.
Michael Foordba097ec2010-04-03 17:03:11 +0000672
Benjamin Peterson692428e2009-03-23 21:50:21 +0000673
Georg Brandl8ec7f652007-08-15 14:28:01 +0000674.. _unittest-contents:
675
676Classes and functions
677---------------------
678
Benjamin Peterson99721e02009-03-23 23:10:14 +0000679This section describes in depth the API of :mod:`unittest`.
680
681
682.. _testcase-objects:
683
684Test cases
685~~~~~~~~~~
Georg Brandl8ec7f652007-08-15 14:28:01 +0000686
Ezio Melottidd7c5932011-03-10 23:00:48 +0200687.. class:: TestCase(methodName='runTest')
Georg Brandl8ec7f652007-08-15 14:28:01 +0000688
689 Instances of the :class:`TestCase` class represent the smallest testable units
690 in the :mod:`unittest` universe. This class is intended to be used as a base
691 class, with specific tests being implemented by concrete subclasses. This class
692 implements the interface needed by the test runner to allow it to drive the
693 test, and methods that the test code can use to check for and report various
694 kinds of failure.
695
696 Each instance of :class:`TestCase` will run a single test method: the method
697 named *methodName*. If you remember, we had an earlier example that went
698 something like this::
699
700 def suite():
701 suite = unittest.TestSuite()
Ezio Melotti68beef62010-02-28 03:11:07 +0000702 suite.addTest(WidgetTestCase('test_default_size'))
703 suite.addTest(WidgetTestCase('test_resize'))
Georg Brandl8ec7f652007-08-15 14:28:01 +0000704 return suite
705
706 Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
707 single test.
708
Benjamin Peterson99721e02009-03-23 23:10:14 +0000709 *methodName* defaults to :meth:`runTest`.
710
711 :class:`TestCase` instances provide three groups of methods: one group used
712 to run the test, another used by the test implementation to check conditions
713 and report failures, and some inquiry methods allowing information about the
714 test itself to be gathered.
715
716 Methods in the first group (running the test) are:
717
718
719 .. method:: setUp()
720
721 Method called to prepare the test fixture. This is called immediately
722 before calling the test method; any exception raised by this method will
723 be considered an error rather than a test failure. The default
724 implementation does nothing.
725
726
727 .. method:: tearDown()
728
729 Method called immediately after the test method has been called and the
730 result recorded. This is called even if the test method raised an
731 exception, so the implementation in subclasses may need to be particularly
732 careful about checking internal state. Any exception raised by this
733 method will be considered an error rather than a test failure. This
734 method will only be called if the :meth:`setUp` succeeds, regardless of
735 the outcome of the test method. The default implementation does nothing.
736
737
Michael Foordba097ec2010-04-03 17:03:11 +0000738 .. method:: setUpClass()
739
740 A class method called before tests in an individual class run.
741 ``setUpClass`` is called with the class as the only argument
Michael Foord09e29802010-04-04 22:41:54 +0000742 and must be decorated as a :func:`classmethod`::
Michael Foordba097ec2010-04-03 17:03:11 +0000743
744 @classmethod
745 def setUpClass(cls):
746 ...
747
748 See `Class and Module Fixtures`_ for more details.
749
750 .. versionadded:: 2.7
751
752
753 .. method:: tearDownClass()
754
755 A class method called after tests in an individual class have run.
756 ``tearDownClass`` is called with the class as the only argument
757 and must be decorated as a :meth:`classmethod`::
758
759 @classmethod
760 def tearDownClass(cls):
761 ...
762
763 See `Class and Module Fixtures`_ for more details.
764
765 .. versionadded:: 2.7
766
767
Ezio Melottidd7c5932011-03-10 23:00:48 +0200768 .. method:: run(result=None)
Benjamin Peterson99721e02009-03-23 23:10:14 +0000769
770 Run the test, collecting the result into the test result object passed as
Ezio Melottidd7c5932011-03-10 23:00:48 +0200771 *result*. If *result* is omitted or ``None``, a temporary result
Ezio Melottic2f5a592009-06-30 22:51:06 +0000772 object is created (by calling the :meth:`defaultTestResult` method) and
773 used. The result object is not returned to :meth:`run`'s caller.
Benjamin Peterson99721e02009-03-23 23:10:14 +0000774
775 The same effect may be had by simply calling the :class:`TestCase`
776 instance.
777
778
Benjamin Peterson47d97382009-03-26 20:05:50 +0000779 .. method:: skipTest(reason)
Benjamin Peterson99721e02009-03-23 23:10:14 +0000780
Stefan Krah4a769052010-05-19 15:59:40 +0000781 Calling this during a test method or :meth:`setUp` skips the current
Benjamin Peterson31b78062009-03-23 23:13:36 +0000782 test. See :ref:`unittest-skipping` for more information.
Benjamin Peterson99721e02009-03-23 23:10:14 +0000783
Georg Brandl90aae552010-04-10 11:15:24 +0000784 .. versionadded:: 2.7
785
Benjamin Peterson99721e02009-03-23 23:10:14 +0000786
787 .. method:: debug()
788
789 Run the test without collecting the result. This allows exceptions raised
790 by the test to be propagated to the caller, and can be used to support
791 running tests under a debugger.
792
Ezio Melottidd7c5932011-03-10 23:00:48 +0200793 .. _assert-methods:
Benjamin Peterson99721e02009-03-23 23:10:14 +0000794
Ezio Melottidd7c5932011-03-10 23:00:48 +0200795 The :class:`TestCase` class provides a number of methods to check for and
796 report failures, such as:
Benjamin Peterson99721e02009-03-23 23:10:14 +0000797
Ezio Melottidd7c5932011-03-10 23:00:48 +0200798 +-----------------------------------------+-----------------------------+---------------+
799 | Method | Checks that | New in |
800 +=========================================+=============================+===============+
801 | :meth:`assertEqual(a, b) | ``a == b`` | |
802 | <TestCase.assertEqual>` | | |
803 +-----------------------------------------+-----------------------------+---------------+
804 | :meth:`assertNotEqual(a, b) | ``a != b`` | |
805 | <TestCase.assertNotEqual>` | | |
806 +-----------------------------------------+-----------------------------+---------------+
807 | :meth:`assertTrue(x) | ``bool(x) is True`` | |
808 | <TestCase.assertTrue>` | | |
809 +-----------------------------------------+-----------------------------+---------------+
810 | :meth:`assertFalse(x) | ``bool(x) is False`` | |
811 | <TestCase.assertFalse>` | | |
812 +-----------------------------------------+-----------------------------+---------------+
813 | :meth:`assertIs(a, b) | ``a is b`` | 2.7 |
814 | <TestCase.assertIs>` | | |
815 +-----------------------------------------+-----------------------------+---------------+
816 | :meth:`assertIsNot(a, b) | ``a is not b`` | 2.7 |
817 | <TestCase.assertIsNot>` | | |
818 +-----------------------------------------+-----------------------------+---------------+
819 | :meth:`assertIsNone(x) | ``x is None`` | 2.7 |
820 | <TestCase.assertIsNone>` | | |
821 +-----------------------------------------+-----------------------------+---------------+
822 | :meth:`assertIsNotNone(x) | ``x is not None`` | 2.7 |
823 | <TestCase.assertIsNotNone>` | | |
824 +-----------------------------------------+-----------------------------+---------------+
825 | :meth:`assertIn(a, b) | ``a in b`` | 2.7 |
826 | <TestCase.assertIn>` | | |
827 +-----------------------------------------+-----------------------------+---------------+
828 | :meth:`assertNotIn(a, b) | ``a not in b`` | 2.7 |
829 | <TestCase.assertNotIn>` | | |
830 +-----------------------------------------+-----------------------------+---------------+
831 | :meth:`assertIsInstance(a, b) | ``isinstance(a, b)`` | 2.7 |
832 | <TestCase.assertIsInstance>` | | |
833 +-----------------------------------------+-----------------------------+---------------+
834 | :meth:`assertNotIsInstance(a, b) | ``not isinstance(a, b)`` | 2.7 |
835 | <TestCase.assertNotIsInstance>` | | |
836 +-----------------------------------------+-----------------------------+---------------+
Benjamin Peterson99721e02009-03-23 23:10:14 +0000837
Ezio Melottidd7c5932011-03-10 23:00:48 +0200838 All the assert methods (except :meth:`assertRaises`,
839 :meth:`assertRaisesRegexp`)
840 accept a *msg* argument that, if specified, is used as the error message on
841 failure (see also :data:`longMessage`).
Benjamin Peterson99721e02009-03-23 23:10:14 +0000842
Ezio Melottidd7c5932011-03-10 23:00:48 +0200843 .. method:: assertEqual(first, second, msg=None)
Benjamin Peterson99721e02009-03-23 23:10:14 +0000844
845 Test that *first* and *second* are equal. If the values do not compare
Ezio Melottidd7c5932011-03-10 23:00:48 +0200846 equal, the test will fail.
Gregory P. Smith28399852009-03-31 16:54:10 +0000847
848 In addition, if *first* and *second* are the exact same type and one of
Michael Foordfe6349c2010-02-08 22:41:16 +0000849 list, tuple, dict, set, frozenset or unicode or any type that a subclass
Ezio Melotti055d70d2012-01-16 08:21:24 +0200850 registers with :meth:`addTypeEqualityFunc` the type-specific equality
Ezio Melottidd7c5932011-03-10 23:00:48 +0200851 function will be called in order to generate a more useful default
852 error message (see also the :ref:`list of type-specific methods
853 <type-specific-methods>`).
Gregory P. Smith28399852009-03-31 16:54:10 +0000854
855 .. versionchanged:: 2.7
Ezio Melotti055d70d2012-01-16 08:21:24 +0200856 Added the automatic calling of type-specific equality function.
Benjamin Peterson99721e02009-03-23 23:10:14 +0000857
Gregory P. Smith65ff0052009-03-31 19:59:14 +0000858
Ezio Melottidd7c5932011-03-10 23:00:48 +0200859 .. method:: assertNotEqual(first, second, msg=None)
Benjamin Peterson99721e02009-03-23 23:10:14 +0000860
861 Test that *first* and *second* are not equal. If the values do compare
Ezio Melottidd7c5932011-03-10 23:00:48 +0200862 equal, the test will fail.
Benjamin Peterson99721e02009-03-23 23:10:14 +0000863
Ezio Melottidd7c5932011-03-10 23:00:48 +0200864 .. method:: assertTrue(expr, msg=None)
865 assertFalse(expr, msg=None)
866
867 Test that *expr* is true (or false).
868
869 Note that this is equivalent to ``bool(expr) is True`` and not to ``expr
870 is True`` (use ``assertIs(expr, True)`` for the latter). This method
871 should also be avoided when more specific methods are available (e.g.
872 ``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they
873 provide a better error message in case of failure.
Gregory P. Smith65ff0052009-03-31 19:59:14 +0000874
Benjamin Peterson99721e02009-03-23 23:10:14 +0000875
Ezio Melottidd7c5932011-03-10 23:00:48 +0200876 .. method:: assertIs(first, second, msg=None)
877 assertIsNot(first, second, msg=None)
Benjamin Peterson99721e02009-03-23 23:10:14 +0000878
Ezio Melottidd7c5932011-03-10 23:00:48 +0200879 Test that *first* and *second* evaluate (or don't evaluate) to the same object.
Gregory P. Smith28399852009-03-31 16:54:10 +0000880
881 .. versionadded:: 2.7
882
883
Ezio Melottidd7c5932011-03-10 23:00:48 +0200884 .. method:: assertIsNone(expr, msg=None)
885 assertIsNotNone(expr, msg=None)
Gregory P. Smith28399852009-03-31 16:54:10 +0000886
Ezio Melottidd7c5932011-03-10 23:00:48 +0200887 Test that *expr* is (or is not) None.
Michael Foordba097ec2010-04-03 17:03:11 +0000888
889 .. versionadded:: 2.7
890
891
Gregory P. Smith28399852009-03-31 16:54:10 +0000892 .. method:: assertIn(first, second, msg=None)
893 assertNotIn(first, second, msg=None)
894
Ezio Melottidd7c5932011-03-10 23:00:48 +0200895 Test that *first* is (or is not) in *second*.
Gregory P. Smith28399852009-03-31 16:54:10 +0000896
897 .. versionadded:: 2.7
898
899
Ezio Melottidd7c5932011-03-10 23:00:48 +0200900 .. method:: assertIsInstance(obj, cls, msg=None)
901 assertNotIsInstance(obj, cls, msg=None)
Gregory P. Smith28399852009-03-31 16:54:10 +0000902
Ezio Melottidd7c5932011-03-10 23:00:48 +0200903 Test that *obj* is (or is not) an instance of *cls* (which can be a
904 class or a tuple of classes, as supported by :func:`isinstance`).
Ezio Melotti080b6f02011-12-19 07:04:48 +0200905 To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.
Gregory P. Smith28399852009-03-31 16:54:10 +0000906
907 .. versionadded:: 2.7
908
909
Ezio Melottidd7c5932011-03-10 23:00:48 +0200910 It is also possible to check that exceptions and warnings are raised using
911 the following methods:
Gregory P. Smith28399852009-03-31 16:54:10 +0000912
Ezio Melottidd7c5932011-03-10 23:00:48 +0200913 +---------------------------------------------------------+--------------------------------------+------------+
914 | Method | Checks that | New in |
915 +=========================================================+======================================+============+
Éric Araujoa7cbe282011-09-01 19:49:31 +0200916 | :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | |
Ezio Melottidd7c5932011-03-10 23:00:48 +0200917 | <TestCase.assertRaises>` | | |
918 +---------------------------------------------------------+--------------------------------------+------------+
Éric Araujoa7cbe282011-09-01 19:49:31 +0200919 | :meth:`assertRaisesRegexp(exc, re, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 2.7 |
920 | <TestCase.assertRaisesRegexp>` | and the message matches *re* | |
Ezio Melottidd7c5932011-03-10 23:00:48 +0200921 +---------------------------------------------------------+--------------------------------------+------------+
Gregory P. Smith28399852009-03-31 16:54:10 +0000922
Ezio Melottidd7c5932011-03-10 23:00:48 +0200923 .. method:: assertRaises(exception, callable, *args, **kwds)
924 assertRaises(exception)
Benjamin Peterson99721e02009-03-23 23:10:14 +0000925
926 Test that an exception is raised when *callable* is called with any
927 positional or keyword arguments that are also passed to
928 :meth:`assertRaises`. The test passes if *exception* is raised, is an
929 error if another exception is raised, or fails if no exception is raised.
930 To catch any of a group of exceptions, a tuple containing the exception
931 classes may be passed as *exception*.
932
Ezio Melottidd7c5932011-03-10 23:00:48 +0200933 If only the *exception* argument is given, returns a context manager so
934 that the code under test can be written inline rather than as a function::
Benjamin Peterson7233acc2009-03-29 03:31:40 +0000935
Michael Foord1f3fa8a2010-02-05 21:07:38 +0000936 with self.assertRaises(SomeException):
Benjamin Peterson7233acc2009-03-29 03:31:40 +0000937 do_something()
938
Kristján Valur Jónssone2a77982009-08-27 22:20:21 +0000939 The context manager will store the caught exception object in its
Georg Brandldc3694b2010-02-07 17:02:22 +0000940 :attr:`exception` attribute. This can be useful if the intention
Michael Foord1f3fa8a2010-02-05 21:07:38 +0000941 is to perform additional checks on the exception raised::
942
943 with self.assertRaises(SomeException) as cm:
944 do_something()
945
Georg Brandldc3694b2010-02-07 17:02:22 +0000946 the_exception = cm.exception
Michael Foordba7732e2010-02-05 23:28:12 +0000947 self.assertEqual(the_exception.error_code, 3)
Kristján Valur Jónssone2a77982009-08-27 22:20:21 +0000948
Benjamin Peterson99721e02009-03-23 23:10:14 +0000949 .. versionchanged:: 2.7
Benjamin Peterson7233acc2009-03-29 03:31:40 +0000950 Added the ability to use :meth:`assertRaises` as a context manager.
Benjamin Peterson99721e02009-03-23 23:10:14 +0000951
Gregory P. Smith65ff0052009-03-31 19:59:14 +0000952
Ezio Melottidd7c5932011-03-10 23:00:48 +0200953 .. method:: assertRaisesRegexp(exception, regexp, callable, *args, **kwds)
954 assertRaisesRegexp(exception, regexp)
Benjamin Peterson99721e02009-03-23 23:10:14 +0000955
Gregory P. Smith28399852009-03-31 16:54:10 +0000956 Like :meth:`assertRaises` but also tests that *regexp* matches
957 on the string representation of the raised exception. *regexp* may be
958 a regular expression object or a string containing a regular expression
959 suitable for use by :func:`re.search`. Examples::
960
Terry Jan Reedy8d750032013-06-29 13:15:36 -0400961 self.assertRaisesRegexp(ValueError, "invalid literal for.*XYZ'$",
Gregory P. Smith28399852009-03-31 16:54:10 +0000962 int, 'XYZ')
963
964 or::
965
966 with self.assertRaisesRegexp(ValueError, 'literal'):
967 int('XYZ')
968
969 .. versionadded:: 2.7
970
971
Gregory P. Smith28399852009-03-31 16:54:10 +0000972
Ezio Melottidd7c5932011-03-10 23:00:48 +0200973 There are also other methods used to perform more specific checks, such as:
974
975 +---------------------------------------+--------------------------------+--------------+
976 | Method | Checks that | New in |
977 +=======================================+================================+==============+
978 | :meth:`assertAlmostEqual(a, b) | ``round(a-b, 7) == 0`` | |
979 | <TestCase.assertAlmostEqual>` | | |
980 +---------------------------------------+--------------------------------+--------------+
981 | :meth:`assertNotAlmostEqual(a, b) | ``round(a-b, 7) != 0`` | |
982 | <TestCase.assertNotAlmostEqual>` | | |
983 +---------------------------------------+--------------------------------+--------------+
984 | :meth:`assertGreater(a, b) | ``a > b`` | 2.7 |
985 | <TestCase.assertGreater>` | | |
986 +---------------------------------------+--------------------------------+--------------+
987 | :meth:`assertGreaterEqual(a, b) | ``a >= b`` | 2.7 |
988 | <TestCase.assertGreaterEqual>` | | |
989 +---------------------------------------+--------------------------------+--------------+
990 | :meth:`assertLess(a, b) | ``a < b`` | 2.7 |
991 | <TestCase.assertLess>` | | |
992 +---------------------------------------+--------------------------------+--------------+
993 | :meth:`assertLessEqual(a, b) | ``a <= b`` | 2.7 |
994 | <TestCase.assertLessEqual>` | | |
995 +---------------------------------------+--------------------------------+--------------+
996 | :meth:`assertRegexpMatches(s, re) | ``regex.search(s)`` | 2.7 |
997 | <TestCase.assertRegexpMatches>` | | |
998 +---------------------------------------+--------------------------------+--------------+
999 | :meth:`assertNotRegexpMatches(s, re) | ``not regex.search(s)`` | 2.7 |
1000 | <TestCase.assertNotRegexpMatches>` | | |
1001 +---------------------------------------+--------------------------------+--------------+
1002 | :meth:`assertItemsEqual(a, b) | sorted(a) == sorted(b) and | 2.7 |
1003 | <TestCase.assertItemsEqual>` | works with unhashable objs | |
1004 +---------------------------------------+--------------------------------+--------------+
1005 | :meth:`assertDictContainsSubset(a, b) | all the key/value pairs | 2.7 |
Éric Araujoa7cbe282011-09-01 19:49:31 +02001006 | <TestCase.assertDictContainsSubset>` | in *a* exist in *b* | |
Ezio Melottidd7c5932011-03-10 23:00:48 +02001007 +---------------------------------------+--------------------------------+--------------+
1008
1009
1010 .. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None)
1011 assertNotAlmostEqual(first, second, places=7, msg=None, delta=None)
1012
1013 Test that *first* and *second* are approximately (or not approximately)
1014 equal by computing the difference, rounding to the given number of
1015 decimal *places* (default 7), and comparing to zero. Note that these
1016 methods round the values to the given number of *decimal places* (i.e.
1017 like the :func:`round` function) and not *significant digits*.
1018
1019 If *delta* is supplied instead of *places* then the difference
Ezio Melottid5991422013-08-11 13:04:50 +03001020 between *first* and *second* must be less or equal to (or greater than) *delta*.
Ezio Melottidd7c5932011-03-10 23:00:48 +02001021
1022 Supplying both *delta* and *places* raises a ``TypeError``.
1023
1024 .. versionchanged:: 2.7
1025 :meth:`assertAlmostEqual` automatically considers almost equal objects
1026 that compare equal. :meth:`assertNotAlmostEqual` automatically fails
1027 if the objects compare equal. Added the *delta* keyword argument.
1028
1029
1030
1031 .. method:: assertGreater(first, second, msg=None)
1032 assertGreaterEqual(first, second, msg=None)
1033 assertLess(first, second, msg=None)
1034 assertLessEqual(first, second, msg=None)
1035
1036 Test that *first* is respectively >, >=, < or <= than *second* depending
1037 on the method name. If not, the test will fail::
1038
1039 >>> self.assertGreaterEqual(3, 4)
1040 AssertionError: "3" unexpectedly not greater than or equal to "4"
Gregory P. Smith28399852009-03-31 16:54:10 +00001041
1042 .. versionadded:: 2.7
1043
1044
Ezio Melottidd7c5932011-03-10 23:00:48 +02001045 .. method:: assertRegexpMatches(text, regexp, msg=None)
Gregory P. Smith28399852009-03-31 16:54:10 +00001046
Ezio Melottidd7c5932011-03-10 23:00:48 +02001047 Test that a *regexp* search matches *text*. In case
1048 of failure, the error message will include the pattern and the *text* (or
1049 the pattern and the part of *text* that unexpectedly matched). *regexp*
1050 may be a regular expression object or a string containing a regular
1051 expression suitable for use by :func:`re.search`.
Gregory P. Smith28399852009-03-31 16:54:10 +00001052
1053 .. versionadded:: 2.7
1054
1055
Ezio Melottidd7c5932011-03-10 23:00:48 +02001056 .. method:: assertNotRegexpMatches(text, regexp, msg=None)
Michael Foordf2dfef12009-04-05 19:19:28 +00001057
Ezio Melottidd7c5932011-03-10 23:00:48 +02001058 Verifies that a *regexp* search does not match *text*. Fails with an error
1059 message including the pattern and the part of *text* that matches. *regexp*
1060 may be a regular expression object or a string containing a regular
1061 expression suitable for use by :func:`re.search`.
Michael Foordf2dfef12009-04-05 19:19:28 +00001062
1063 .. versionadded:: 2.7
1064
1065
Ezio Melottidd7c5932011-03-10 23:00:48 +02001066 .. method:: assertItemsEqual(actual, expected, msg=None)
Michael Foordf2dfef12009-04-05 19:19:28 +00001067
Ezio Melottidd7c5932011-03-10 23:00:48 +02001068 Test that sequence *expected* contains the same elements as *actual*,
1069 regardless of their order. When they don't, an error message listing the
1070 differences between the sequences will be generated.
1071
1072 Duplicate elements are *not* ignored when comparing *actual* and
1073 *expected*. It verifies if each element has the same count in both
1074 sequences. It is the equivalent of ``assertEqual(sorted(expected),
1075 sorted(actual))`` but it works with sequences of unhashable objects as
1076 well.
Michael Foordf2dfef12009-04-05 19:19:28 +00001077
Ezio Melotti183dc462013-04-29 12:26:01 +03001078 In Python 3, this method is named ``assertCountEqual``.
1079
Michael Foordf2dfef12009-04-05 19:19:28 +00001080 .. versionadded:: 2.7
1081
1082
Ezio Melottidd7c5932011-03-10 23:00:48 +02001083 .. method:: assertDictContainsSubset(expected, actual, msg=None)
Georg Brandlf895cf52009-10-01 20:59:31 +00001084
Ezio Melottidd7c5932011-03-10 23:00:48 +02001085 Tests whether the key/value pairs in dictionary *actual* are a
1086 superset of those in *expected*. If not, an error message listing
1087 the missing keys and mismatched values is generated.
1088
1089 .. versionadded:: 2.7
1090 .. deprecated:: 3.2
1091
1092
1093
1094 .. _type-specific-methods:
1095
1096 The :meth:`assertEqual` method dispatches the equality check for objects of
1097 the same type to different type-specific methods. These methods are already
1098 implemented for most of the built-in types, but it's also possible to
1099 register new methods using :meth:`addTypeEqualityFunc`:
1100
1101 .. method:: addTypeEqualityFunc(typeobj, function)
1102
1103 Registers a type-specific method called by :meth:`assertEqual` to check
1104 if two objects of exactly the same *typeobj* (not subclasses) compare
1105 equal. *function* must take two positional arguments and a third msg=None
1106 keyword argument just as :meth:`assertEqual` does. It must raise
1107 :data:`self.failureException(msg) <failureException>` when inequality
1108 between the first two parameters is detected -- possibly providing useful
1109 information and explaining the inequalities in details in the error
1110 message.
1111
1112 .. versionadded:: 2.7
1113
1114 The list of type-specific methods automatically used by
1115 :meth:`~TestCase.assertEqual` are summarized in the following table. Note
1116 that it's usually not necessary to invoke these methods directly.
1117
1118 +-----------------------------------------+-----------------------------+--------------+
1119 | Method | Used to compare | New in |
1120 +=========================================+=============================+==============+
1121 | :meth:`assertMultiLineEqual(a, b) | strings | 2.7 |
1122 | <TestCase.assertMultiLineEqual>` | | |
1123 +-----------------------------------------+-----------------------------+--------------+
1124 | :meth:`assertSequenceEqual(a, b) | sequences | 2.7 |
1125 | <TestCase.assertSequenceEqual>` | | |
1126 +-----------------------------------------+-----------------------------+--------------+
1127 | :meth:`assertListEqual(a, b) | lists | 2.7 |
1128 | <TestCase.assertListEqual>` | | |
1129 +-----------------------------------------+-----------------------------+--------------+
1130 | :meth:`assertTupleEqual(a, b) | tuples | 2.7 |
1131 | <TestCase.assertTupleEqual>` | | |
1132 +-----------------------------------------+-----------------------------+--------------+
1133 | :meth:`assertSetEqual(a, b) | sets or frozensets | 2.7 |
1134 | <TestCase.assertSetEqual>` | | |
1135 +-----------------------------------------+-----------------------------+--------------+
1136 | :meth:`assertDictEqual(a, b) | dicts | 2.7 |
1137 | <TestCase.assertDictEqual>` | | |
1138 +-----------------------------------------+-----------------------------+--------------+
1139
1140
1141
1142 .. method:: assertMultiLineEqual(first, second, msg=None)
1143
1144 Test that the multiline string *first* is equal to the string *second*.
1145 When not equal a diff of the two strings highlighting the differences
1146 will be included in the error message. This method is used by default
1147 when comparing strings with :meth:`assertEqual`.
Georg Brandlf895cf52009-10-01 20:59:31 +00001148
1149 .. versionadded:: 2.7
1150
1151
Ezio Melottidd7c5932011-03-10 23:00:48 +02001152 .. method:: assertSequenceEqual(seq1, seq2, msg=None, seq_type=None)
Georg Brandlf895cf52009-10-01 20:59:31 +00001153
Ezio Melottidd7c5932011-03-10 23:00:48 +02001154 Tests that two sequences are equal. If a *seq_type* is supplied, both
1155 *seq1* and *seq2* must be instances of *seq_type* or a failure will
1156 be raised. If the sequences are different an error message is
1157 constructed that shows the difference between the two.
1158
1159 This method is not called directly by :meth:`assertEqual`, but
1160 it's used to implement :meth:`assertListEqual` and
1161 :meth:`assertTupleEqual`.
Georg Brandlf895cf52009-10-01 20:59:31 +00001162
1163 .. versionadded:: 2.7
1164
1165
Ezio Melottidd7c5932011-03-10 23:00:48 +02001166 .. method:: assertListEqual(list1, list2, msg=None)
1167 assertTupleEqual(tuple1, tuple2, msg=None)
Gregory P. Smith28399852009-03-31 16:54:10 +00001168
Ezio Melotti6bb9c732012-08-29 17:50:42 +03001169 Tests that two lists or tuples are equal. If not, an error message is
Ezio Melottidd7c5932011-03-10 23:00:48 +02001170 constructed that shows only the differences between the two. An error
1171 is also raised if either of the parameters are of the wrong type.
1172 These methods are used by default when comparing lists or tuples with
1173 :meth:`assertEqual`.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001174
Ezio Melottidd7c5932011-03-10 23:00:48 +02001175 .. versionadded:: 2.7
Gregory P. Smith65ff0052009-03-31 19:59:14 +00001176
Benjamin Peterson99721e02009-03-23 23:10:14 +00001177
Ezio Melottidd7c5932011-03-10 23:00:48 +02001178 .. method:: assertSetEqual(set1, set2, msg=None)
Benjamin Peterson99721e02009-03-23 23:10:14 +00001179
Ezio Melottidd7c5932011-03-10 23:00:48 +02001180 Tests that two sets are equal. If not, an error message is constructed
1181 that lists the differences between the sets. This method is used by
1182 default when comparing sets or frozensets with :meth:`assertEqual`.
1183
1184 Fails if either of *set1* or *set2* does not have a :meth:`set.difference`
1185 method.
1186
1187 .. versionadded:: 2.7
1188
1189
1190 .. method:: assertDictEqual(expected, actual, msg=None)
1191
1192 Test that two dictionaries are equal. If not, an error message is
1193 constructed that shows the differences in the dictionaries. This
1194 method will be used by default to compare dictionaries in
1195 calls to :meth:`assertEqual`.
1196
1197 .. versionadded:: 2.7
1198
1199
1200
1201 .. _other-methods-and-attrs:
1202
1203 Finally the :class:`TestCase` provides the following methods and attributes:
1204
1205
1206 .. method:: fail(msg=None)
1207
1208 Signals a test failure unconditionally, with *msg* or ``None`` for
Benjamin Peterson99721e02009-03-23 23:10:14 +00001209 the error message.
1210
1211
1212 .. attribute:: failureException
1213
1214 This class attribute gives the exception raised by the test method. If a
1215 test framework needs to use a specialized exception, possibly to carry
1216 additional information, it must subclass this exception in order to "play
1217 fair" with the framework. The initial value of this attribute is
1218 :exc:`AssertionError`.
1219
Michael Foord345b2fe2009-04-02 03:20:38 +00001220
1221 .. attribute:: longMessage
1222
Ezio Melottidd7c5932011-03-10 23:00:48 +02001223 If set to ``True`` then any explicit failure message you pass in to the
1224 :ref:`assert methods <assert-methods>` will be appended to the end of the
1225 normal failure message. The normal messages contain useful information
1226 about the objects involved, for example the message from assertEqual
1227 shows you the repr of the two unequal objects. Setting this attribute
1228 to ``True`` allows you to have a custom error message in addition to the
1229 normal one.
Michael Foord345b2fe2009-04-02 03:20:38 +00001230
Ezio Melottidd7c5932011-03-10 23:00:48 +02001231 This attribute defaults to ``False``, meaning that a custom message passed
Michael Foord345b2fe2009-04-02 03:20:38 +00001232 to an assert method will silence the normal message.
1233
1234 The class setting can be overridden in individual tests by assigning an
Ezio Melottidd7c5932011-03-10 23:00:48 +02001235 instance attribute to ``True`` or ``False`` before calling the assert methods.
Michael Foord345b2fe2009-04-02 03:20:38 +00001236
1237 .. versionadded:: 2.7
1238
1239
Michael Foord8dde2012010-06-05 21:57:03 +00001240 .. attribute:: maxDiff
1241
1242 This attribute controls the maximum length of diffs output by assert
1243 methods that report diffs on failure. It defaults to 80*8 characters.
1244 Assert methods affected by this attribute are
1245 :meth:`assertSequenceEqual` (including all the sequence comparison
1246 methods that delegate to it), :meth:`assertDictEqual` and
1247 :meth:`assertMultiLineEqual`.
1248
1249 Setting ``maxDiff`` to None means that there is no maximum length of
1250 diffs.
1251
1252 .. versionadded:: 2.7
1253
1254
Benjamin Peterson99721e02009-03-23 23:10:14 +00001255 Testing frameworks can use the following methods to collect information on
1256 the test:
1257
1258
1259 .. method:: countTestCases()
1260
1261 Return the number of tests represented by this test object. For
1262 :class:`TestCase` instances, this will always be ``1``.
1263
1264
1265 .. method:: defaultTestResult()
1266
1267 Return an instance of the test result class that should be used for this
1268 test case class (if no other result instance is provided to the
1269 :meth:`run` method).
1270
1271 For :class:`TestCase` instances, this will always be an instance of
1272 :class:`TestResult`; subclasses of :class:`TestCase` should override this
1273 as necessary.
1274
1275
1276 .. method:: id()
1277
1278 Return a string identifying the specific test case. This is usually the
1279 full name of the test method, including the module and class name.
1280
1281
1282 .. method:: shortDescription()
1283
Ezio Melottidd7c5932011-03-10 23:00:48 +02001284 Returns a description of the test, or ``None`` if no description
Gregory P. Smith28399852009-03-31 16:54:10 +00001285 has been provided. The default implementation of this method
1286 returns the first line of the test method's docstring, if available,
Michael Foorddb43b5a2010-02-10 14:25:12 +00001287 or :const:`None`.
Gregory P. Smith28399852009-03-31 16:54:10 +00001288
1289
Gregory P. Smith28399852009-03-31 16:54:10 +00001290
Ezio Melottidd7c5932011-03-10 23:00:48 +02001291 .. method:: addCleanup(function, *args, **kwargs)
Michael Foorde2fb98f2009-05-02 20:15:05 +00001292
1293 Add a function to be called after :meth:`tearDown` to cleanup resources
1294 used during the test. Functions will be called in reverse order to the
1295 order they are added (LIFO). They are called with any arguments and
1296 keyword arguments passed into :meth:`addCleanup` when they are
1297 added.
1298
1299 If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called,
1300 then any cleanup functions added will still be called.
1301
1302 .. versionadded:: 2.7
1303
1304
1305 .. method:: doCleanups()
1306
Barry Warsawfa900d42010-04-12 14:40:49 +00001307 This method is called unconditionally after :meth:`tearDown`, or
Michael Foorde2fb98f2009-05-02 20:15:05 +00001308 after :meth:`setUp` if :meth:`setUp` raises an exception.
1309
1310 It is responsible for calling all the cleanup functions added by
1311 :meth:`addCleanup`. If you need cleanup functions to be called
1312 *prior* to :meth:`tearDown` then you can call :meth:`doCleanups`
1313 yourself.
1314
1315 :meth:`doCleanups` pops methods off the stack of cleanup
1316 functions one at a time, so it can be called at any time.
1317
1318 .. versionadded:: 2.7
1319
1320
Ezio Melottidd7c5932011-03-10 23:00:48 +02001321.. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None)
Georg Brandl8ec7f652007-08-15 14:28:01 +00001322
1323 This class implements the portion of the :class:`TestCase` interface which
Georg Brandl2fcd1732009-05-30 10:45:40 +00001324 allows the test runner to drive the test, but does not provide the methods
1325 which test code can use to check and report errors. This is used to create
1326 test cases using legacy test code, allowing it to be integrated into a
1327 :mod:`unittest`-based test framework.
Georg Brandl8ec7f652007-08-15 14:28:01 +00001328
1329
Ezio Melottidd7c5932011-03-10 23:00:48 +02001330Deprecated aliases
1331##################
1332
1333For historical reasons, some of the :class:`TestCase` methods had one or more
1334aliases that are now deprecated. The following table lists the correct names
1335along with their deprecated aliases:
1336
1337 ============================== ===============================
1338 Method Name Deprecated alias(es)
1339 ============================== ===============================
1340 :meth:`.assertEqual` failUnlessEqual, assertEquals
1341 :meth:`.assertNotEqual` failIfEqual
1342 :meth:`.assertTrue` failUnless, assert\_
1343 :meth:`.assertFalse` failIf
1344 :meth:`.assertRaises` failUnlessRaises
1345 :meth:`.assertAlmostEqual` failUnlessAlmostEqual
1346 :meth:`.assertNotAlmostEqual` failIfAlmostEqual
1347 ============================== ===============================
1348
1349 .. deprecated:: 2.7
1350 the aliases listed in the second column
1351
1352
1353
Benjamin Peterson99721e02009-03-23 23:10:14 +00001354.. _testsuite-objects:
1355
1356Grouping tests
1357~~~~~~~~~~~~~~
1358
Ezio Melottidd7c5932011-03-10 23:00:48 +02001359.. class:: TestSuite(tests=())
Georg Brandl8ec7f652007-08-15 14:28:01 +00001360
1361 This class represents an aggregation of individual tests cases and test suites.
1362 The class presents the interface needed by the test runner to allow it to be run
1363 as any other test case. Running a :class:`TestSuite` instance is the same as
1364 iterating over the suite, running each test individually.
1365
1366 If *tests* is given, it must be an iterable of individual test cases or other
1367 test suites that will be used to build the suite initially. Additional methods
1368 are provided to add test cases and suites to the collection later on.
1369
Benjamin Peterson176a56c2009-05-25 00:48:58 +00001370 :class:`TestSuite` objects behave much like :class:`TestCase` objects, except
1371 they do not actually implement a test. Instead, they are used to aggregate
1372 tests into groups of tests that should be run together. Some additional
1373 methods are available to add tests to :class:`TestSuite` instances:
Benjamin Peterson99721e02009-03-23 23:10:14 +00001374
1375
1376 .. method:: TestSuite.addTest(test)
1377
1378 Add a :class:`TestCase` or :class:`TestSuite` to the suite.
1379
1380
1381 .. method:: TestSuite.addTests(tests)
1382
1383 Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
1384 instances to this test suite.
1385
Georg Brandl2fcd1732009-05-30 10:45:40 +00001386 This is equivalent to iterating over *tests*, calling :meth:`addTest` for
1387 each element.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001388
1389 :class:`TestSuite` shares the following methods with :class:`TestCase`:
1390
1391
1392 .. method:: run(result)
1393
1394 Run the tests associated with this suite, collecting the result into the
1395 test result object passed as *result*. Note that unlike
1396 :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
1397 be passed in.
1398
1399
1400 .. method:: debug()
1401
1402 Run the tests associated with this suite without collecting the
1403 result. This allows exceptions raised by the test to be propagated to the
1404 caller and can be used to support running tests under a debugger.
1405
1406
1407 .. method:: countTestCases()
1408
1409 Return the number of tests represented by this test object, including all
1410 individual tests and sub-suites.
1411
Georg Brandl9bc66822009-04-27 17:04:23 +00001412
1413 .. method:: __iter__()
1414
1415 Tests grouped by a :class:`TestSuite` are always accessed by iteration.
1416 Subclasses can lazily provide tests by overriding :meth:`__iter__`. Note
1417 that this method maybe called several times on a single suite
1418 (for example when counting tests or comparing for equality)
1419 so the tests returned must be the same for repeated iterations.
1420
1421 .. versionchanged:: 2.7
1422 In earlier versions the :class:`TestSuite` accessed tests directly rather
1423 than through iteration, so overriding :meth:`__iter__` wasn't sufficient
1424 for providing tests.
1425
Benjamin Peterson99721e02009-03-23 23:10:14 +00001426 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
1427 is invoked by a :class:`TestRunner` rather than by the end-user test harness.
1428
1429
Benjamin Peterson99721e02009-03-23 23:10:14 +00001430Loading and running tests
1431~~~~~~~~~~~~~~~~~~~~~~~~~
1432
Georg Brandl8ec7f652007-08-15 14:28:01 +00001433.. class:: TestLoader()
1434
Benjamin Peterson99721e02009-03-23 23:10:14 +00001435 The :class:`TestLoader` class is used to create test suites from classes and
1436 modules. Normally, there is no need to create an instance of this class; the
1437 :mod:`unittest` module provides an instance that can be shared as
Ezio Melotti217e6a62012-04-29 10:52:18 +03001438 :data:`unittest.defaultTestLoader`. Using a subclass or instance, however,
1439 allows customization of some configurable properties.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001440
1441 :class:`TestLoader` objects have the following methods:
Georg Brandl8ec7f652007-08-15 14:28:01 +00001442
1443
Benjamin Peterson99721e02009-03-23 23:10:14 +00001444 .. method:: loadTestsFromTestCase(testCaseClass)
Georg Brandl8ec7f652007-08-15 14:28:01 +00001445
Benjamin Peterson99721e02009-03-23 23:10:14 +00001446 Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
1447 :class:`testCaseClass`.
1448
1449
1450 .. method:: loadTestsFromModule(module)
1451
1452 Return a suite of all tests cases contained in the given module. This
1453 method searches *module* for classes derived from :class:`TestCase` and
1454 creates an instance of the class for each test method defined for the
1455 class.
1456
Georg Brandl16a57f62009-04-27 15:29:09 +00001457 .. note::
Benjamin Peterson99721e02009-03-23 23:10:14 +00001458
1459 While using a hierarchy of :class:`TestCase`\ -derived classes can be
1460 convenient in sharing fixtures and helper functions, defining test
1461 methods on base classes that are not intended to be instantiated
1462 directly does not play well with this method. Doing so, however, can
1463 be useful when the fixtures are different and defined in subclasses.
1464
Michael Foordb4a81c82009-05-29 20:33:46 +00001465 If a module provides a ``load_tests`` function it will be called to
1466 load the tests. This allows modules to customize test loading.
1467 This is the `load_tests protocol`_.
1468
1469 .. versionchanged:: 2.7
1470 Support for ``load_tests`` added.
1471
Benjamin Peterson99721e02009-03-23 23:10:14 +00001472
Ezio Melottidd7c5932011-03-10 23:00:48 +02001473 .. method:: loadTestsFromName(name, module=None)
Benjamin Peterson99721e02009-03-23 23:10:14 +00001474
1475 Return a suite of all tests cases given a string specifier.
1476
1477 The specifier *name* is a "dotted name" that may resolve either to a
1478 module, a test case class, a test method within a test case class, a
1479 :class:`TestSuite` instance, or a callable object which returns a
1480 :class:`TestCase` or :class:`TestSuite` instance. These checks are
1481 applied in the order listed here; that is, a method on a possible test
1482 case class will be picked up as "a test method within a test case class",
1483 rather than "a callable object".
1484
1485 For example, if you have a module :mod:`SampleTests` containing a
Georg Brandl2fcd1732009-05-30 10:45:40 +00001486 :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
1487 methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
1488 specifier ``'SampleTests.SampleTestCase'`` would cause this method to
1489 return a suite which will run all three test methods. Using the specifier
1490 ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test
1491 suite which will run only the :meth:`test_two` test method. The specifier
1492 can refer to modules and packages which have not been imported; they will
1493 be imported as a side-effect.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001494
1495 The method optionally resolves *name* relative to the given *module*.
1496
1497
Ezio Melottidd7c5932011-03-10 23:00:48 +02001498 .. method:: loadTestsFromNames(names, module=None)
Benjamin Peterson99721e02009-03-23 23:10:14 +00001499
1500 Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
1501 than a single name. The return value is a test suite which supports all
1502 the tests defined for each name.
1503
1504
1505 .. method:: getTestCaseNames(testCaseClass)
1506
1507 Return a sorted sequence of method names found within *testCaseClass*;
1508 this should be a subclass of :class:`TestCase`.
1509
Michael Foordb4a81c82009-05-29 20:33:46 +00001510
1511 .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None)
1512
1513 Find and return all test modules from the specified start directory,
1514 recursing into subdirectories to find them. Only test files that match
Michael Foorde91ea562009-09-13 19:07:03 +00001515 *pattern* will be loaded. (Using shell style pattern matching.) Only
1516 module names that are importable (i.e. are valid Python identifiers) will
1517 be loaded.
Michael Foordb4a81c82009-05-29 20:33:46 +00001518
1519 All test modules must be importable from the top level of the project. If
1520 the start directory is not the top level directory then the top level
1521 directory must be specified separately.
1522
Michael Foorde91ea562009-09-13 19:07:03 +00001523 If importing a module fails, for example due to a syntax error, then this
1524 will be recorded as a single error and discovery will continue.
1525
Michael Foordb4a81c82009-05-29 20:33:46 +00001526 If a test package name (directory with :file:`__init__.py`) matches the
1527 pattern then the package will be checked for a ``load_tests``
1528 function. If this exists then it will be called with *loader*, *tests*,
1529 *pattern*.
1530
Michael Foorddc0460a2009-09-13 19:08:18 +00001531 If load_tests exists then discovery does *not* recurse into the package,
Michael Foordb4a81c82009-05-29 20:33:46 +00001532 ``load_tests`` is responsible for loading all tests in the package.
1533
1534 The pattern is deliberately not stored as a loader attribute so that
1535 packages can continue discovery themselves. *top_level_dir* is stored so
1536 ``load_tests`` does not need to pass this argument in to
1537 ``loader.discover()``.
1538
Michael Foordba097ec2010-04-03 17:03:11 +00001539 *start_dir* can be a dotted module name as well as a directory.
1540
Michael Foord17565e52009-09-27 20:08:23 +00001541 .. versionadded:: 2.7
Michael Foordb4a81c82009-05-29 20:33:46 +00001542
Benjamin Peterson99721e02009-03-23 23:10:14 +00001543 The following attributes of a :class:`TestLoader` can be configured either by
1544 subclassing or assignment on an instance:
1545
1546
1547 .. attribute:: testMethodPrefix
1548
1549 String giving the prefix of method names which will be interpreted as test
1550 methods. The default value is ``'test'``.
1551
1552 This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
1553 methods.
1554
1555
1556 .. attribute:: sortTestMethodsUsing
1557
1558 Function to be used to compare method names when sorting them in
1559 :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
1560 default value is the built-in :func:`cmp` function; the attribute can also
1561 be set to :const:`None` to disable the sort.
1562
1563
1564 .. attribute:: suiteClass
1565
1566 Callable object that constructs a test suite from a list of tests. No
1567 methods on the resulting object are needed. The default value is the
1568 :class:`TestSuite` class.
1569
1570 This affects all the :meth:`loadTestsFrom\*` methods.
1571
1572
Benjamin Peterson99721e02009-03-23 23:10:14 +00001573.. class:: TestResult
1574
1575 This class is used to compile information about which tests have succeeded
1576 and which have failed.
1577
1578 A :class:`TestResult` object stores the results of a set of tests. The
1579 :class:`TestCase` and :class:`TestSuite` classes ensure that results are
1580 properly recorded; test authors do not need to worry about recording the
1581 outcome of tests.
1582
1583 Testing frameworks built on top of :mod:`unittest` may want access to the
1584 :class:`TestResult` object generated by running a set of tests for reporting
1585 purposes; a :class:`TestResult` instance is returned by the
1586 :meth:`TestRunner.run` method for this purpose.
1587
1588 :class:`TestResult` instances have the following attributes that will be of
1589 interest when inspecting the results of running a set of tests:
1590
1591
1592 .. attribute:: errors
1593
1594 A list containing 2-tuples of :class:`TestCase` instances and strings
1595 holding formatted tracebacks. Each tuple represents a test which raised an
1596 unexpected exception.
1597
1598 .. versionchanged:: 2.2
Benjamin Peterson99721e02009-03-23 23:10:14 +00001599 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1600
1601
1602 .. attribute:: failures
1603
1604 A list containing 2-tuples of :class:`TestCase` instances and strings
1605 holding formatted tracebacks. Each tuple represents a test where a failure
Ezio Melotti26765832013-09-07 15:19:30 +03001606 was explicitly signalled using the :meth:`TestCase.assert\*` methods.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001607
1608 .. versionchanged:: 2.2
Benjamin Peterson99721e02009-03-23 23:10:14 +00001609 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1610
1611 .. attribute:: skipped
1612
1613 A list containing 2-tuples of :class:`TestCase` instances and strings
1614 holding the reason for skipping the test.
1615
1616 .. versionadded:: 2.7
1617
1618 .. attribute:: expectedFailures
1619
Georg Brandl09302282010-10-06 09:32:48 +00001620 A list containing 2-tuples of :class:`TestCase` instances and strings
1621 holding formatted tracebacks. Each tuple represents an expected failure
Benjamin Peterson99721e02009-03-23 23:10:14 +00001622 of the test case.
1623
1624 .. attribute:: unexpectedSuccesses
1625
1626 A list containing :class:`TestCase` instances that were marked as expected
1627 failures, but succeeded.
1628
1629 .. attribute:: shouldStop
1630
1631 Set to ``True`` when the execution of tests should stop by :meth:`stop`.
1632
1633
1634 .. attribute:: testsRun
1635
1636 The total number of tests run so far.
1637
1638
Michael Foordba097ec2010-04-03 17:03:11 +00001639 .. attribute:: buffer
1640
1641 If set to true, ``sys.stdout`` and ``sys.stderr`` will be buffered in between
1642 :meth:`startTest` and :meth:`stopTest` being called. Collected output will
1643 only be echoed onto the real ``sys.stdout`` and ``sys.stderr`` if the test
1644 fails or errors. Any output is also attached to the failure / error message.
1645
1646 .. versionadded:: 2.7
1647
1648
1649 .. attribute:: failfast
1650
1651 If set to true :meth:`stop` will be called on the first failure or error,
1652 halting the test run.
1653
1654 .. versionadded:: 2.7
1655
1656
Benjamin Peterson99721e02009-03-23 23:10:14 +00001657 .. method:: wasSuccessful()
1658
Ezio Melottidd7c5932011-03-10 23:00:48 +02001659 Return ``True`` if all tests run so far have passed, otherwise returns
1660 ``False``.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001661
1662
1663 .. method:: stop()
1664
1665 This method can be called to signal that the set of tests being run should
Ezio Melottidd7c5932011-03-10 23:00:48 +02001666 be aborted by setting the :attr:`shouldStop` attribute to ``True``.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001667 :class:`TestRunner` objects should respect this flag and return without
1668 running any additional tests.
1669
1670 For example, this feature is used by the :class:`TextTestRunner` class to
1671 stop the test framework when the user signals an interrupt from the
1672 keyboard. Interactive tools which provide :class:`TestRunner`
1673 implementations can use this in a similar manner.
1674
1675 The following methods of the :class:`TestResult` class are used to maintain
1676 the internal data structures, and may be extended in subclasses to support
1677 additional reporting requirements. This is particularly useful in building
1678 tools which support interactive reporting while tests are being run.
1679
1680
1681 .. method:: startTest(test)
1682
1683 Called when the test case *test* is about to be run.
1684
Benjamin Peterson99721e02009-03-23 23:10:14 +00001685 .. method:: stopTest(test)
1686
1687 Called after the test case *test* has been executed, regardless of the
1688 outcome.
1689
Michael Foord07ef4872009-05-02 22:43:34 +00001690 .. method:: startTestRun(test)
1691
1692 Called once before any tests are executed.
1693
1694 .. versionadded:: 2.7
1695
1696
1697 .. method:: stopTestRun(test)
1698
Ezio Melotti7b4e02c2010-01-27 20:25:11 +00001699 Called once after all tests are executed.
Michael Foord07ef4872009-05-02 22:43:34 +00001700
1701 .. versionadded:: 2.7
1702
1703
Benjamin Peterson99721e02009-03-23 23:10:14 +00001704 .. method:: addError(test, err)
1705
Ezio Melottiad543072013-09-07 15:23:36 +03001706 Called when the test case *test* raises an unexpected exception. *err* is a
Benjamin Peterson99721e02009-03-23 23:10:14 +00001707 tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
1708 traceback)``.
1709
1710 The default implementation appends a tuple ``(test, formatted_err)`` to
1711 the instance's :attr:`errors` attribute, where *formatted_err* is a
1712 formatted traceback derived from *err*.
1713
1714
1715 .. method:: addFailure(test, err)
1716
Michael Foordb4a81c82009-05-29 20:33:46 +00001717 Called when the test case *test* signals a failure. *err* is a tuple of
1718 the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
Benjamin Peterson99721e02009-03-23 23:10:14 +00001719
1720 The default implementation appends a tuple ``(test, formatted_err)`` to
1721 the instance's :attr:`failures` attribute, where *formatted_err* is a
1722 formatted traceback derived from *err*.
1723
1724
1725 .. method:: addSuccess(test)
1726
1727 Called when the test case *test* succeeds.
1728
1729 The default implementation does nothing.
1730
1731
1732 .. method:: addSkip(test, reason)
1733
1734 Called when the test case *test* is skipped. *reason* is the reason the
1735 test gave for skipping.
1736
1737 The default implementation appends a tuple ``(test, reason)`` to the
1738 instance's :attr:`skipped` attribute.
1739
1740
1741 .. method:: addExpectedFailure(test, err)
1742
1743 Called when the test case *test* fails, but was marked with the
1744 :func:`expectedFailure` decorator.
1745
1746 The default implementation appends a tuple ``(test, formatted_err)`` to
1747 the instance's :attr:`expectedFailures` attribute, where *formatted_err*
1748 is a formatted traceback derived from *err*.
1749
1750
1751 .. method:: addUnexpectedSuccess(test)
1752
1753 Called when the test case *test* was marked with the
1754 :func:`expectedFailure` decorator, but succeeded.
1755
1756 The default implementation appends the test to the instance's
1757 :attr:`unexpectedSuccesses` attribute.
Georg Brandl8ec7f652007-08-15 14:28:01 +00001758
Michael Foorddb43b5a2010-02-10 14:25:12 +00001759.. class:: TextTestResult(stream, descriptions, verbosity)
1760
1761 A concrete implementation of :class:`TestResult` used by the
1762 :class:`TextTestRunner`.
1763
1764 .. versionadded:: 2.7
1765 This class was previously named ``_TextTestResult``. The old name still
1766 exists as an alias but is deprecated.
Georg Brandl8ec7f652007-08-15 14:28:01 +00001767
1768.. data:: defaultTestLoader
1769
1770 Instance of the :class:`TestLoader` class intended to be shared. If no
1771 customization of the :class:`TestLoader` is needed, this instance can be used
1772 instead of repeatedly creating new instances.
1773
1774
Ezio Melottidd7c5932011-03-10 23:00:48 +02001775.. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1)
Georg Brandl8ec7f652007-08-15 14:28:01 +00001776
1777 A basic test runner implementation which prints results on standard error. It
1778 has a few configurable parameters, but is essentially very simple. Graphical
1779 applications which run test suites should provide alternate implementations.
1780
Georg Brandl9bc66822009-04-27 17:04:23 +00001781 .. method:: _makeResult()
1782
1783 This method returns the instance of ``TestResult`` used by :meth:`run`.
1784 It is not intended to be called directly, but can be overridden in
1785 subclasses to provide a custom ``TestResult``.
1786
Michael Foorddb43b5a2010-02-10 14:25:12 +00001787 ``_makeResult()`` instantiates the class or callable passed in the
1788 ``TextTestRunner`` constructor as the ``resultclass`` argument. It
Michael Foordefc2f492010-04-08 04:33:20 +00001789 defaults to :class:`TextTestResult` if no ``resultclass`` is provided.
Michael Foorddb43b5a2010-02-10 14:25:12 +00001790 The result class is instantiated with the following arguments::
1791
1792 stream, descriptions, verbosity
1793
Georg Brandl8ec7f652007-08-15 14:28:01 +00001794
Ezio Melotti30abb3a2012-04-30 19:05:57 +03001795.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader[, exit[, verbosity[, failfast[, catchbreak[, buffer]]]]]]]]]])
Michael Foord55430352010-04-05 00:39:50 +00001796
Ezio Melotti30abb3a2012-04-30 19:05:57 +03001797 A command-line program that loads a set of tests from *module* and runs them;
1798 this is primarily for making test modules conveniently executable.
1799 The simplest use for this function is to include the following line at the
1800 end of a test script::
Georg Brandl8ec7f652007-08-15 14:28:01 +00001801
1802 if __name__ == '__main__':
1803 unittest.main()
1804
Michael Foord5d31e052009-05-11 17:59:43 +00001805 You can run tests with more detailed information by passing in the verbosity
1806 argument::
1807
1808 if __name__ == '__main__':
1809 unittest.main(verbosity=2)
1810
Ezio Melotti30abb3a2012-04-30 19:05:57 +03001811 The *argv* argument can be a list of options passed to the program, with the
1812 first element being the program name. If not specified or ``None``,
1813 the values of :data:`sys.argv` are used.
1814
Georg Brandl8ec7f652007-08-15 14:28:01 +00001815 The *testRunner* argument can either be a test runner class or an already
Michael Foord829f6b82009-05-02 11:43:06 +00001816 created instance of it. By default ``main`` calls :func:`sys.exit` with
1817 an exit code indicating success or failure of the tests run.
1818
Ezio Melotti30abb3a2012-04-30 19:05:57 +03001819 The *testLoader* argument has to be a :class:`TestLoader` instance,
1820 and defaults to :data:`defaultTestLoader`.
1821
Michael Foord829f6b82009-05-02 11:43:06 +00001822 ``main`` supports being used from the interactive interpreter by passing in the
1823 argument ``exit=False``. This displays the result on standard output without
1824 calling :func:`sys.exit`::
1825
1826 >>> from unittest import main
1827 >>> main(module='test_module', exit=False)
1828
Ezio Melotti30abb3a2012-04-30 19:05:57 +03001829 The *failfast*, *catchbreak* and *buffer* parameters have the same
Éric Araujoa8132ec2010-12-16 03:53:53 +00001830 effect as the same-name `command-line options`_.
Michael Foordddb20df2010-04-04 23:28:44 +00001831
Michael Foord829f6b82009-05-02 11:43:06 +00001832 Calling ``main`` actually returns an instance of the ``TestProgram`` class.
1833 This stores the result of the tests run as the ``result`` attribute.
1834
1835 .. versionchanged:: 2.7
Ezio Melotti30abb3a2012-04-30 19:05:57 +03001836 The *exit*, *verbosity*, *failfast*, *catchbreak* and *buffer*
Michael Foordddb20df2010-04-04 23:28:44 +00001837 parameters were added.
Michael Foordb4a81c82009-05-29 20:33:46 +00001838
1839
1840load_tests Protocol
1841###################
1842
Michael Foord17565e52009-09-27 20:08:23 +00001843.. versionadded:: 2.7
1844
Michael Foordb4a81c82009-05-29 20:33:46 +00001845Modules or packages can customize how tests are loaded from them during normal
1846test runs or test discovery by implementing a function called ``load_tests``.
1847
1848If a test module defines ``load_tests`` it will be called by
1849:meth:`TestLoader.loadTestsFromModule` with the following arguments::
1850
1851 load_tests(loader, standard_tests, None)
1852
1853It should return a :class:`TestSuite`.
1854
1855*loader* is the instance of :class:`TestLoader` doing the loading.
1856*standard_tests* are the tests that would be loaded by default from the
1857module. It is common for test modules to only want to add or remove tests
1858from the standard set of tests.
1859The third argument is used when loading packages as part of test discovery.
1860
1861A typical ``load_tests`` function that loads tests from a specific set of
1862:class:`TestCase` classes may look like::
1863
1864 test_cases = (TestCase1, TestCase2, TestCase3)
1865
1866 def load_tests(loader, tests, pattern):
1867 suite = TestSuite()
1868 for test_class in test_cases:
1869 tests = loader.loadTestsFromTestCase(test_class)
1870 suite.addTests(tests)
1871 return suite
1872
1873If discovery is started, either from the command line or by calling
1874:meth:`TestLoader.discover`, with a pattern that matches a package
1875name then the package :file:`__init__.py` will be checked for ``load_tests``.
1876
1877.. note::
1878
Ezio Melotti79b2dba2013-02-28 08:28:11 +02001879 The default pattern is ``'test*.py'``. This matches all Python files
1880 that start with ``'test'`` but *won't* match any test directories.
Michael Foordb4a81c82009-05-29 20:33:46 +00001881
Ezio Melotti79b2dba2013-02-28 08:28:11 +02001882 A pattern like ``'test*'`` will match test packages as well as
Michael Foordb4a81c82009-05-29 20:33:46 +00001883 modules.
1884
1885If the package :file:`__init__.py` defines ``load_tests`` then it will be
1886called and discovery not continued into the package. ``load_tests``
1887is called with the following arguments::
1888
1889 load_tests(loader, standard_tests, pattern)
1890
1891This should return a :class:`TestSuite` representing all the tests
1892from the package. (``standard_tests`` will only contain tests
1893collected from :file:`__init__.py`.)
1894
1895Because the pattern is passed into ``load_tests`` the package is free to
1896continue (and potentially modify) test discovery. A 'do nothing'
1897``load_tests`` function for a test package would look like::
1898
1899 def load_tests(loader, standard_tests, pattern):
1900 # top level directory cached on loader instance
1901 this_dir = os.path.dirname(__file__)
1902 package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
1903 standard_tests.addTests(package_tests)
1904 return standard_tests
Michael Foordba097ec2010-04-03 17:03:11 +00001905
1906
1907Class and Module Fixtures
1908-------------------------
1909
Michael Foord09e29802010-04-04 22:41:54 +00001910Class and module level fixtures are implemented in :class:`TestSuite`. When
1911the test suite encounters a test from a new class then :meth:`tearDownClass`
1912from the previous class (if there is one) is called, followed by
1913:meth:`setUpClass` from the new class.
Michael Foordba097ec2010-04-03 17:03:11 +00001914
Michael Foord09e29802010-04-04 22:41:54 +00001915Similarly if a test is from a different module from the previous test then
1916``tearDownModule`` from the previous module is run, followed by
1917``setUpModule`` from the new module.
Michael Foordba097ec2010-04-03 17:03:11 +00001918
Michael Foord09e29802010-04-04 22:41:54 +00001919After all the tests have run the final ``tearDownClass`` and
1920``tearDownModule`` are run.
Michael Foordba097ec2010-04-03 17:03:11 +00001921
Michael Foord09e29802010-04-04 22:41:54 +00001922Note that shared fixtures do not play well with [potential] features like test
1923parallelization and they break test isolation. They should be used with care.
Michael Foordba097ec2010-04-03 17:03:11 +00001924
Michael Foord09e29802010-04-04 22:41:54 +00001925The default ordering of tests created by the unittest test loaders is to group
1926all tests from the same modules and classes together. This will lead to
1927``setUpClass`` / ``setUpModule`` (etc) being called exactly once per class and
1928module. If you randomize the order, so that tests from different modules and
1929classes are adjacent to each other, then these shared fixture functions may be
1930called multiple times in a single test run.
Michael Foordba097ec2010-04-03 17:03:11 +00001931
Michael Foord09e29802010-04-04 22:41:54 +00001932Shared fixtures are not intended to work with suites with non-standard
1933ordering. A ``BaseTestSuite`` still exists for frameworks that don't want to
1934support shared fixtures.
Michael Foordba097ec2010-04-03 17:03:11 +00001935
Michael Foord09e29802010-04-04 22:41:54 +00001936If there are any exceptions raised during one of the shared fixture functions
1937the test is reported as an error. Because there is no corresponding test
1938instance an ``_ErrorHolder`` object (that has the same interface as a
1939:class:`TestCase`) is created to represent the error. If you are just using
1940the standard unittest test runner then this detail doesn't matter, but if you
1941are a framework author it may be relevant.
Michael Foordba097ec2010-04-03 17:03:11 +00001942
1943
1944setUpClass and tearDownClass
1945~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1946
1947These must be implemented as class methods::
1948
1949 import unittest
1950
1951 class Test(unittest.TestCase):
1952 @classmethod
1953 def setUpClass(cls):
1954 cls._connection = createExpensiveConnectionObject()
1955
1956 @classmethod
1957 def tearDownClass(cls):
1958 cls._connection.destroy()
1959
Michael Foord09e29802010-04-04 22:41:54 +00001960If you want the ``setUpClass`` and ``tearDownClass`` on base classes called
1961then you must call up to them yourself. The implementations in
1962:class:`TestCase` are empty.
Michael Foordba097ec2010-04-03 17:03:11 +00001963
Michael Foord09e29802010-04-04 22:41:54 +00001964If an exception is raised during a ``setUpClass`` then the tests in the class
1965are not run and the ``tearDownClass`` is not run. Skipped classes will not
Michael Foord8dde2012010-06-05 21:57:03 +00001966have ``setUpClass`` or ``tearDownClass`` run. If the exception is a
Ezio Melotti352def02013-03-27 20:11:55 +02001967:exc:`SkipTest` exception then the class will be reported as having been skipped
Michael Foord8dde2012010-06-05 21:57:03 +00001968instead of as an error.
Michael Foordba097ec2010-04-03 17:03:11 +00001969
1970
1971setUpModule and tearDownModule
1972~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1973
1974These should be implemented as functions::
1975
1976 def setUpModule():
1977 createConnection()
1978
1979 def tearDownModule():
1980 closeConnection()
1981
Michael Foord09e29802010-04-04 22:41:54 +00001982If an exception is raised in a ``setUpModule`` then none of the tests in the
Michael Foord8dde2012010-06-05 21:57:03 +00001983module will be run and the ``tearDownModule`` will not be run. If the exception is a
Ezio Melotti352def02013-03-27 20:11:55 +02001984:exc:`SkipTest` exception then the module will be reported as having been skipped
Michael Foord8dde2012010-06-05 21:57:03 +00001985instead of as an error.
Michael Foordba097ec2010-04-03 17:03:11 +00001986
1987
Michael Foord55430352010-04-05 00:39:50 +00001988Signal Handling
1989---------------
1990
Éric Araujoa8132ec2010-12-16 03:53:53 +00001991The :option:`-c/--catch <unittest -c>` command-line option to unittest,
1992along with the ``catchbreak`` parameter to :func:`unittest.main()`, provide
1993more friendly handling of control-C during a test run. With catch break
1994behavior enabled control-C will allow the currently running test to complete,
1995and the test run will then end and report all the results so far. A second
1996control-c will raise a :exc:`KeyboardInterrupt` in the usual way.
Michael Foord55430352010-04-05 00:39:50 +00001997
Michael Foord5c322ec2010-04-25 19:02:46 +00001998The control-c handling signal handler attempts to remain compatible with code or
1999tests that install their own :const:`signal.SIGINT` handler. If the ``unittest``
2000handler is called but *isn't* the installed :const:`signal.SIGINT` handler,
2001i.e. it has been replaced by the system under test and delegated to, then it
2002calls the default handler. This will normally be the expected behavior by code
2003that replaces an installed handler and delegates to it. For individual tests
2004that need ``unittest`` control-c handling disabled the :func:`removeHandler`
2005decorator can be used.
2006
2007There are a few utility functions for framework authors to enable control-c
2008handling functionality within test frameworks.
Michael Foord55430352010-04-05 00:39:50 +00002009
2010.. function:: installHandler()
2011
Michael Foord31655032010-04-05 10:26:26 +00002012 Install the control-c handler. When a :const:`signal.SIGINT` is received
2013 (usually in response to the user pressing control-c) all registered results
Michael Foord55430352010-04-05 00:39:50 +00002014 have :meth:`~TestResult.stop` called.
2015
Michael Foord47b54402010-04-26 23:36:47 +00002016 .. versionadded:: 2.7
2017
Michael Foord55430352010-04-05 00:39:50 +00002018.. function:: registerResult(result)
2019
Michael Foord31655032010-04-05 10:26:26 +00002020 Register a :class:`TestResult` object for control-c handling. Registering a
Michael Foord55430352010-04-05 00:39:50 +00002021 result stores a weak reference to it, so it doesn't prevent the result from
2022 being garbage collected.
2023
Michael Foord5c322ec2010-04-25 19:02:46 +00002024 Registering a :class:`TestResult` object has no side-effects if control-c
2025 handling is not enabled, so test frameworks can unconditionally register
2026 all results they create independently of whether or not handling is enabled.
2027
Michael Foord47b54402010-04-26 23:36:47 +00002028 .. versionadded:: 2.7
2029
Michael Foord55430352010-04-05 00:39:50 +00002030.. function:: removeResult(result)
2031
Michael Foord31655032010-04-05 10:26:26 +00002032 Remove a registered result. Once a result has been removed then
Michael Foordd341ec82010-04-05 10:30:14 +00002033 :meth:`~TestResult.stop` will no longer be called on that result object in
Michael Foord31655032010-04-05 10:26:26 +00002034 response to a control-c.
Michael Foord55430352010-04-05 00:39:50 +00002035
Michael Foord47b54402010-04-26 23:36:47 +00002036 .. versionadded:: 2.7
2037
Michael Foord5c322ec2010-04-25 19:02:46 +00002038.. function:: removeHandler(function=None)
2039
2040 When called without arguments this function removes the control-c handler
2041 if it has been installed. This function can also be used as a test decorator
2042 to temporarily remove the handler whilst the test is being executed::
2043
2044 @unittest.removeHandler
2045 def test_signal_handling(self):
2046 ...
2047
Michael Foord47b54402010-04-26 23:36:47 +00002048 .. versionadded:: 2.7
2049