Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`unittest` --- Unit testing framework |
| 2 | ========================================== |
| 3 | |
| 4 | .. module:: unittest |
| 5 | :synopsis: Unit testing framework for Python. |
Terry Jan Reedy | fa089b9 | 2016-06-11 15:02:54 -0400 | [diff] [blame] | 6 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 7 | .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com> |
| 8 | .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com> |
| 9 | .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 10 | .. sectionauthor:: Raymond Hettinger <python@rcn.com> |
| 11 | |
Terry Jan Reedy | fa089b9 | 2016-06-11 15:02:54 -0400 | [diff] [blame] | 12 | **Source code:** :source:`Lib/unittest/__init__.py` |
| 13 | |
| 14 | -------------- |
| 15 | |
Ezio Melotti | 22170ed | 2010-11-20 09:57:27 +0000 | [diff] [blame] | 16 | (If you are already familiar with the basic concepts of testing, you might want |
| 17 | to skip to :ref:`the list of assert methods <assert-methods>`.) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 18 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 19 | The :mod:`unittest` unit testing framework was originally inspired by JUnit |
| 20 | and has a similar flavor as major unit testing frameworks in other |
| 21 | languages. It supports test automation, sharing of setup and shutdown code |
| 22 | for tests, aggregation of tests into collections, and independence of the |
| 23 | tests from the reporting framework. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 24 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 25 | To achieve this, :mod:`unittest` supports some important concepts in an |
| 26 | object-oriented way: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 27 | |
| 28 | test fixture |
| 29 | A :dfn:`test fixture` represents the preparation needed to perform one or more |
| 30 | tests, and any associate cleanup actions. This may involve, for example, |
| 31 | creating temporary or proxy databases, directories, or starting a server |
| 32 | process. |
| 33 | |
| 34 | test case |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 35 | A :dfn:`test case` is the individual unit of testing. It checks for a specific |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 36 | response to a particular set of inputs. :mod:`unittest` provides a base class, |
| 37 | :class:`TestCase`, which may be used to create new test cases. |
| 38 | |
| 39 | test suite |
| 40 | A :dfn:`test suite` is a collection of test cases, test suites, or both. It is |
| 41 | used to aggregate tests that should be executed together. |
| 42 | |
| 43 | test runner |
| 44 | A :dfn:`test runner` is a component which orchestrates the execution of tests |
| 45 | and provides the outcome to the user. The runner may use a graphical interface, |
| 46 | a textual interface, or return a special value to indicate the results of |
| 47 | executing the tests. |
| 48 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 49 | |
| 50 | .. seealso:: |
| 51 | |
| 52 | Module :mod:`doctest` |
| 53 | Another test-support module with a very different flavor. |
| 54 | |
R David Murray | a1005ed | 2015-07-04 15:44:14 -0400 | [diff] [blame] | 55 | `Simple Smalltalk Testing: With Patterns <https://web.archive.org/web/20150315073817/http://www.xprogramming.com/testfram.htm>`_ |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 56 | Kent Beck's original paper on testing frameworks using the pattern shared |
| 57 | by :mod:`unittest`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 58 | |
Sanyam Khurana | 1b4587a | 2017-12-06 22:09:33 +0530 | [diff] [blame] | 59 | `Nose <https://nose.readthedocs.io/>`_ and `py.test <https://docs.pytest.org/>`_ |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 60 | Third-party unittest frameworks with a lighter-weight syntax for writing |
| 61 | tests. For example, ``assert func(10) == 42``. |
Raymond Hettinger | 6b232cd | 2009-03-24 00:22:53 +0000 | [diff] [blame] | 62 | |
Georg Brandl | e73778c | 2014-10-29 08:36:35 +0100 | [diff] [blame] | 63 | `The Python Testing Tools Taxonomy <https://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_ |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 64 | An extensive list of Python testing tools including functional testing |
| 65 | frameworks and mock object libraries. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 66 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 67 | `Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_ |
| 68 | A special-interest-group for discussion of testing, and testing tools, |
| 69 | in Python. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 70 | |
Michael Foord | 90efac7 | 2011-01-03 15:39:49 +0000 | [diff] [blame] | 71 | The script :file:`Tools/unittestgui/unittestgui.py` in the Python source distribution is |
| 72 | a GUI tool for test discovery and execution. This is intended largely for ease of use |
Senthil Kumaran | 847c33c | 2012-10-27 11:04:55 -0700 | [diff] [blame] | 73 | for those new to unit testing. For production environments it is |
| 74 | recommended that tests be driven by a continuous integration system such as |
Serhiy Storchaka | 6dff020 | 2016-05-07 10:49:07 +0300 | [diff] [blame] | 75 | `Buildbot <https://buildbot.net/>`_, `Jenkins <https://jenkins.io/>`_ |
Senthil Kumaran | 847c33c | 2012-10-27 11:04:55 -0700 | [diff] [blame] | 76 | or `Hudson <http://hudson-ci.org/>`_. |
Michael Foord | 90efac7 | 2011-01-03 15:39:49 +0000 | [diff] [blame] | 77 | |
| 78 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 79 | .. _unittest-minimal-example: |
| 80 | |
| 81 | Basic example |
| 82 | ------------- |
| 83 | |
| 84 | The :mod:`unittest` module provides a rich set of tools for constructing and |
| 85 | running tests. This section demonstrates that a small subset of the tools |
| 86 | suffice to meet the needs of most users. |
| 87 | |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 88 | Here is a short script to test three string methods:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 89 | |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 90 | import unittest |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 91 | |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 92 | class TestStringMethods(unittest.TestCase): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 93 | |
Serhiy Storchaka | dba9039 | 2016-05-10 12:01:23 +0300 | [diff] [blame] | 94 | def test_upper(self): |
| 95 | self.assertEqual('foo'.upper(), 'FOO') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 96 | |
Serhiy Storchaka | dba9039 | 2016-05-10 12:01:23 +0300 | [diff] [blame] | 97 | def test_isupper(self): |
| 98 | self.assertTrue('FOO'.isupper()) |
| 99 | self.assertFalse('Foo'.isupper()) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 100 | |
Serhiy Storchaka | dba9039 | 2016-05-10 12:01:23 +0300 | [diff] [blame] | 101 | def test_split(self): |
| 102 | s = 'hello world' |
| 103 | self.assertEqual(s.split(), ['hello', 'world']) |
| 104 | # check that s.split fails when the separator is not a string |
| 105 | with self.assertRaises(TypeError): |
| 106 | s.split(2) |
Benjamin Peterson | 847a411 | 2010-03-14 15:04:17 +0000 | [diff] [blame] | 107 | |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 108 | if __name__ == '__main__': |
| 109 | unittest.main() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 110 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 111 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 112 | A testcase is created by subclassing :class:`unittest.TestCase`. The three |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 113 | individual tests are defined with methods whose names start with the letters |
| 114 | ``test``. This naming convention informs the test runner about which methods |
| 115 | represent tests. |
| 116 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 117 | The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 118 | expected result; :meth:`~TestCase.assertTrue` or :meth:`~TestCase.assertFalse` |
| 119 | to verify a condition; or :meth:`~TestCase.assertRaises` to verify that a |
| 120 | specific exception gets raised. These methods are used instead of the |
| 121 | :keyword:`assert` statement so the test runner can accumulate all test results |
| 122 | and produce a report. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 123 | |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 124 | The :meth:`~TestCase.setUp` and :meth:`~TestCase.tearDown` methods allow you |
| 125 | to define instructions that will be executed before and after each test method. |
Benjamin Peterson | 8a6ddb9 | 2016-01-07 22:01:26 -0800 | [diff] [blame] | 126 | They are covered in more detail in the section :ref:`organizing-tests`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 127 | |
| 128 | The final block shows a simple way to run the tests. :func:`unittest.main` |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 129 | provides a command-line interface to the test script. When run from the command |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 130 | line, the above script produces an output that looks like this:: |
| 131 | |
| 132 | ... |
| 133 | ---------------------------------------------------------------------- |
| 134 | Ran 3 tests in 0.000s |
| 135 | |
| 136 | OK |
| 137 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 138 | Passing the ``-v`` option to your test script will instruct :func:`unittest.main` |
| 139 | to enable a higher level of verbosity, and produce the following output:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 140 | |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 141 | test_isupper (__main__.TestStringMethods) ... ok |
| 142 | test_split (__main__.TestStringMethods) ... ok |
| 143 | test_upper (__main__.TestStringMethods) ... ok |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 144 | |
| 145 | ---------------------------------------------------------------------- |
Ezio Melotti | 2e3998f | 2015-03-24 12:42:41 +0200 | [diff] [blame] | 146 | Ran 3 tests in 0.001s |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 147 | |
| 148 | OK |
| 149 | |
| 150 | The above examples show the most commonly used :mod:`unittest` features which |
| 151 | are sufficient to meet many everyday testing needs. The remainder of the |
| 152 | documentation explores the full feature set from first principles. |
| 153 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 154 | |
| 155 | .. _unittest-command-line-interface: |
| 156 | |
Éric Araujo | 76338ec | 2010-11-26 23:46:18 +0000 | [diff] [blame] | 157 | Command-Line Interface |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 158 | ---------------------- |
| 159 | |
| 160 | The unittest module can be used from the command line to run tests from |
| 161 | modules, classes or even individual test methods:: |
| 162 | |
| 163 | python -m unittest test_module1 test_module2 |
| 164 | python -m unittest test_module.TestClass |
| 165 | python -m unittest test_module.TestClass.test_method |
| 166 | |
| 167 | You can pass in a list with any combination of module names, and fully |
| 168 | qualified class or method names. |
| 169 | |
Michael Foord | 37d120a | 2010-12-04 01:11:21 +0000 | [diff] [blame] | 170 | Test modules can be specified by file path as well:: |
| 171 | |
| 172 | python -m unittest tests/test_something.py |
| 173 | |
| 174 | This allows you to use the shell filename completion to specify the test module. |
| 175 | The file specified must still be importable as a module. The path is converted |
| 176 | to a module name by removing the '.py' and converting path separators into '.'. |
| 177 | If you want to execute a test file that isn't importable as a module you should |
| 178 | execute the file directly instead. |
| 179 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 180 | You can run tests with more detail (higher verbosity) by passing in the -v flag:: |
| 181 | |
| 182 | python -m unittest -v test_module |
| 183 | |
Michael Foord | 086f308 | 2010-11-21 21:28:01 +0000 | [diff] [blame] | 184 | When executed without arguments :ref:`unittest-test-discovery` is started:: |
| 185 | |
| 186 | python -m unittest |
| 187 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 188 | For a list of all the command-line options:: |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 189 | |
| 190 | python -m unittest -h |
| 191 | |
Georg Brandl | 67b21b7 | 2010-08-17 15:07:14 +0000 | [diff] [blame] | 192 | .. versionchanged:: 3.2 |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 193 | In earlier versions it was only possible to run individual test methods and |
| 194 | not modules or classes. |
| 195 | |
| 196 | |
Éric Araujo | 76338ec | 2010-11-26 23:46:18 +0000 | [diff] [blame] | 197 | Command-line options |
| 198 | ~~~~~~~~~~~~~~~~~~~~ |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 199 | |
Éric Araujo | d3309df | 2010-11-21 03:09:17 +0000 | [diff] [blame] | 200 | :program:`unittest` supports these command-line options: |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 201 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 202 | .. program:: unittest |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 203 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 204 | .. cmdoption:: -b, --buffer |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 205 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 206 | The standard output and standard error streams are buffered during the test |
| 207 | run. Output during a passing test is discarded. Output is echoed normally |
| 208 | on test fail or error and is added to the failure messages. |
Benjamin Peterson | d7c3ed5 | 2010-06-27 22:32:30 +0000 | [diff] [blame] | 209 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 210 | .. cmdoption:: -c, --catch |
Benjamin Peterson | d7c3ed5 | 2010-06-27 22:32:30 +0000 | [diff] [blame] | 211 | |
Serhiy Storchaka | 0424eaf | 2015-09-12 17:45:25 +0300 | [diff] [blame] | 212 | :kbd:`Control-C` during the test run waits for the current test to end and then |
| 213 | reports all the results so far. A second :kbd:`Control-C` raises the normal |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 214 | :exc:`KeyboardInterrupt` exception. |
Benjamin Peterson | d7c3ed5 | 2010-06-27 22:32:30 +0000 | [diff] [blame] | 215 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 216 | See `Signal Handling`_ for the functions that provide this functionality. |
Benjamin Peterson | d7c3ed5 | 2010-06-27 22:32:30 +0000 | [diff] [blame] | 217 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 218 | .. cmdoption:: -f, --failfast |
| 219 | |
| 220 | Stop the test run on the first error or failure. |
Benjamin Peterson | d7c3ed5 | 2010-06-27 22:32:30 +0000 | [diff] [blame] | 221 | |
Jonas Haag | 5b48dc6 | 2017-11-25 16:23:52 +0100 | [diff] [blame] | 222 | .. cmdoption:: -k |
| 223 | |
| 224 | Only run test methods and classes that match the pattern or substring. |
| 225 | This option may be used multiple times, in which case all test cases that |
| 226 | match of the given patterns are included. |
| 227 | |
| 228 | Patterns that contain a wildcard character (``*``) are matched against the |
| 229 | test name using :meth:`fnmatch.fnmatchcase`; otherwise simple case-sensitive |
| 230 | substring matching is used. |
| 231 | |
| 232 | Patterns are matched against the fully qualified test method name as |
| 233 | imported by the test loader. |
| 234 | |
| 235 | For example, ``-k foo`` matches ``foo_tests.SomeTest.test_something``, |
| 236 | ``bar_tests.SomeTest.test_foo``, but not ``bar_tests.FooTest.test_something``. |
| 237 | |
Robert Collins | f0c819a | 2015-03-06 13:46:35 +1300 | [diff] [blame] | 238 | .. cmdoption:: --locals |
| 239 | |
| 240 | Show local variables in tracebacks. |
| 241 | |
Ezio Melotti | 7afd3f5 | 2010-04-20 09:32:54 +0000 | [diff] [blame] | 242 | .. versionadded:: 3.2 |
Éric Araujo | d6c5f74 | 2010-12-16 00:07:01 +0000 | [diff] [blame] | 243 | The command-line options ``-b``, ``-c`` and ``-f`` were added. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 244 | |
Robert Collins | f0c819a | 2015-03-06 13:46:35 +1300 | [diff] [blame] | 245 | .. versionadded:: 3.5 |
| 246 | The command-line option ``--locals``. |
| 247 | |
Jonas Haag | 5b48dc6 | 2017-11-25 16:23:52 +0100 | [diff] [blame] | 248 | .. versionadded:: 3.7 |
| 249 | The command-line option ``-k``. |
| 250 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 251 | The command line can also be used for test discovery, for running all of the |
| 252 | tests in a project or just a subset. |
| 253 | |
| 254 | |
| 255 | .. _unittest-test-discovery: |
| 256 | |
| 257 | Test Discovery |
| 258 | -------------- |
| 259 | |
Ezio Melotti | 7afd3f5 | 2010-04-20 09:32:54 +0000 | [diff] [blame] | 260 | .. versionadded:: 3.2 |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 261 | |
Ezio Melotti | 3d99584 | 2011-03-08 16:17:35 +0200 | [diff] [blame] | 262 | Unittest supports simple test discovery. In order to be compatible with test |
| 263 | discovery, all of the test files must be :ref:`modules <tut-modules>` or |
Larry Hastings | 3732ed2 | 2014-03-15 21:13:56 -0700 | [diff] [blame] | 264 | :ref:`packages <tut-packages>` (including :term:`namespace packages |
| 265 | <namespace package>`) importable from the top-level directory of |
| 266 | the project (this means that their filenames must be valid :ref:`identifiers |
| 267 | <identifiers>`). |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 268 | |
| 269 | Test discovery is implemented in :meth:`TestLoader.discover`, but can also be |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 270 | used from the command line. The basic command-line usage is:: |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 271 | |
| 272 | cd project_directory |
| 273 | python -m unittest discover |
| 274 | |
Michael Foord | 086f308 | 2010-11-21 21:28:01 +0000 | [diff] [blame] | 275 | .. note:: |
| 276 | |
| 277 | As a shortcut, ``python -m unittest`` is the equivalent of |
| 278 | ``python -m unittest discover``. If you want to pass arguments to test |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 279 | discovery the ``discover`` sub-command must be used explicitly. |
Michael Foord | 086f308 | 2010-11-21 21:28:01 +0000 | [diff] [blame] | 280 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 281 | The ``discover`` sub-command has the following options: |
| 282 | |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 283 | .. program:: unittest discover |
| 284 | |
| 285 | .. cmdoption:: -v, --verbose |
| 286 | |
| 287 | Verbose output |
| 288 | |
Chris Jerdonek | d69ad55 | 2013-02-21 18:54:43 -0800 | [diff] [blame] | 289 | .. cmdoption:: -s, --start-directory directory |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 290 | |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 291 | Directory to start discovery (``.`` default) |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 292 | |
Chris Jerdonek | d69ad55 | 2013-02-21 18:54:43 -0800 | [diff] [blame] | 293 | .. cmdoption:: -p, --pattern pattern |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 294 | |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 295 | Pattern to match test files (``test*.py`` default) |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 296 | |
Chris Jerdonek | d69ad55 | 2013-02-21 18:54:43 -0800 | [diff] [blame] | 297 | .. cmdoption:: -t, --top-level-directory directory |
Éric Araujo | 713d303 | 2010-11-18 16:38:46 +0000 | [diff] [blame] | 298 | |
| 299 | Top level directory of project (defaults to start directory) |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 300 | |
Benjamin Peterson | d7c3ed5 | 2010-06-27 22:32:30 +0000 | [diff] [blame] | 301 | The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in |
| 302 | as positional arguments in that order. The following two command lines |
| 303 | are equivalent:: |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 304 | |
Robert Collins | a2b0055 | 2015-08-24 12:14:28 +1200 | [diff] [blame] | 305 | python -m unittest discover -s project_directory -p "*_test.py" |
| 306 | python -m unittest discover project_directory "*_test.py" |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 307 | |
Michael Foord | 16f3e90 | 2010-05-08 15:13:42 +0000 | [diff] [blame] | 308 | As well as being a path it is possible to pass a package name, for example |
| 309 | ``myproject.subpackage.test``, as the start directory. The package name you |
| 310 | supply will then be imported and its location on the filesystem will be used |
| 311 | as the start directory. |
| 312 | |
| 313 | .. caution:: |
| 314 | |
Senthil Kumaran | 916bd38 | 2010-10-15 12:55:19 +0000 | [diff] [blame] | 315 | Test discovery loads tests by importing them. Once test discovery has found |
| 316 | all the test files from the start directory you specify it turns the paths |
| 317 | into package names to import. For example :file:`foo/bar/baz.py` will be |
Michael Foord | 16f3e90 | 2010-05-08 15:13:42 +0000 | [diff] [blame] | 318 | imported as ``foo.bar.baz``. |
| 319 | |
| 320 | If you have a package installed globally and attempt test discovery on |
| 321 | a different copy of the package then the import *could* happen from the |
| 322 | wrong place. If this happens test discovery will warn you and exit. |
| 323 | |
| 324 | If you supply the start directory as a package name rather than a |
| 325 | path to a directory then discover assumes that whichever location it |
| 326 | imports from is the location you intended, so you will not get the |
| 327 | warning. |
| 328 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 329 | Test modules and packages can customize test loading and discovery by through |
| 330 | the `load_tests protocol`_. |
| 331 | |
Larry Hastings | 3732ed2 | 2014-03-15 21:13:56 -0700 | [diff] [blame] | 332 | .. versionchanged:: 3.4 |
| 333 | Test discovery supports :term:`namespace packages <namespace package>`. |
| 334 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 335 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 336 | .. _organizing-tests: |
| 337 | |
| 338 | Organizing test code |
| 339 | -------------------- |
| 340 | |
| 341 | The basic building blocks of unit testing are :dfn:`test cases` --- single |
| 342 | scenarios that must be set up and checked for correctness. In :mod:`unittest`, |
Raymond Hettinger | 833ad0e | 2011-02-06 21:00:38 +0000 | [diff] [blame] | 343 | test cases are represented by :class:`unittest.TestCase` instances. |
| 344 | To make your own test cases you must write subclasses of |
| 345 | :class:`TestCase` or use :class:`FunctionTestCase`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 346 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 347 | The testing code of a :class:`TestCase` instance should be entirely self |
| 348 | contained, such that it can be run either in isolation or in arbitrary |
| 349 | combination with any number of other test cases. |
| 350 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 351 | The simplest :class:`TestCase` subclass will simply implement a test method |
| 352 | (i.e. a method whose name starts with ``test``) in order to perform specific |
| 353 | testing code:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 354 | |
| 355 | import unittest |
| 356 | |
| 357 | class DefaultWidgetSizeTestCase(unittest.TestCase): |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 358 | def test_default_widget_size(self): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 359 | widget = Widget('The widget') |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 360 | self.assertEqual(widget.size(), (50, 50)) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 361 | |
Sandro Tosi | 41b2404 | 2012-01-21 10:59:37 +0100 | [diff] [blame] | 362 | Note that in order to test something, we use one of the :meth:`assert\*` |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 363 | methods provided by the :class:`TestCase` base class. If the test fails, an |
Gregory P. Smith | dff4675 | 2018-05-17 10:08:45 -0500 | [diff] [blame] | 364 | exception will be raised with an explanatory message, and :mod:`unittest` |
| 365 | will identify the test case as a :dfn:`failure`. Any other exceptions will be |
| 366 | treated as :dfn:`errors`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 367 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 368 | Tests can be numerous, and their set-up can be repetitive. Luckily, we |
| 369 | can factor out set-up code by implementing a method called |
| 370 | :meth:`~TestCase.setUp`, which the testing framework will automatically |
| 371 | call for every single test we run:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 372 | |
| 373 | import unittest |
| 374 | |
Berker Peksag | ab75e02 | 2016-08-06 03:00:03 +0300 | [diff] [blame] | 375 | class WidgetTestCase(unittest.TestCase): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 376 | def setUp(self): |
| 377 | self.widget = Widget('The widget') |
| 378 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 379 | def test_default_widget_size(self): |
Ezio Melotti | 2d6c39b | 2010-02-04 20:27:41 +0000 | [diff] [blame] | 380 | self.assertEqual(self.widget.size(), (50,50), |
| 381 | 'incorrect default size') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 382 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 383 | def test_widget_resize(self): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 384 | self.widget.resize(100,150) |
Ezio Melotti | 2d6c39b | 2010-02-04 20:27:41 +0000 | [diff] [blame] | 385 | self.assertEqual(self.widget.size(), (100,150), |
| 386 | 'wrong size after resize') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 387 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 388 | .. note:: |
| 389 | The order in which the various tests will be run is determined |
| 390 | by sorting the test method names with respect to the built-in |
| 391 | ordering for strings. |
| 392 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 393 | If the :meth:`~TestCase.setUp` method raises an exception while the test is |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 394 | running, the framework will consider the test to have suffered an error, and |
| 395 | the test method will not be executed. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 396 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 397 | Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 398 | after the test method has been run:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 399 | |
| 400 | import unittest |
| 401 | |
Berker Peksag | ab75e02 | 2016-08-06 03:00:03 +0300 | [diff] [blame] | 402 | class WidgetTestCase(unittest.TestCase): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 403 | def setUp(self): |
| 404 | self.widget = Widget('The widget') |
| 405 | |
| 406 | def tearDown(self): |
| 407 | self.widget.dispose() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 408 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 409 | If :meth:`~TestCase.setUp` succeeded, :meth:`~TestCase.tearDown` will be |
| 410 | run whether the test method succeeded or not. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 411 | |
Gregory P. Smith | dff4675 | 2018-05-17 10:08:45 -0500 | [diff] [blame] | 412 | Such a working environment for the testing code is called a |
| 413 | :dfn:`test fixture`. A new TestCase instance is created as a unique |
| 414 | test fixture used to execute each individual test method. Thus |
Berker Peksag | 2e5566d | 2018-08-04 00:56:55 +0300 | [diff] [blame] | 415 | :meth:`~TestCase.setUp`, :meth:`~TestCase.tearDown`, and :meth:`~TestCase.__init__` |
Gregory P. Smith | dff4675 | 2018-05-17 10:08:45 -0500 | [diff] [blame] | 416 | will be called once per test. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 417 | |
Gregory P. Smith | dff4675 | 2018-05-17 10:08:45 -0500 | [diff] [blame] | 418 | It is recommended that you use TestCase implementations to group tests together |
| 419 | according to the features they test. :mod:`unittest` provides a mechanism for |
| 420 | this: the :dfn:`test suite`, represented by :mod:`unittest`'s |
| 421 | :class:`TestSuite` class. In most cases, calling :func:`unittest.main` will do |
| 422 | the right thing and collect all the module's test cases for you and execute |
| 423 | them. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 424 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 425 | However, should you want to customize the building of your test suite, |
| 426 | you can do it yourself:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 427 | |
| 428 | def suite(): |
| 429 | suite = unittest.TestSuite() |
Berker Peksag | 9255104 | 2017-10-13 06:41:57 +0300 | [diff] [blame] | 430 | suite.addTest(WidgetTestCase('test_default_widget_size')) |
| 431 | suite.addTest(WidgetTestCase('test_widget_resize')) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 432 | return suite |
| 433 | |
Berker Peksag | 9255104 | 2017-10-13 06:41:57 +0300 | [diff] [blame] | 434 | if __name__ == '__main__': |
| 435 | runner = unittest.TextTestRunner() |
| 436 | runner.run(suite()) |
| 437 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 438 | You can place the definitions of test cases and test suites in the same modules |
| 439 | as the code they are to test (such as :file:`widget.py`), but there are several |
| 440 | advantages to placing the test code in a separate module, such as |
| 441 | :file:`test_widget.py`: |
| 442 | |
| 443 | * The test module can be run standalone from the command line. |
| 444 | |
| 445 | * The test code can more easily be separated from shipped code. |
| 446 | |
| 447 | * There is less temptation to change test code to fit the code it tests without |
| 448 | a good reason. |
| 449 | |
| 450 | * Test code should be modified much less frequently than the code it tests. |
| 451 | |
| 452 | * Tested code can be refactored more easily. |
| 453 | |
| 454 | * Tests for modules written in C must be in separate modules anyway, so why not |
| 455 | be consistent? |
| 456 | |
| 457 | * If the testing strategy changes, there is no need to change the source code. |
| 458 | |
| 459 | |
| 460 | .. _legacy-unit-tests: |
| 461 | |
| 462 | Re-using old test code |
| 463 | ---------------------- |
| 464 | |
| 465 | Some users will find that they have existing test code that they would like to |
| 466 | run from :mod:`unittest`, without converting every old test function to a |
| 467 | :class:`TestCase` subclass. |
| 468 | |
| 469 | For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class. |
| 470 | This subclass of :class:`TestCase` can be used to wrap an existing test |
| 471 | function. Set-up and tear-down functions can also be provided. |
| 472 | |
| 473 | Given the following test function:: |
| 474 | |
| 475 | def testSomething(): |
| 476 | something = makeSomething() |
| 477 | assert something.name is not None |
| 478 | # ... |
| 479 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 480 | one can create an equivalent test case instance as follows, with optional |
| 481 | set-up and tear-down methods:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 482 | |
| 483 | testcase = unittest.FunctionTestCase(testSomething, |
| 484 | setUp=makeSomethingDB, |
| 485 | tearDown=deleteSomethingDB) |
| 486 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 487 | .. note:: |
| 488 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 489 | Even though :class:`FunctionTestCase` can be used to quickly convert an |
| 490 | existing test base over to a :mod:`unittest`\ -based system, this approach is |
| 491 | not recommended. Taking the time to set up proper :class:`TestCase` |
| 492 | subclasses will make future test refactorings infinitely easier. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 493 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 494 | In some cases, the existing tests may have been written using the :mod:`doctest` |
| 495 | module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can |
| 496 | automatically build :class:`unittest.TestSuite` instances from the existing |
| 497 | :mod:`doctest`\ -based tests. |
| 498 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 499 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 500 | .. _unittest-skipping: |
| 501 | |
| 502 | Skipping tests and expected failures |
| 503 | ------------------------------------ |
| 504 | |
Michael Foord | f5c851a | 2010-02-05 21:48:03 +0000 | [diff] [blame] | 505 | .. versionadded:: 3.1 |
| 506 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 507 | Unittest supports skipping individual test methods and even whole classes of |
Serhiy Storchaka | d65c949 | 2015-11-02 14:10:23 +0200 | [diff] [blame] | 508 | tests. In addition, it supports marking a test as an "expected failure," a test |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 509 | that is broken and will fail, but shouldn't be counted as a failure on a |
| 510 | :class:`TestResult`. |
| 511 | |
| 512 | Skipping a test is simply a matter of using the :func:`skip` :term:`decorator` |
| 513 | or one of its conditional variants. |
| 514 | |
Ezio Melotti | fed69ba | 2013-03-01 21:26:04 +0200 | [diff] [blame] | 515 | Basic skipping looks like this:: |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 516 | |
| 517 | class MyTestCase(unittest.TestCase): |
| 518 | |
| 519 | @unittest.skip("demonstrating skipping") |
| 520 | def test_nothing(self): |
| 521 | self.fail("shouldn't happen") |
| 522 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 523 | @unittest.skipIf(mylib.__version__ < (1, 3), |
| 524 | "not supported in this library version") |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 525 | def test_format(self): |
| 526 | # Tests that work for only a certain version of the library. |
| 527 | pass |
| 528 | |
| 529 | @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") |
| 530 | def test_windows_support(self): |
| 531 | # windows specific testing code |
| 532 | pass |
| 533 | |
Ezio Melotti | fed69ba | 2013-03-01 21:26:04 +0200 | [diff] [blame] | 534 | This is the output of running the example above in verbose mode:: |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 535 | |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 536 | test_format (__main__.MyTestCase) ... skipped 'not supported in this library version' |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 537 | test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping' |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 538 | test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows' |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 539 | |
| 540 | ---------------------------------------------------------------------- |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 541 | Ran 3 tests in 0.005s |
| 542 | |
| 543 | OK (skipped=3) |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 544 | |
Ezio Melotti | fed69ba | 2013-03-01 21:26:04 +0200 | [diff] [blame] | 545 | Classes can be skipped just like methods:: |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 546 | |
Sandro Tosi | 317075d | 2012-03-31 18:34:59 +0200 | [diff] [blame] | 547 | @unittest.skip("showing class skipping") |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 548 | class MySkippedTestCase(unittest.TestCase): |
| 549 | def test_not_run(self): |
| 550 | pass |
| 551 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 552 | :meth:`TestCase.setUp` can also skip the test. This is useful when a resource |
| 553 | that needs to be set up is not available. |
| 554 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 555 | Expected failures use the :func:`expectedFailure` decorator. :: |
| 556 | |
| 557 | class ExpectedFailureTestCase(unittest.TestCase): |
| 558 | @unittest.expectedFailure |
| 559 | def test_fail(self): |
| 560 | self.assertEqual(1, 0, "broken") |
| 561 | |
| 562 | It's easy to roll your own skipping decorators by making a decorator that calls |
| 563 | :func:`skip` on the test when it wants it to be skipped. This decorator skips |
Ezio Melotti | fed69ba | 2013-03-01 21:26:04 +0200 | [diff] [blame] | 564 | the test unless the passed object has a certain attribute:: |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 565 | |
| 566 | def skipUnlessHasattr(obj, attr): |
| 567 | if hasattr(obj, attr): |
| 568 | return lambda func: func |
Ezio Melotti | 265281a | 2013-03-27 20:11:55 +0200 | [diff] [blame] | 569 | return unittest.skip("{!r} doesn't have {!r}".format(obj, attr)) |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 570 | |
| 571 | The following decorators implement test skipping and expected failures: |
| 572 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 573 | .. decorator:: skip(reason) |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 574 | |
| 575 | Unconditionally skip the decorated test. *reason* should describe why the |
| 576 | test is being skipped. |
| 577 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 578 | .. decorator:: skipIf(condition, reason) |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 579 | |
| 580 | Skip the decorated test if *condition* is true. |
| 581 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 582 | .. decorator:: skipUnless(condition, reason) |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 583 | |
Georg Brandl | 6faee4e | 2010-09-21 14:48:28 +0000 | [diff] [blame] | 584 | Skip the decorated test unless *condition* is true. |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 585 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 586 | .. decorator:: expectedFailure |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 587 | |
| 588 | Mark the test as an expected failure. If the test fails when run, the test |
| 589 | is not counted as a failure. |
| 590 | |
Ezio Melotti | 265281a | 2013-03-27 20:11:55 +0200 | [diff] [blame] | 591 | .. exception:: SkipTest(reason) |
| 592 | |
| 593 | This exception is raised to skip a test. |
| 594 | |
| 595 | Usually you can use :meth:`TestCase.skipTest` or one of the skipping |
| 596 | decorators instead of raising this directly. |
| 597 | |
R David Murray | 42fa110 | 2014-01-03 13:03:36 -0500 | [diff] [blame] | 598 | Skipped tests will not have :meth:`~TestCase.setUp` or :meth:`~TestCase.tearDown` run around them. |
| 599 | Skipped classes will not have :meth:`~TestCase.setUpClass` or :meth:`~TestCase.tearDownClass` run. |
| 600 | Skipped modules will not have :func:`setUpModule` or :func:`tearDownModule` run. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 601 | |
Benjamin Peterson | 5254c04 | 2009-03-23 22:25:03 +0000 | [diff] [blame] | 602 | |
Antoine Pitrou | c9b3ef2 | 2013-03-20 20:16:47 +0100 | [diff] [blame] | 603 | .. _subtests: |
| 604 | |
| 605 | Distinguishing test iterations using subtests |
| 606 | --------------------------------------------- |
| 607 | |
| 608 | .. versionadded:: 3.4 |
| 609 | |
| 610 | When some of your tests differ only by a some very small differences, for |
| 611 | instance some parameters, unittest allows you to distinguish them inside |
| 612 | the body of a test method using the :meth:`~TestCase.subTest` context manager. |
| 613 | |
| 614 | For example, the following test:: |
| 615 | |
| 616 | class NumbersTest(unittest.TestCase): |
| 617 | |
| 618 | def test_even(self): |
| 619 | """ |
| 620 | Test that numbers between 0 and 5 are all even. |
| 621 | """ |
| 622 | for i in range(0, 6): |
| 623 | with self.subTest(i=i): |
| 624 | self.assertEqual(i % 2, 0) |
| 625 | |
| 626 | will produce the following output:: |
| 627 | |
| 628 | ====================================================================== |
| 629 | FAIL: test_even (__main__.NumbersTest) (i=1) |
| 630 | ---------------------------------------------------------------------- |
| 631 | Traceback (most recent call last): |
| 632 | File "subtests.py", line 32, in test_even |
| 633 | self.assertEqual(i % 2, 0) |
| 634 | AssertionError: 1 != 0 |
| 635 | |
| 636 | ====================================================================== |
| 637 | FAIL: test_even (__main__.NumbersTest) (i=3) |
| 638 | ---------------------------------------------------------------------- |
| 639 | Traceback (most recent call last): |
| 640 | File "subtests.py", line 32, in test_even |
| 641 | self.assertEqual(i % 2, 0) |
| 642 | AssertionError: 1 != 0 |
| 643 | |
| 644 | ====================================================================== |
| 645 | FAIL: test_even (__main__.NumbersTest) (i=5) |
| 646 | ---------------------------------------------------------------------- |
| 647 | Traceback (most recent call last): |
| 648 | File "subtests.py", line 32, in test_even |
| 649 | self.assertEqual(i % 2, 0) |
| 650 | AssertionError: 1 != 0 |
| 651 | |
| 652 | Without using a subtest, execution would stop after the first failure, |
| 653 | and the error would be less easy to diagnose because the value of ``i`` |
| 654 | wouldn't be displayed:: |
| 655 | |
| 656 | ====================================================================== |
| 657 | FAIL: test_even (__main__.NumbersTest) |
| 658 | ---------------------------------------------------------------------- |
| 659 | Traceback (most recent call last): |
| 660 | File "subtests.py", line 32, in test_even |
| 661 | self.assertEqual(i % 2, 0) |
| 662 | AssertionError: 1 != 0 |
| 663 | |
| 664 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 665 | .. _unittest-contents: |
| 666 | |
| 667 | Classes and functions |
| 668 | --------------------- |
| 669 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 670 | This section describes in depth the API of :mod:`unittest`. |
| 671 | |
| 672 | |
| 673 | .. _testcase-objects: |
| 674 | |
| 675 | Test cases |
| 676 | ~~~~~~~~~~ |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 677 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 678 | .. class:: TestCase(methodName='runTest') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 679 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 680 | Instances of the :class:`TestCase` class represent the logical test units |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 681 | in the :mod:`unittest` universe. This class is intended to be used as a base |
| 682 | class, with specific tests being implemented by concrete subclasses. This class |
| 683 | implements the interface needed by the test runner to allow it to drive the |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 684 | tests, and methods that the test code can use to check for and report various |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 685 | kinds of failure. |
| 686 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 687 | Each instance of :class:`TestCase` will run a single base method: the method |
Robert Collins | e02f6c2 | 2015-07-23 06:37:26 +1200 | [diff] [blame] | 688 | named *methodName*. |
| 689 | In most uses of :class:`TestCase`, you will neither change |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 690 | the *methodName* nor reimplement the default ``runTest()`` method. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 691 | |
Michael Foord | 1341bb0 | 2011-03-14 19:01:46 -0400 | [diff] [blame] | 692 | .. versionchanged:: 3.2 |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 693 | :class:`TestCase` can be instantiated successfully without providing a |
| 694 | *methodName*. This makes it easier to experiment with :class:`TestCase` |
| 695 | from the interactive interpreter. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 696 | |
| 697 | :class:`TestCase` instances provide three groups of methods: one group used |
| 698 | to run the test, another used by the test implementation to check conditions |
| 699 | and report failures, and some inquiry methods allowing information about the |
| 700 | test itself to be gathered. |
| 701 | |
| 702 | Methods in the first group (running the test) are: |
| 703 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 704 | .. method:: setUp() |
| 705 | |
| 706 | Method called to prepare the test fixture. This is called immediately |
Terry Jan Reedy | 7f84d1e | 2014-04-15 23:44:14 -0400 | [diff] [blame] | 707 | before calling the test method; other than :exc:`AssertionError` or :exc:`SkipTest`, |
| 708 | any exception raised by this method will be considered an error rather than |
Terry Jan Reedy | 6ac4240 | 2014-04-15 23:38:18 -0400 | [diff] [blame] | 709 | a test failure. The default implementation does nothing. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 710 | |
| 711 | |
| 712 | .. method:: tearDown() |
| 713 | |
| 714 | Method called immediately after the test method has been called and the |
| 715 | result recorded. This is called even if the test method raised an |
| 716 | exception, so the implementation in subclasses may need to be particularly |
Ezio Melotti | 8dea74f | 2016-03-13 09:40:09 +0200 | [diff] [blame] | 717 | careful about checking internal state. Any exception, other than |
| 718 | :exc:`AssertionError` or :exc:`SkipTest`, raised by this method will be |
| 719 | considered an additional error rather than a test failure (thus increasing |
| 720 | the total number of reported errors). This method will only be called if |
| 721 | the :meth:`setUp` succeeds, regardless of the outcome of the test method. |
| 722 | The default implementation does nothing. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 723 | |
| 724 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 725 | .. method:: setUpClass() |
| 726 | |
Ville Skyttä | c33bb5d | 2018-08-23 17:49:18 +0300 | [diff] [blame] | 727 | A class method called before tests in an individual class are run. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 728 | ``setUpClass`` is called with the class as the only argument |
| 729 | and must be decorated as a :func:`classmethod`:: |
| 730 | |
| 731 | @classmethod |
| 732 | def setUpClass(cls): |
| 733 | ... |
| 734 | |
| 735 | See `Class and Module Fixtures`_ for more details. |
| 736 | |
| 737 | .. versionadded:: 3.2 |
| 738 | |
| 739 | |
| 740 | .. method:: tearDownClass() |
| 741 | |
| 742 | A class method called after tests in an individual class have run. |
| 743 | ``tearDownClass`` is called with the class as the only argument |
| 744 | and must be decorated as a :meth:`classmethod`:: |
| 745 | |
| 746 | @classmethod |
| 747 | def tearDownClass(cls): |
| 748 | ... |
| 749 | |
| 750 | See `Class and Module Fixtures`_ for more details. |
| 751 | |
| 752 | .. versionadded:: 3.2 |
| 753 | |
| 754 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 755 | .. method:: run(result=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 756 | |
Antoine Pitrou | 2c5e950 | 2013-01-20 01:29:39 +0100 | [diff] [blame] | 757 | Run the test, collecting the result into the :class:`TestResult` object |
| 758 | passed as *result*. If *result* is omitted or ``None``, a temporary |
| 759 | result object is created (by calling the :meth:`defaultTestResult` |
| 760 | method) and used. The result object is returned to :meth:`run`'s |
| 761 | caller. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 762 | |
| 763 | The same effect may be had by simply calling the :class:`TestCase` |
| 764 | instance. |
| 765 | |
Michael Foord | 1341bb0 | 2011-03-14 19:01:46 -0400 | [diff] [blame] | 766 | .. versionchanged:: 3.3 |
| 767 | Previous versions of ``run`` did not return the result. Neither did |
| 768 | calling an instance. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 769 | |
Benjamin Peterson | e549ead | 2009-03-28 21:42:05 +0000 | [diff] [blame] | 770 | .. method:: skipTest(reason) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 771 | |
Stefan Krah | a5bf3f5 | 2010-05-19 16:09:41 +0000 | [diff] [blame] | 772 | Calling this during a test method or :meth:`setUp` skips the current |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 773 | test. See :ref:`unittest-skipping` for more information. |
| 774 | |
Ezio Melotti | 7afd3f5 | 2010-04-20 09:32:54 +0000 | [diff] [blame] | 775 | .. versionadded:: 3.1 |
Benjamin Peterson | 08bf91c | 2010-04-11 16:12:57 +0000 | [diff] [blame] | 776 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 777 | |
Antoine Pitrou | c9b3ef2 | 2013-03-20 20:16:47 +0100 | [diff] [blame] | 778 | .. method:: subTest(msg=None, **params) |
| 779 | |
| 780 | Return a context manager which executes the enclosed code block as a |
| 781 | subtest. *msg* and *params* are optional, arbitrary values which are |
| 782 | displayed whenever a subtest fails, allowing you to identify them |
| 783 | clearly. |
| 784 | |
| 785 | A test case can contain any number of subtest declarations, and |
| 786 | they can be arbitrarily nested. |
| 787 | |
| 788 | See :ref:`subtests` for more information. |
| 789 | |
| 790 | .. versionadded:: 3.4 |
| 791 | |
| 792 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 793 | .. method:: debug() |
| 794 | |
| 795 | Run the test without collecting the result. This allows exceptions raised |
| 796 | by the test to be propagated to the caller, and can be used to support |
| 797 | running tests under a debugger. |
| 798 | |
Ezio Melotti | 22170ed | 2010-11-20 09:57:27 +0000 | [diff] [blame] | 799 | .. _assert-methods: |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 800 | |
Ezio Melotti | f418db2 | 2016-01-12 11:03:31 +0200 | [diff] [blame] | 801 | The :class:`TestCase` class provides several assert methods to check for and |
| 802 | report failures. The following table lists the most commonly used methods |
| 803 | (see the tables below for more assert methods): |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 804 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 805 | +-----------------------------------------+-----------------------------+---------------+ |
| 806 | | Method | Checks that | New in | |
| 807 | +=========================================+=============================+===============+ |
| 808 | | :meth:`assertEqual(a, b) | ``a == b`` | | |
| 809 | | <TestCase.assertEqual>` | | | |
| 810 | +-----------------------------------------+-----------------------------+---------------+ |
| 811 | | :meth:`assertNotEqual(a, b) | ``a != b`` | | |
| 812 | | <TestCase.assertNotEqual>` | | | |
| 813 | +-----------------------------------------+-----------------------------+---------------+ |
| 814 | | :meth:`assertTrue(x) | ``bool(x) is True`` | | |
| 815 | | <TestCase.assertTrue>` | | | |
| 816 | +-----------------------------------------+-----------------------------+---------------+ |
| 817 | | :meth:`assertFalse(x) | ``bool(x) is False`` | | |
| 818 | | <TestCase.assertFalse>` | | | |
| 819 | +-----------------------------------------+-----------------------------+---------------+ |
| 820 | | :meth:`assertIs(a, b) | ``a is b`` | 3.1 | |
| 821 | | <TestCase.assertIs>` | | | |
| 822 | +-----------------------------------------+-----------------------------+---------------+ |
| 823 | | :meth:`assertIsNot(a, b) | ``a is not b`` | 3.1 | |
| 824 | | <TestCase.assertIsNot>` | | | |
| 825 | +-----------------------------------------+-----------------------------+---------------+ |
| 826 | | :meth:`assertIsNone(x) | ``x is None`` | 3.1 | |
| 827 | | <TestCase.assertIsNone>` | | | |
| 828 | +-----------------------------------------+-----------------------------+---------------+ |
| 829 | | :meth:`assertIsNotNone(x) | ``x is not None`` | 3.1 | |
| 830 | | <TestCase.assertIsNotNone>` | | | |
| 831 | +-----------------------------------------+-----------------------------+---------------+ |
| 832 | | :meth:`assertIn(a, b) | ``a in b`` | 3.1 | |
| 833 | | <TestCase.assertIn>` | | | |
| 834 | +-----------------------------------------+-----------------------------+---------------+ |
| 835 | | :meth:`assertNotIn(a, b) | ``a not in b`` | 3.1 | |
| 836 | | <TestCase.assertNotIn>` | | | |
| 837 | +-----------------------------------------+-----------------------------+---------------+ |
| 838 | | :meth:`assertIsInstance(a, b) | ``isinstance(a, b)`` | 3.2 | |
| 839 | | <TestCase.assertIsInstance>` | | | |
| 840 | +-----------------------------------------+-----------------------------+---------------+ |
| 841 | | :meth:`assertNotIsInstance(a, b) | ``not isinstance(a, b)`` | 3.2 | |
| 842 | | <TestCase.assertNotIsInstance>` | | | |
| 843 | +-----------------------------------------+-----------------------------+---------------+ |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 844 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 845 | All the assert methods accept a *msg* argument that, if specified, is used |
| 846 | as the error message on failure (see also :data:`longMessage`). |
| 847 | Note that the *msg* keyword argument can be passed to :meth:`assertRaises`, |
| 848 | :meth:`assertRaisesRegex`, :meth:`assertWarns`, :meth:`assertWarnsRegex` |
| 849 | only when they are used as a context manager. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 850 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 851 | .. method:: assertEqual(first, second, msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 852 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 853 | Test that *first* and *second* are equal. If the values do not |
Ezio Melotti | addc6f5 | 2010-12-18 20:00:04 +0000 | [diff] [blame] | 854 | compare equal, the test will fail. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 855 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 856 | In addition, if *first* and *second* are the exact same type and one of |
Michael Foord | 0283495 | 2010-02-08 23:10:39 +0000 | [diff] [blame] | 857 | list, tuple, dict, set, frozenset or str or any type that a subclass |
Ezio Melotti | 9ecb6be | 2012-01-16 08:28:54 +0200 | [diff] [blame] | 858 | registers with :meth:`addTypeEqualityFunc` the type-specific equality |
Michael Foord | 0283495 | 2010-02-08 23:10:39 +0000 | [diff] [blame] | 859 | function will be called in order to generate a more useful default |
Ezio Melotti | 22170ed | 2010-11-20 09:57:27 +0000 | [diff] [blame] | 860 | error message (see also the :ref:`list of type-specific methods |
| 861 | <type-specific-methods>`). |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 862 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 863 | .. versionchanged:: 3.1 |
Ezio Melotti | 9ecb6be | 2012-01-16 08:28:54 +0200 | [diff] [blame] | 864 | Added the automatic calling of type-specific equality function. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 865 | |
Michael Foord | 28a817e | 2010-02-09 00:03:57 +0000 | [diff] [blame] | 866 | .. versionchanged:: 3.2 |
| 867 | :meth:`assertMultiLineEqual` added as the default type equality |
| 868 | function for comparing strings. |
Michael Foord | 0283495 | 2010-02-08 23:10:39 +0000 | [diff] [blame] | 869 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 870 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 871 | .. method:: assertNotEqual(first, second, msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 872 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 873 | Test that *first* and *second* are not equal. If the values do |
Ezio Melotti | addc6f5 | 2010-12-18 20:00:04 +0000 | [diff] [blame] | 874 | compare equal, the test will fail. |
Benjamin Peterson | 70e32c8 | 2009-03-24 01:00:11 +0000 | [diff] [blame] | 875 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 876 | .. method:: assertTrue(expr, msg=None) |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 877 | assertFalse(expr, msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 878 | |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 879 | Test that *expr* is true (or false). |
Benjamin Peterson | 4ac9ce4 | 2009-10-04 14:49:41 +0000 | [diff] [blame] | 880 | |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 881 | Note that this is equivalent to ``bool(expr) is True`` and not to ``expr |
| 882 | is True`` (use ``assertIs(expr, True)`` for the latter). This method |
| 883 | should also be avoided when more specific methods are available (e.g. |
| 884 | ``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they |
| 885 | provide a better error message in case of failure. |
Benjamin Peterson | 4ac9ce4 | 2009-10-04 14:49:41 +0000 | [diff] [blame] | 886 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 887 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 888 | .. method:: assertIs(first, second, msg=None) |
| 889 | assertIsNot(first, second, msg=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 890 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 891 | Test that *first* and *second* evaluate (or don't evaluate) to the |
Ezio Melotti | addc6f5 | 2010-12-18 20:00:04 +0000 | [diff] [blame] | 892 | same object. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 893 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 894 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 895 | |
| 896 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 897 | .. method:: assertIsNone(expr, msg=None) |
Ezio Melotti | 9794a26 | 2010-11-04 14:52:13 +0000 | [diff] [blame] | 898 | assertIsNotNone(expr, msg=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 899 | |
Serhiy Storchaka | ecf41da | 2016-10-19 16:29:26 +0300 | [diff] [blame] | 900 | Test that *expr* is (or is not) ``None``. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 901 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 902 | .. versionadded:: 3.1 |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 903 | |
| 904 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 905 | .. method:: assertIn(first, second, msg=None) |
| 906 | assertNotIn(first, second, msg=None) |
| 907 | |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 908 | Test that *first* is (or is not) in *second*. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 909 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 910 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 911 | |
| 912 | |
Ezio Melotti | 9c02c2f | 2010-11-03 20:45:31 +0000 | [diff] [blame] | 913 | .. method:: assertIsInstance(obj, cls, msg=None) |
Ezio Melotti | 9794a26 | 2010-11-04 14:52:13 +0000 | [diff] [blame] | 914 | assertNotIsInstance(obj, cls, msg=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 915 | |
Ezio Melotti | 9794a26 | 2010-11-04 14:52:13 +0000 | [diff] [blame] | 916 | Test that *obj* is (or is not) an instance of *cls* (which can be a |
| 917 | class or a tuple of classes, as supported by :func:`isinstance`). |
Ezio Melotti | 80a61e8 | 2011-12-19 07:04:48 +0200 | [diff] [blame] | 918 | To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 919 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 920 | .. versionadded:: 3.2 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 921 | |
| 922 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 923 | |
Ezio Melotti | f418db2 | 2016-01-12 11:03:31 +0200 | [diff] [blame] | 924 | It is also possible to check the production of exceptions, warnings, and |
Antoine Pitrou | 7c89ae2 | 2013-09-15 02:01:39 +0200 | [diff] [blame] | 925 | log messages using the following methods: |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 926 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 927 | +---------------------------------------------------------+--------------------------------------+------------+ |
| 928 | | Method | Checks that | New in | |
| 929 | +=========================================================+======================================+============+ |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 930 | | :meth:`assertRaises(exc, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 931 | | <TestCase.assertRaises>` | | | |
| 932 | +---------------------------------------------------------+--------------------------------------+------------+ |
Ezio Melotti | 560a778 | 2013-09-13 22:17:40 +0300 | [diff] [blame] | 933 | | :meth:`assertRaisesRegex(exc, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *exc* | 3.1 | |
| 934 | | <TestCase.assertRaisesRegex>` | and the message matches regex *r* | | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 935 | +---------------------------------------------------------+--------------------------------------+------------+ |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 936 | | :meth:`assertWarns(warn, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 937 | | <TestCase.assertWarns>` | | | |
| 938 | +---------------------------------------------------------+--------------------------------------+------------+ |
Ezio Melotti | 560a778 | 2013-09-13 22:17:40 +0300 | [diff] [blame] | 939 | | :meth:`assertWarnsRegex(warn, r, fun, *args, **kwds) | ``fun(*args, **kwds)`` raises *warn* | 3.2 | |
| 940 | | <TestCase.assertWarnsRegex>` | and the message matches regex *r* | | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 941 | +---------------------------------------------------------+--------------------------------------+------------+ |
Georg Brandl | ed007d5 | 2013-11-24 16:09:26 +0100 | [diff] [blame] | 942 | | :meth:`assertLogs(logger, level) | The ``with`` block logs on *logger* | 3.4 | |
| 943 | | <TestCase.assertLogs>` | with minimum *level* | | |
Antoine Pitrou | 7c89ae2 | 2013-09-15 02:01:39 +0200 | [diff] [blame] | 944 | +---------------------------------------------------------+--------------------------------------+------------+ |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 945 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 946 | .. method:: assertRaises(exception, callable, *args, **kwds) |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 947 | assertRaises(exception, msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 948 | |
| 949 | Test that an exception is raised when *callable* is called with any |
| 950 | positional or keyword arguments that are also passed to |
| 951 | :meth:`assertRaises`. The test passes if *exception* is raised, is an |
| 952 | error if another exception is raised, or fails if no exception is raised. |
| 953 | To catch any of a group of exceptions, a tuple containing the exception |
| 954 | classes may be passed as *exception*. |
| 955 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 956 | If only the *exception* and possibly the *msg* arguments are given, |
| 957 | return a context manager so that the code under test can be written |
| 958 | inline rather than as a function:: |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 959 | |
Michael Foord | 41531f2 | 2010-02-05 21:13:40 +0000 | [diff] [blame] | 960 | with self.assertRaises(SomeException): |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 961 | do_something() |
| 962 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 963 | When used as a context manager, :meth:`assertRaises` accepts the |
| 964 | additional keyword argument *msg*. |
| 965 | |
Kristján Valur Jónsson | 92a653a | 2009-11-13 16:10:13 +0000 | [diff] [blame] | 966 | The context manager will store the caught exception object in its |
Ezio Melotti | 4900823 | 2010-02-08 21:57:48 +0000 | [diff] [blame] | 967 | :attr:`exception` attribute. This can be useful if the intention |
Michael Foord | 41531f2 | 2010-02-05 21:13:40 +0000 | [diff] [blame] | 968 | is to perform additional checks on the exception raised:: |
Kristján Valur Jónsson | 92a653a | 2009-11-13 16:10:13 +0000 | [diff] [blame] | 969 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 970 | with self.assertRaises(SomeException) as cm: |
| 971 | do_something() |
Michael Foord | 41531f2 | 2010-02-05 21:13:40 +0000 | [diff] [blame] | 972 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 973 | the_exception = cm.exception |
| 974 | self.assertEqual(the_exception.error_code, 3) |
Michael Foord | 41531f2 | 2010-02-05 21:13:40 +0000 | [diff] [blame] | 975 | |
Ezio Melotti | 4900823 | 2010-02-08 21:57:48 +0000 | [diff] [blame] | 976 | .. versionchanged:: 3.1 |
Benjamin Peterson | ded31c4 | 2009-03-30 15:04:16 +0000 | [diff] [blame] | 977 | Added the ability to use :meth:`assertRaises` as a context manager. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 978 | |
Ezio Melotti | 4900823 | 2010-02-08 21:57:48 +0000 | [diff] [blame] | 979 | .. versionchanged:: 3.2 |
| 980 | Added the :attr:`exception` attribute. |
| 981 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 982 | .. versionchanged:: 3.3 |
| 983 | Added the *msg* keyword argument when used as a context manager. |
| 984 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 985 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 986 | .. method:: assertRaisesRegex(exception, regex, callable, *args, **kwds) |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 987 | assertRaisesRegex(exception, regex, msg=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 988 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 989 | Like :meth:`assertRaises` but also tests that *regex* matches |
| 990 | on the string representation of the raised exception. *regex* may be |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 991 | a regular expression object or a string containing a regular expression |
| 992 | suitable for use by :func:`re.search`. Examples:: |
| 993 | |
Terry Jan Reedy | c4565a9 | 2013-06-29 13:15:43 -0400 | [diff] [blame] | 994 | self.assertRaisesRegex(ValueError, "invalid literal for.*XYZ'$", |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 995 | int, 'XYZ') |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 996 | |
| 997 | or:: |
| 998 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 999 | with self.assertRaisesRegex(ValueError, 'literal'): |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1000 | int('XYZ') |
| 1001 | |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1002 | .. versionadded:: 3.1 |
| 1003 | under the name ``assertRaisesRegexp``. |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1004 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1005 | .. versionchanged:: 3.2 |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1006 | Renamed to :meth:`assertRaisesRegex`. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1007 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1008 | .. versionchanged:: 3.3 |
| 1009 | Added the *msg* keyword argument when used as a context manager. |
| 1010 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1011 | |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1012 | .. method:: assertWarns(warning, callable, *args, **kwds) |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1013 | assertWarns(warning, msg=None) |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1014 | |
| 1015 | Test that a warning is triggered when *callable* is called with any |
| 1016 | positional or keyword arguments that are also passed to |
| 1017 | :meth:`assertWarns`. The test passes if *warning* is triggered and |
Terry Jan Reedy | 778cba7 | 2013-07-30 22:31:06 -0400 | [diff] [blame] | 1018 | fails if it isn't. Any exception is an error. |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1019 | To catch any of a group of warnings, a tuple containing the warning |
| 1020 | classes may be passed as *warnings*. |
| 1021 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1022 | If only the *warning* and possibly the *msg* arguments are given, |
Terry Jan Reedy | 778cba7 | 2013-07-30 22:31:06 -0400 | [diff] [blame] | 1023 | return a context manager so that the code under test can be written |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1024 | inline rather than as a function:: |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1025 | |
| 1026 | with self.assertWarns(SomeWarning): |
| 1027 | do_something() |
| 1028 | |
Terry Jan Reedy | 9eda66d | 2013-07-27 16:15:29 -0400 | [diff] [blame] | 1029 | When used as a context manager, :meth:`assertWarns` accepts the |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1030 | additional keyword argument *msg*. |
| 1031 | |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1032 | The context manager will store the caught warning object in its |
| 1033 | :attr:`warning` attribute, and the source line which triggered the |
| 1034 | warnings in the :attr:`filename` and :attr:`lineno` attributes. |
| 1035 | This can be useful if the intention is to perform additional checks |
Terry Jan Reedy | 778cba7 | 2013-07-30 22:31:06 -0400 | [diff] [blame] | 1036 | on the warning caught:: |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1037 | |
| 1038 | with self.assertWarns(SomeWarning) as cm: |
| 1039 | do_something() |
| 1040 | |
| 1041 | self.assertIn('myfile.py', cm.filename) |
| 1042 | self.assertEqual(320, cm.lineno) |
| 1043 | |
| 1044 | This method works regardless of the warning filters in place when it |
| 1045 | is called. |
| 1046 | |
| 1047 | .. versionadded:: 3.2 |
| 1048 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1049 | .. versionchanged:: 3.3 |
| 1050 | Added the *msg* keyword argument when used as a context manager. |
| 1051 | |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1052 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1053 | .. method:: assertWarnsRegex(warning, regex, callable, *args, **kwds) |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1054 | assertWarnsRegex(warning, regex, msg=None) |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1055 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1056 | Like :meth:`assertWarns` but also tests that *regex* matches on the |
| 1057 | message of the triggered warning. *regex* may be a regular expression |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1058 | object or a string containing a regular expression suitable for use |
| 1059 | by :func:`re.search`. Example:: |
| 1060 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1061 | self.assertWarnsRegex(DeprecationWarning, |
| 1062 | r'legacy_function\(\) is deprecated', |
| 1063 | legacy_function, 'XYZ') |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1064 | |
| 1065 | or:: |
| 1066 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1067 | with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'): |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1068 | frobnicate('/etc/passwd') |
| 1069 | |
| 1070 | .. versionadded:: 3.2 |
| 1071 | |
Ezio Melotti | b4dc250 | 2011-05-06 15:01:41 +0300 | [diff] [blame] | 1072 | .. versionchanged:: 3.3 |
| 1073 | Added the *msg* keyword argument when used as a context manager. |
Antoine Pitrou | 4bc12ef | 2010-09-06 19:25:46 +0000 | [diff] [blame] | 1074 | |
Antoine Pitrou | 0715b9f | 2013-09-14 19:45:47 +0200 | [diff] [blame] | 1075 | .. method:: assertLogs(logger=None, level=None) |
| 1076 | |
| 1077 | A context manager to test that at least one message is logged on |
| 1078 | the *logger* or one of its children, with at least the given |
| 1079 | *level*. |
| 1080 | |
| 1081 | If given, *logger* should be a :class:`logging.Logger` object or a |
| 1082 | :class:`str` giving the name of a logger. The default is the root |
| 1083 | logger, which will catch all messages. |
| 1084 | |
| 1085 | If given, *level* should be either a numeric logging level or |
| 1086 | its string equivalent (for example either ``"ERROR"`` or |
| 1087 | :attr:`logging.ERROR`). The default is :attr:`logging.INFO`. |
| 1088 | |
| 1089 | The test passes if at least one message emitted inside the ``with`` |
| 1090 | block matches the *logger* and *level* conditions, otherwise it fails. |
| 1091 | |
| 1092 | The object returned by the context manager is a recording helper |
| 1093 | which keeps tracks of the matching log messages. It has two |
| 1094 | attributes: |
| 1095 | |
| 1096 | .. attribute:: records |
| 1097 | |
| 1098 | A list of :class:`logging.LogRecord` objects of the matching |
| 1099 | log messages. |
| 1100 | |
| 1101 | .. attribute:: output |
| 1102 | |
| 1103 | A list of :class:`str` objects with the formatted output of |
| 1104 | matching messages. |
| 1105 | |
| 1106 | Example:: |
| 1107 | |
| 1108 | with self.assertLogs('foo', level='INFO') as cm: |
| 1109 | logging.getLogger('foo').info('first message') |
| 1110 | logging.getLogger('foo.bar').error('second message') |
| 1111 | self.assertEqual(cm.output, ['INFO:foo:first message', |
| 1112 | 'ERROR:foo.bar:second message']) |
| 1113 | |
| 1114 | .. versionadded:: 3.4 |
| 1115 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1116 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1117 | There are also other methods used to perform more specific checks, such as: |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1118 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1119 | +---------------------------------------+--------------------------------+--------------+ |
| 1120 | | Method | Checks that | New in | |
| 1121 | +=======================================+================================+==============+ |
| 1122 | | :meth:`assertAlmostEqual(a, b) | ``round(a-b, 7) == 0`` | | |
| 1123 | | <TestCase.assertAlmostEqual>` | | | |
| 1124 | +---------------------------------------+--------------------------------+--------------+ |
| 1125 | | :meth:`assertNotAlmostEqual(a, b) | ``round(a-b, 7) != 0`` | | |
| 1126 | | <TestCase.assertNotAlmostEqual>` | | | |
| 1127 | +---------------------------------------+--------------------------------+--------------+ |
| 1128 | | :meth:`assertGreater(a, b) | ``a > b`` | 3.1 | |
| 1129 | | <TestCase.assertGreater>` | | | |
| 1130 | +---------------------------------------+--------------------------------+--------------+ |
| 1131 | | :meth:`assertGreaterEqual(a, b) | ``a >= b`` | 3.1 | |
| 1132 | | <TestCase.assertGreaterEqual>` | | | |
| 1133 | +---------------------------------------+--------------------------------+--------------+ |
| 1134 | | :meth:`assertLess(a, b) | ``a < b`` | 3.1 | |
| 1135 | | <TestCase.assertLess>` | | | |
| 1136 | +---------------------------------------+--------------------------------+--------------+ |
| 1137 | | :meth:`assertLessEqual(a, b) | ``a <= b`` | 3.1 | |
| 1138 | | <TestCase.assertLessEqual>` | | | |
| 1139 | +---------------------------------------+--------------------------------+--------------+ |
Ezio Melotti | 560a778 | 2013-09-13 22:17:40 +0300 | [diff] [blame] | 1140 | | :meth:`assertRegex(s, r) | ``r.search(s)`` | 3.1 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1141 | | <TestCase.assertRegex>` | | | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1142 | +---------------------------------------+--------------------------------+--------------+ |
Ezio Melotti | 560a778 | 2013-09-13 22:17:40 +0300 | [diff] [blame] | 1143 | | :meth:`assertNotRegex(s, r) | ``not r.search(s)`` | 3.2 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1144 | | <TestCase.assertNotRegex>` | | | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1145 | +---------------------------------------+--------------------------------+--------------+ |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 1146 | | :meth:`assertCountEqual(a, b) | *a* and *b* have the same | 3.2 | |
Raymond Hettinger | 6e165b3 | 2010-11-27 09:31:37 +0000 | [diff] [blame] | 1147 | | <TestCase.assertCountEqual>` | elements in the same number, | | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1148 | | | regardless of their order | | |
| 1149 | +---------------------------------------+--------------------------------+--------------+ |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1150 | |
| 1151 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1152 | .. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None) |
| 1153 | assertNotAlmostEqual(first, second, places=7, msg=None, delta=None) |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1154 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1155 | Test that *first* and *second* are approximately (or not approximately) |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 1156 | equal by computing the difference, rounding to the given number of |
| 1157 | decimal *places* (default 7), and comparing to zero. Note that these |
| 1158 | methods round the values to the given number of *decimal places* (i.e. |
| 1159 | like the :func:`round` function) and not *significant digits*. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1160 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1161 | If *delta* is supplied instead of *places* then the difference |
Ezio Melotti | d51914c | 2013-08-11 13:04:50 +0300 | [diff] [blame] | 1162 | between *first* and *second* must be less or equal to (or greater than) *delta*. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1163 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1164 | Supplying both *delta* and *places* raises a ``TypeError``. |
Benjamin Peterson | f47ed4a | 2009-04-11 20:45:40 +0000 | [diff] [blame] | 1165 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1166 | .. versionchanged:: 3.2 |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1167 | :meth:`assertAlmostEqual` automatically considers almost equal objects |
| 1168 | that compare equal. :meth:`assertNotAlmostEqual` automatically fails |
| 1169 | if the objects compare equal. Added the *delta* keyword argument. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1170 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1171 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1172 | .. method:: assertGreater(first, second, msg=None) |
| 1173 | assertGreaterEqual(first, second, msg=None) |
| 1174 | assertLess(first, second, msg=None) |
| 1175 | assertLessEqual(first, second, msg=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1176 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1177 | Test that *first* is respectively >, >=, < or <= than *second* depending |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 1178 | on the method name. If not, the test will fail:: |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1179 | |
| 1180 | >>> self.assertGreaterEqual(3, 4) |
| 1181 | AssertionError: "3" unexpectedly not greater than or equal to "4" |
| 1182 | |
| 1183 | .. versionadded:: 3.1 |
| 1184 | |
| 1185 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1186 | .. method:: assertRegex(text, regex, msg=None) |
| 1187 | assertNotRegex(text, regex, msg=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1188 | |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1189 | Test that a *regex* search matches (or does not match) *text*. In case |
Ezio Melotti | 4841fd6 | 2010-11-05 15:43:40 +0000 | [diff] [blame] | 1190 | of failure, the error message will include the pattern and the *text* (or |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1191 | the pattern and the part of *text* that unexpectedly matched). *regex* |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1192 | may be a regular expression object or a string containing a regular |
| 1193 | expression suitable for use by :func:`re.search`. |
| 1194 | |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1195 | .. versionadded:: 3.1 |
| 1196 | under the name ``assertRegexpMatches``. |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1197 | .. versionchanged:: 3.2 |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1198 | The method ``assertRegexpMatches()`` has been renamed to |
| 1199 | :meth:`.assertRegex`. |
| 1200 | .. versionadded:: 3.2 |
| 1201 | :meth:`.assertNotRegex`. |
Jim Fasarakis-Hilliard | 74921ed | 2017-06-09 23:27:20 +0300 | [diff] [blame] | 1202 | .. versionadded:: 3.5 |
| 1203 | The name ``assertNotRegexpMatches`` is a deprecated alias |
| 1204 | for :meth:`.assertNotRegex`. |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1205 | |
| 1206 | |
Raymond Hettinger | 57bd00a | 2010-12-24 21:51:48 +0000 | [diff] [blame] | 1207 | .. method:: assertCountEqual(first, second, msg=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1208 | |
Raymond Hettinger | 57bd00a | 2010-12-24 21:51:48 +0000 | [diff] [blame] | 1209 | Test that sequence *first* contains the same elements as *second*, |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1210 | regardless of their order. When they don't, an error message listing the |
| 1211 | differences between the sequences will be generated. |
| 1212 | |
Raymond Hettinger | 57bd00a | 2010-12-24 21:51:48 +0000 | [diff] [blame] | 1213 | Duplicate elements are *not* ignored when comparing *first* and |
| 1214 | *second*. It verifies whether each element has the same count in both |
Raymond Hettinger | 6e165b3 | 2010-11-27 09:31:37 +0000 | [diff] [blame] | 1215 | sequences. Equivalent to: |
Raymond Hettinger | 57bd00a | 2010-12-24 21:51:48 +0000 | [diff] [blame] | 1216 | ``assertEqual(Counter(list(first)), Counter(list(second)))`` |
Raymond Hettinger | 6e165b3 | 2010-11-27 09:31:37 +0000 | [diff] [blame] | 1217 | but works with sequences of unhashable objects as well. |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1218 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1219 | .. versionadded:: 3.2 |
| 1220 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1221 | |
Ezio Melotti | 22170ed | 2010-11-20 09:57:27 +0000 | [diff] [blame] | 1222 | .. _type-specific-methods: |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1223 | |
Ezio Melotti | 22170ed | 2010-11-20 09:57:27 +0000 | [diff] [blame] | 1224 | The :meth:`assertEqual` method dispatches the equality check for objects of |
| 1225 | the same type to different type-specific methods. These methods are already |
| 1226 | implemented for most of the built-in types, but it's also possible to |
| 1227 | register new methods using :meth:`addTypeEqualityFunc`: |
| 1228 | |
| 1229 | .. method:: addTypeEqualityFunc(typeobj, function) |
| 1230 | |
| 1231 | Registers a type-specific method called by :meth:`assertEqual` to check |
| 1232 | if two objects of exactly the same *typeobj* (not subclasses) compare |
| 1233 | equal. *function* must take two positional arguments and a third msg=None |
| 1234 | keyword argument just as :meth:`assertEqual` does. It must raise |
| 1235 | :data:`self.failureException(msg) <failureException>` when inequality |
| 1236 | between the first two parameters is detected -- possibly providing useful |
| 1237 | information and explaining the inequalities in details in the error |
| 1238 | message. |
| 1239 | |
| 1240 | .. versionadded:: 3.1 |
| 1241 | |
| 1242 | The list of type-specific methods automatically used by |
| 1243 | :meth:`~TestCase.assertEqual` are summarized in the following table. Note |
| 1244 | that it's usually not necessary to invoke these methods directly. |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1245 | |
| 1246 | +-----------------------------------------+-----------------------------+--------------+ |
| 1247 | | Method | Used to compare | New in | |
| 1248 | +=========================================+=============================+==============+ |
| 1249 | | :meth:`assertMultiLineEqual(a, b) | strings | 3.1 | |
| 1250 | | <TestCase.assertMultiLineEqual>` | | | |
| 1251 | +-----------------------------------------+-----------------------------+--------------+ |
| 1252 | | :meth:`assertSequenceEqual(a, b) | sequences | 3.1 | |
| 1253 | | <TestCase.assertSequenceEqual>` | | | |
| 1254 | +-----------------------------------------+-----------------------------+--------------+ |
| 1255 | | :meth:`assertListEqual(a, b) | lists | 3.1 | |
| 1256 | | <TestCase.assertListEqual>` | | | |
| 1257 | +-----------------------------------------+-----------------------------+--------------+ |
| 1258 | | :meth:`assertTupleEqual(a, b) | tuples | 3.1 | |
| 1259 | | <TestCase.assertTupleEqual>` | | | |
| 1260 | +-----------------------------------------+-----------------------------+--------------+ |
| 1261 | | :meth:`assertSetEqual(a, b) | sets or frozensets | 3.1 | |
| 1262 | | <TestCase.assertSetEqual>` | | | |
| 1263 | +-----------------------------------------+-----------------------------+--------------+ |
| 1264 | | :meth:`assertDictEqual(a, b) | dicts | 3.1 | |
| 1265 | | <TestCase.assertDictEqual>` | | | |
| 1266 | +-----------------------------------------+-----------------------------+--------------+ |
| 1267 | |
| 1268 | |
| 1269 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1270 | .. method:: assertMultiLineEqual(first, second, msg=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1271 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1272 | Test that the multiline string *first* is equal to the string *second*. |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1273 | When not equal a diff of the two strings highlighting the differences |
| 1274 | will be included in the error message. This method is used by default |
| 1275 | when comparing strings with :meth:`assertEqual`. |
| 1276 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1277 | .. versionadded:: 3.1 |
| 1278 | |
| 1279 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1280 | .. method:: assertSequenceEqual(first, second, msg=None, seq_type=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1281 | |
| 1282 | Tests that two sequences are equal. If a *seq_type* is supplied, both |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1283 | *first* and *second* must be instances of *seq_type* or a failure will |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1284 | be raised. If the sequences are different an error message is |
| 1285 | constructed that shows the difference between the two. |
| 1286 | |
Ezio Melotti | 22170ed | 2010-11-20 09:57:27 +0000 | [diff] [blame] | 1287 | This method is not called directly by :meth:`assertEqual`, but |
| 1288 | it's used to implement :meth:`assertListEqual` and |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1289 | :meth:`assertTupleEqual`. |
| 1290 | |
| 1291 | .. versionadded:: 3.1 |
| 1292 | |
| 1293 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1294 | .. method:: assertListEqual(first, second, msg=None) |
| 1295 | assertTupleEqual(first, second, msg=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1296 | |
Ezio Melotti | 49ccd51 | 2012-08-29 17:50:42 +0300 | [diff] [blame] | 1297 | Tests that two lists or tuples are equal. If not, an error message is |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1298 | constructed that shows only the differences between the two. An error |
| 1299 | is also raised if either of the parameters are of the wrong type. |
| 1300 | These methods are used by default when comparing lists or tuples with |
| 1301 | :meth:`assertEqual`. |
| 1302 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1303 | .. versionadded:: 3.1 |
| 1304 | |
| 1305 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1306 | .. method:: assertSetEqual(first, second, msg=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1307 | |
| 1308 | Tests that two sets are equal. If not, an error message is constructed |
| 1309 | that lists the differences between the sets. This method is used by |
| 1310 | default when comparing sets or frozensets with :meth:`assertEqual`. |
| 1311 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1312 | Fails if either of *first* or *second* does not have a :meth:`set.difference` |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1313 | method. |
| 1314 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1315 | .. versionadded:: 3.1 |
| 1316 | |
| 1317 | |
Michael Foord | e180d39 | 2011-01-28 19:51:48 +0000 | [diff] [blame] | 1318 | .. method:: assertDictEqual(first, second, msg=None) |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1319 | |
| 1320 | Test that two dictionaries are equal. If not, an error message is |
| 1321 | constructed that shows the differences in the dictionaries. This |
| 1322 | method will be used by default to compare dictionaries in |
| 1323 | calls to :meth:`assertEqual`. |
| 1324 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1325 | .. versionadded:: 3.1 |
| 1326 | |
| 1327 | |
| 1328 | |
Ezio Melotti | 22170ed | 2010-11-20 09:57:27 +0000 | [diff] [blame] | 1329 | .. _other-methods-and-attrs: |
| 1330 | |
Ezio Melotti | 4370b30 | 2010-11-03 20:39:14 +0000 | [diff] [blame] | 1331 | Finally the :class:`TestCase` provides the following methods and attributes: |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1332 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1333 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 1334 | .. method:: fail(msg=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1335 | |
Ezio Melotti | 75b2a5e | 2010-11-20 10:13:45 +0000 | [diff] [blame] | 1336 | Signals a test failure unconditionally, with *msg* or ``None`` for |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1337 | the error message. |
| 1338 | |
| 1339 | |
| 1340 | .. attribute:: failureException |
| 1341 | |
| 1342 | This class attribute gives the exception raised by the test method. If a |
| 1343 | test framework needs to use a specialized exception, possibly to carry |
| 1344 | additional information, it must subclass this exception in order to "play |
| 1345 | fair" with the framework. The initial value of this attribute is |
| 1346 | :exc:`AssertionError`. |
| 1347 | |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1348 | |
| 1349 | .. attribute:: longMessage |
| 1350 | |
Guido van Rossum | 4a45235 | 2016-10-13 14:23:01 -0700 | [diff] [blame] | 1351 | This class attribute determines what happens when a custom failure message |
| 1352 | is passed as the msg argument to an assertXYY call that fails. |
| 1353 | ``True`` is the default value. In this case, the custom message is appended |
| 1354 | to the end of the standard failure message. |
| 1355 | When set to ``False``, the custom message replaces the standard message. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1356 | |
Guido van Rossum | 4a45235 | 2016-10-13 14:23:01 -0700 | [diff] [blame] | 1357 | The class setting can be overridden in individual test methods by assigning |
| 1358 | an instance attribute, self.longMessage, to ``True`` or ``False`` before |
| 1359 | calling the assert methods. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1360 | |
Guido van Rossum | 4a45235 | 2016-10-13 14:23:01 -0700 | [diff] [blame] | 1361 | The class setting gets reset before each test call. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1362 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 1363 | .. versionadded:: 3.1 |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1364 | |
| 1365 | |
Michael Foord | 98b3e76 | 2010-06-05 21:59:55 +0000 | [diff] [blame] | 1366 | .. attribute:: maxDiff |
| 1367 | |
| 1368 | This attribute controls the maximum length of diffs output by assert |
| 1369 | methods that report diffs on failure. It defaults to 80*8 characters. |
| 1370 | Assert methods affected by this attribute are |
| 1371 | :meth:`assertSequenceEqual` (including all the sequence comparison |
| 1372 | methods that delegate to it), :meth:`assertDictEqual` and |
| 1373 | :meth:`assertMultiLineEqual`. |
| 1374 | |
Serhiy Storchaka | ecf41da | 2016-10-19 16:29:26 +0300 | [diff] [blame] | 1375 | Setting ``maxDiff`` to ``None`` means that there is no maximum length of |
Michael Foord | 98b3e76 | 2010-06-05 21:59:55 +0000 | [diff] [blame] | 1376 | diffs. |
| 1377 | |
| 1378 | .. versionadded:: 3.2 |
| 1379 | |
| 1380 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1381 | Testing frameworks can use the following methods to collect information on |
| 1382 | the test: |
| 1383 | |
| 1384 | |
| 1385 | .. method:: countTestCases() |
| 1386 | |
| 1387 | Return the number of tests represented by this test object. For |
| 1388 | :class:`TestCase` instances, this will always be ``1``. |
| 1389 | |
| 1390 | |
| 1391 | .. method:: defaultTestResult() |
| 1392 | |
| 1393 | Return an instance of the test result class that should be used for this |
| 1394 | test case class (if no other result instance is provided to the |
| 1395 | :meth:`run` method). |
| 1396 | |
| 1397 | For :class:`TestCase` instances, this will always be an instance of |
| 1398 | :class:`TestResult`; subclasses of :class:`TestCase` should override this |
| 1399 | as necessary. |
| 1400 | |
| 1401 | |
| 1402 | .. method:: id() |
| 1403 | |
| 1404 | Return a string identifying the specific test case. This is usually the |
| 1405 | full name of the test method, including the module and class name. |
| 1406 | |
| 1407 | |
| 1408 | .. method:: shortDescription() |
| 1409 | |
Ezio Melotti | 75b2a5e | 2010-11-20 10:13:45 +0000 | [diff] [blame] | 1410 | Returns a description of the test, or ``None`` if no description |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1411 | has been provided. The default implementation of this method |
| 1412 | returns the first line of the test method's docstring, if available, |
Ezio Melotti | 75b2a5e | 2010-11-20 10:13:45 +0000 | [diff] [blame] | 1413 | or ``None``. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1414 | |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1415 | .. versionchanged:: 3.1 |
Michael Foord | 34c9462 | 2010-02-10 15:51:42 +0000 | [diff] [blame] | 1416 | In 3.1 this was changed to add the test name to the short description |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1417 | even in the presence of a docstring. This caused compatibility issues |
Michael Foord | 34c9462 | 2010-02-10 15:51:42 +0000 | [diff] [blame] | 1418 | with unittest extensions and adding the test name was moved to the |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 1419 | :class:`TextTestResult` in Python 3.2. |
Benjamin Peterson | 7fe73a1 | 2009-04-04 16:35:46 +0000 | [diff] [blame] | 1420 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1421 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 1422 | .. method:: addCleanup(function, *args, **kwargs) |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1423 | |
| 1424 | Add a function to be called after :meth:`tearDown` to cleanup resources |
| 1425 | used during the test. Functions will be called in reverse order to the |
Serhiy Storchaka | 4ecfa45 | 2016-05-16 09:31:54 +0300 | [diff] [blame] | 1426 | order they are added (:abbr:`LIFO (last-in, first-out)`). They |
| 1427 | are called with any arguments and keyword arguments passed into |
| 1428 | :meth:`addCleanup` when they are added. |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1429 | |
| 1430 | If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called, |
| 1431 | then any cleanup functions added will still be called. |
| 1432 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame] | 1433 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1434 | |
| 1435 | |
| 1436 | .. method:: doCleanups() |
| 1437 | |
Barry Warsaw | 0c9fd63 | 2010-04-12 14:50:57 +0000 | [diff] [blame] | 1438 | This method is called unconditionally after :meth:`tearDown`, or |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1439 | after :meth:`setUp` if :meth:`setUp` raises an exception. |
| 1440 | |
| 1441 | It is responsible for calling all the cleanup functions added by |
| 1442 | :meth:`addCleanup`. If you need cleanup functions to be called |
| 1443 | *prior* to :meth:`tearDown` then you can call :meth:`doCleanups` |
| 1444 | yourself. |
| 1445 | |
| 1446 | :meth:`doCleanups` pops methods off the stack of cleanup |
| 1447 | functions one at a time, so it can be called at any time. |
| 1448 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame] | 1449 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1450 | |
| 1451 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 1452 | .. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1453 | |
| 1454 | This class implements the portion of the :class:`TestCase` interface which |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1455 | allows the test runner to drive the test, but does not provide the methods |
| 1456 | which test code can use to check and report errors. This is used to create |
| 1457 | test cases using legacy test code, allowing it to be integrated into a |
| 1458 | :mod:`unittest`-based test framework. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1459 | |
| 1460 | |
Ezio Melotti | 2baf1a6 | 2010-11-22 12:56:58 +0000 | [diff] [blame] | 1461 | .. _deprecated-aliases: |
| 1462 | |
Ezio Melotti | 8f2e07b | 2010-11-04 19:09:28 +0000 | [diff] [blame] | 1463 | Deprecated aliases |
| 1464 | ################## |
| 1465 | |
| 1466 | For historical reasons, some of the :class:`TestCase` methods had one or more |
| 1467 | aliases that are now deprecated. The following table lists the correct names |
| 1468 | along with their deprecated aliases: |
| 1469 | |
Jim Fasarakis-Hilliard | 74921ed | 2017-06-09 23:27:20 +0300 | [diff] [blame] | 1470 | ============================== ====================== ======================= |
Ezio Melotti | 2baf1a6 | 2010-11-22 12:56:58 +0000 | [diff] [blame] | 1471 | Method Name Deprecated alias Deprecated alias |
Jim Fasarakis-Hilliard | 74921ed | 2017-06-09 23:27:20 +0300 | [diff] [blame] | 1472 | ============================== ====================== ======================= |
Ezio Melotti | 2baf1a6 | 2010-11-22 12:56:58 +0000 | [diff] [blame] | 1473 | :meth:`.assertEqual` failUnlessEqual assertEquals |
| 1474 | :meth:`.assertNotEqual` failIfEqual assertNotEquals |
| 1475 | :meth:`.assertTrue` failUnless assert\_ |
Ezio Melotti | 8f2e07b | 2010-11-04 19:09:28 +0000 | [diff] [blame] | 1476 | :meth:`.assertFalse` failIf |
| 1477 | :meth:`.assertRaises` failUnlessRaises |
Ezio Melotti | 2baf1a6 | 2010-11-22 12:56:58 +0000 | [diff] [blame] | 1478 | :meth:`.assertAlmostEqual` failUnlessAlmostEqual assertAlmostEquals |
| 1479 | :meth:`.assertNotAlmostEqual` failIfAlmostEqual assertNotAlmostEquals |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1480 | :meth:`.assertRegex` assertRegexpMatches |
Jim Fasarakis-Hilliard | 74921ed | 2017-06-09 23:27:20 +0300 | [diff] [blame] | 1481 | :meth:`.assertNotRegex` assertNotRegexpMatches |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1482 | :meth:`.assertRaisesRegex` assertRaisesRegexp |
Jim Fasarakis-Hilliard | 74921ed | 2017-06-09 23:27:20 +0300 | [diff] [blame] | 1483 | ============================== ====================== ======================= |
Ezio Melotti | 8f2e07b | 2010-11-04 19:09:28 +0000 | [diff] [blame] | 1484 | |
Ezio Melotti | 361467e | 2011-04-03 17:37:58 +0300 | [diff] [blame] | 1485 | .. deprecated:: 3.1 |
Ezio Melotti | 2baf1a6 | 2010-11-22 12:56:58 +0000 | [diff] [blame] | 1486 | the fail* aliases listed in the second column. |
| 1487 | .. deprecated:: 3.2 |
| 1488 | the assert* aliases listed in the third column. |
Ezio Melotti | ed3a7d2 | 2010-12-01 02:32:32 +0000 | [diff] [blame] | 1489 | .. deprecated:: 3.2 |
| 1490 | ``assertRegexpMatches`` and ``assertRaisesRegexp`` have been renamed to |
Jim Fasarakis-Hilliard | 74921ed | 2017-06-09 23:27:20 +0300 | [diff] [blame] | 1491 | :meth:`.assertRegex` and :meth:`.assertRaisesRegex`. |
| 1492 | .. deprecated:: 3.5 |
| 1493 | the ``assertNotRegexpMatches`` name in favor of :meth:`.assertNotRegex`. |
Ezio Melotti | 8f2e07b | 2010-11-04 19:09:28 +0000 | [diff] [blame] | 1494 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1495 | .. _testsuite-objects: |
| 1496 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1497 | Grouping tests |
| 1498 | ~~~~~~~~~~~~~~ |
| 1499 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 1500 | .. class:: TestSuite(tests=()) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1501 | |
Martin Panter | 37f183d | 2017-01-18 12:06:38 +0000 | [diff] [blame] | 1502 | This class represents an aggregation of individual test cases and test suites. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1503 | The class presents the interface needed by the test runner to allow it to be run |
| 1504 | as any other test case. Running a :class:`TestSuite` instance is the same as |
| 1505 | iterating over the suite, running each test individually. |
| 1506 | |
| 1507 | If *tests* is given, it must be an iterable of individual test cases or other |
| 1508 | test suites that will be used to build the suite initially. Additional methods |
| 1509 | are provided to add test cases and suites to the collection later on. |
| 1510 | |
Benjamin Peterson | 14a3dd7 | 2009-05-25 00:51:58 +0000 | [diff] [blame] | 1511 | :class:`TestSuite` objects behave much like :class:`TestCase` objects, except |
| 1512 | they do not actually implement a test. Instead, they are used to aggregate |
| 1513 | tests into groups of tests that should be run together. Some additional |
| 1514 | methods are available to add tests to :class:`TestSuite` instances: |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1515 | |
| 1516 | |
| 1517 | .. method:: TestSuite.addTest(test) |
| 1518 | |
| 1519 | Add a :class:`TestCase` or :class:`TestSuite` to the suite. |
| 1520 | |
| 1521 | |
| 1522 | .. method:: TestSuite.addTests(tests) |
| 1523 | |
| 1524 | Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite` |
| 1525 | instances to this test suite. |
| 1526 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1527 | This is equivalent to iterating over *tests*, calling :meth:`addTest` for |
| 1528 | each element. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1529 | |
| 1530 | :class:`TestSuite` shares the following methods with :class:`TestCase`: |
| 1531 | |
| 1532 | |
| 1533 | .. method:: run(result) |
| 1534 | |
| 1535 | Run the tests associated with this suite, collecting the result into the |
| 1536 | test result object passed as *result*. Note that unlike |
| 1537 | :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to |
| 1538 | be passed in. |
| 1539 | |
| 1540 | |
| 1541 | .. method:: debug() |
| 1542 | |
| 1543 | Run the tests associated with this suite without collecting the |
| 1544 | result. This allows exceptions raised by the test to be propagated to the |
| 1545 | caller and can be used to support running tests under a debugger. |
| 1546 | |
| 1547 | |
| 1548 | .. method:: countTestCases() |
| 1549 | |
| 1550 | Return the number of tests represented by this test object, including all |
| 1551 | individual tests and sub-suites. |
| 1552 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1553 | |
| 1554 | .. method:: __iter__() |
| 1555 | |
| 1556 | Tests grouped by a :class:`TestSuite` are always accessed by iteration. |
| 1557 | Subclasses can lazily provide tests by overriding :meth:`__iter__`. Note |
Andrew Svetlov | eb97368 | 2013-08-28 21:28:38 +0300 | [diff] [blame] | 1558 | that this method may be called several times on a single suite (for |
| 1559 | example when counting tests or comparing for equality) so the tests |
| 1560 | returned by repeated iterations before :meth:`TestSuite.run` must be the |
| 1561 | same for each call iteration. After :meth:`TestSuite.run`, callers should |
| 1562 | not rely on the tests returned by this method unless the caller uses a |
| 1563 | subclass that overrides :meth:`TestSuite._removeTestAtIndex` to preserve |
| 1564 | test references. |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1565 | |
Georg Brandl | 853947a | 2010-01-31 18:53:23 +0000 | [diff] [blame] | 1566 | .. versionchanged:: 3.2 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1567 | In earlier versions the :class:`TestSuite` accessed tests directly rather |
| 1568 | than through iteration, so overriding :meth:`__iter__` wasn't sufficient |
| 1569 | for providing tests. |
| 1570 | |
Andrew Svetlov | eb97368 | 2013-08-28 21:28:38 +0300 | [diff] [blame] | 1571 | .. versionchanged:: 3.4 |
| 1572 | In earlier versions the :class:`TestSuite` held references to each |
| 1573 | :class:`TestCase` after :meth:`TestSuite.run`. Subclasses can restore |
| 1574 | that behavior by overriding :meth:`TestSuite._removeTestAtIndex`. |
| 1575 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1576 | In the typical usage of a :class:`TestSuite` object, the :meth:`run` method |
| 1577 | is invoked by a :class:`TestRunner` rather than by the end-user test harness. |
| 1578 | |
| 1579 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1580 | Loading and running tests |
| 1581 | ~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1582 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1583 | .. class:: TestLoader() |
| 1584 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1585 | The :class:`TestLoader` class is used to create test suites from classes and |
| 1586 | modules. Normally, there is no need to create an instance of this class; the |
| 1587 | :mod:`unittest` module provides an instance that can be shared as |
Ezio Melotti | b8e336b | 2012-04-29 10:52:18 +0300 | [diff] [blame] | 1588 | :data:`unittest.defaultTestLoader`. Using a subclass or instance, however, |
| 1589 | allows customization of some configurable properties. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1590 | |
Robert Collins | f920c21 | 2014-10-20 13:24:05 +1300 | [diff] [blame] | 1591 | :class:`TestLoader` objects have the following attributes: |
| 1592 | |
| 1593 | |
| 1594 | .. attribute:: errors |
| 1595 | |
| 1596 | A list of the non-fatal errors encountered while loading tests. Not reset |
| 1597 | by the loader at any point. Fatal errors are signalled by the relevant |
| 1598 | a method raising an exception to the caller. Non-fatal errors are also |
| 1599 | indicated by a synthetic test that will raise the original error when |
| 1600 | run. |
| 1601 | |
| 1602 | .. versionadded:: 3.5 |
| 1603 | |
| 1604 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1605 | :class:`TestLoader` objects have the following methods: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1606 | |
Ezio Melotti | 9c02c2f | 2010-11-03 20:45:31 +0000 | [diff] [blame] | 1607 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1608 | .. method:: loadTestsFromTestCase(testCaseClass) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1609 | |
Martin Panter | 37f183d | 2017-01-18 12:06:38 +0000 | [diff] [blame] | 1610 | Return a suite of all test cases contained in the :class:`TestCase`\ -derived |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1611 | :class:`testCaseClass`. |
| 1612 | |
Robert Collins | e02f6c2 | 2015-07-23 06:37:26 +1200 | [diff] [blame] | 1613 | A test case instance is created for each method named by |
| 1614 | :meth:`getTestCaseNames`. By default these are the method names |
| 1615 | beginning with ``test``. If :meth:`getTestCaseNames` returns no |
| 1616 | methods, but the :meth:`runTest` method is implemented, a single test |
| 1617 | case is created for that method instead. |
| 1618 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1619 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 1620 | .. method:: loadTestsFromModule(module, pattern=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1621 | |
Martin Panter | 37f183d | 2017-01-18 12:06:38 +0000 | [diff] [blame] | 1622 | Return a suite of all test cases contained in the given module. This |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1623 | method searches *module* for classes derived from :class:`TestCase` and |
| 1624 | creates an instance of the class for each test method defined for the |
| 1625 | class. |
| 1626 | |
Georg Brandl | e720c0a | 2009-04-27 16:20:50 +0000 | [diff] [blame] | 1627 | .. note:: |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1628 | |
| 1629 | While using a hierarchy of :class:`TestCase`\ -derived classes can be |
| 1630 | convenient in sharing fixtures and helper functions, defining test |
| 1631 | methods on base classes that are not intended to be instantiated |
| 1632 | directly does not play well with this method. Doing so, however, can |
| 1633 | be useful when the fixtures are different and defined in subclasses. |
| 1634 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1635 | If a module provides a ``load_tests`` function it will be called to |
| 1636 | load the tests. This allows modules to customize test loading. |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 1637 | This is the `load_tests protocol`_. The *pattern* argument is passed as |
| 1638 | the third argument to ``load_tests``. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1639 | |
Georg Brandl | 853947a | 2010-01-31 18:53:23 +0000 | [diff] [blame] | 1640 | .. versionchanged:: 3.2 |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1641 | Support for ``load_tests`` added. |
| 1642 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 1643 | .. versionchanged:: 3.5 |
| 1644 | The undocumented and unofficial *use_load_tests* default argument is |
| 1645 | deprecated and ignored, although it is still accepted for backward |
| 1646 | compatibility. The method also now accepts a keyword-only argument |
| 1647 | *pattern* which is passed to ``load_tests`` as the third argument. |
| 1648 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1649 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 1650 | .. method:: loadTestsFromName(name, module=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1651 | |
Martin Panter | 37f183d | 2017-01-18 12:06:38 +0000 | [diff] [blame] | 1652 | Return a suite of all test cases given a string specifier. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1653 | |
| 1654 | The specifier *name* is a "dotted name" that may resolve either to a |
| 1655 | module, a test case class, a test method within a test case class, a |
| 1656 | :class:`TestSuite` instance, or a callable object which returns a |
| 1657 | :class:`TestCase` or :class:`TestSuite` instance. These checks are |
| 1658 | applied in the order listed here; that is, a method on a possible test |
| 1659 | case class will be picked up as "a test method within a test case class", |
| 1660 | rather than "a callable object". |
| 1661 | |
| 1662 | For example, if you have a module :mod:`SampleTests` containing a |
| 1663 | :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test |
| 1664 | methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1665 | specifier ``'SampleTests.SampleTestCase'`` would cause this method to |
| 1666 | return a suite which will run all three test methods. Using the specifier |
| 1667 | ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test |
| 1668 | suite which will run only the :meth:`test_two` test method. The specifier |
| 1669 | can refer to modules and packages which have not been imported; they will |
| 1670 | be imported as a side-effect. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1671 | |
| 1672 | The method optionally resolves *name* relative to the given *module*. |
| 1673 | |
Martin Panter | 536d70e | 2017-01-14 08:23:08 +0000 | [diff] [blame] | 1674 | .. versionchanged:: 3.5 |
| 1675 | If an :exc:`ImportError` or :exc:`AttributeError` occurs while traversing |
| 1676 | *name* then a synthetic test that raises that error when run will be |
| 1677 | returned. These errors are included in the errors accumulated by |
| 1678 | self.errors. |
Robert Collins | 659dd62 | 2014-10-30 08:27:27 +1300 | [diff] [blame] | 1679 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1680 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 1681 | .. method:: loadTestsFromNames(names, module=None) |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1682 | |
| 1683 | Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather |
| 1684 | than a single name. The return value is a test suite which supports all |
| 1685 | the tests defined for each name. |
| 1686 | |
| 1687 | |
| 1688 | .. method:: getTestCaseNames(testCaseClass) |
| 1689 | |
| 1690 | Return a sorted sequence of method names found within *testCaseClass*; |
| 1691 | this should be a subclass of :class:`TestCase`. |
| 1692 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1693 | |
| 1694 | .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None) |
| 1695 | |
Larry Hastings | 3732ed2 | 2014-03-15 21:13:56 -0700 | [diff] [blame] | 1696 | Find all the test modules by recursing into subdirectories from the |
| 1697 | specified start directory, and return a TestSuite object containing them. |
| 1698 | Only test files that match *pattern* will be loaded. (Using shell style |
| 1699 | pattern matching.) Only module names that are importable (i.e. are valid |
| 1700 | Python identifiers) will be loaded. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1701 | |
| 1702 | All test modules must be importable from the top level of the project. If |
| 1703 | the start directory is not the top level directory then the top level |
| 1704 | directory must be specified separately. |
| 1705 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 1706 | If importing a module fails, for example due to a syntax error, then |
| 1707 | this will be recorded as a single error and discovery will continue. If |
| 1708 | the import failure is due to :exc:`SkipTest` being raised, it will be |
| 1709 | recorded as a skip instead of an error. |
Benjamin Peterson | 4ac9ce4 | 2009-10-04 14:49:41 +0000 | [diff] [blame] | 1710 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 1711 | If a package (a directory containing a file named :file:`__init__.py`) is |
| 1712 | found, the package will be checked for a ``load_tests`` function. If this |
Robert Collins | bf2bda3 | 2014-11-05 03:09:01 +1300 | [diff] [blame] | 1713 | exists then it will be called |
| 1714 | ``package.load_tests(loader, tests, pattern)``. Test discovery takes care |
| 1715 | to ensure that a package is only checked for tests once during an |
| 1716 | invocation, even if the load_tests function itself calls |
| 1717 | ``loader.discover``. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1718 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 1719 | If ``load_tests`` exists then discovery does *not* recurse into the |
| 1720 | package, ``load_tests`` is responsible for loading all tests in the |
| 1721 | package. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1722 | |
| 1723 | The pattern is deliberately not stored as a loader attribute so that |
| 1724 | packages can continue discovery themselves. *top_level_dir* is stored so |
| 1725 | ``load_tests`` does not need to pass this argument in to |
| 1726 | ``loader.discover()``. |
| 1727 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 1728 | *start_dir* can be a dotted module name as well as a directory. |
| 1729 | |
Georg Brandl | 853947a | 2010-01-31 18:53:23 +0000 | [diff] [blame] | 1730 | .. versionadded:: 3.2 |
| 1731 | |
Ezio Melotti | eae2b38 | 2013-03-01 14:47:50 +0200 | [diff] [blame] | 1732 | .. versionchanged:: 3.4 |
Ezio Melotti | 67ddcca | 2013-03-27 20:13:59 +0200 | [diff] [blame] | 1733 | Modules that raise :exc:`SkipTest` on import are recorded as skips, |
Larry Hastings | 3732ed2 | 2014-03-15 21:13:56 -0700 | [diff] [blame] | 1734 | not errors. |
| 1735 | Discovery works for :term:`namespace packages <namespace package>`. |
| 1736 | Paths are sorted before being imported so that execution order is |
| 1737 | the same even if the underlying file system's ordering is not |
| 1738 | dependent on file name. |
Michael Foord | 80cbc9e | 2013-03-18 17:50:12 -0700 | [diff] [blame] | 1739 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 1740 | .. versionchanged:: 3.5 |
| 1741 | Found packages are now checked for ``load_tests`` regardless of |
| 1742 | whether their path matches *pattern*, because it is impossible for |
| 1743 | a package name to match the default pattern. |
| 1744 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1745 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1746 | The following attributes of a :class:`TestLoader` can be configured either by |
| 1747 | subclassing or assignment on an instance: |
| 1748 | |
| 1749 | |
| 1750 | .. attribute:: testMethodPrefix |
| 1751 | |
| 1752 | String giving the prefix of method names which will be interpreted as test |
| 1753 | methods. The default value is ``'test'``. |
| 1754 | |
| 1755 | This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` |
| 1756 | methods. |
| 1757 | |
| 1758 | |
| 1759 | .. attribute:: sortTestMethodsUsing |
| 1760 | |
| 1761 | Function to be used to compare method names when sorting them in |
| 1762 | :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. |
| 1763 | |
| 1764 | |
| 1765 | .. attribute:: suiteClass |
| 1766 | |
| 1767 | Callable object that constructs a test suite from a list of tests. No |
| 1768 | methods on the resulting object are needed. The default value is the |
| 1769 | :class:`TestSuite` class. |
| 1770 | |
| 1771 | This affects all the :meth:`loadTestsFrom\*` methods. |
| 1772 | |
Jonas Haag | 5b48dc6 | 2017-11-25 16:23:52 +0100 | [diff] [blame] | 1773 | .. attribute:: testNamePatterns |
| 1774 | |
| 1775 | List of Unix shell-style wildcard test name patterns that test methods |
| 1776 | have to match to be included in test suites (see ``-v`` option). |
| 1777 | |
| 1778 | If this attribute is not ``None`` (the default), all test methods to be |
| 1779 | included in test suites must match one of the patterns in this list. |
| 1780 | Note that matches are always performed using :meth:`fnmatch.fnmatchcase`, |
| 1781 | so unlike patterns passed to the ``-v`` option, simple substring patterns |
| 1782 | will have to be converted using ``*`` wildcards. |
| 1783 | |
| 1784 | This affects all the :meth:`loadTestsFrom\*` methods. |
| 1785 | |
| 1786 | .. versionadded:: 3.7 |
| 1787 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1788 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1789 | .. class:: TestResult |
| 1790 | |
| 1791 | This class is used to compile information about which tests have succeeded |
| 1792 | and which have failed. |
| 1793 | |
| 1794 | A :class:`TestResult` object stores the results of a set of tests. The |
| 1795 | :class:`TestCase` and :class:`TestSuite` classes ensure that results are |
| 1796 | properly recorded; test authors do not need to worry about recording the |
| 1797 | outcome of tests. |
| 1798 | |
| 1799 | Testing frameworks built on top of :mod:`unittest` may want access to the |
| 1800 | :class:`TestResult` object generated by running a set of tests for reporting |
| 1801 | purposes; a :class:`TestResult` instance is returned by the |
| 1802 | :meth:`TestRunner.run` method for this purpose. |
| 1803 | |
| 1804 | :class:`TestResult` instances have the following attributes that will be of |
| 1805 | interest when inspecting the results of running a set of tests: |
| 1806 | |
| 1807 | |
| 1808 | .. attribute:: errors |
| 1809 | |
| 1810 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1811 | holding formatted tracebacks. Each tuple represents a test which raised an |
| 1812 | unexpected exception. |
| 1813 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1814 | .. attribute:: failures |
| 1815 | |
| 1816 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1817 | holding formatted tracebacks. Each tuple represents a test where a failure |
Ezio Melotti | e220236 | 2013-09-07 15:19:30 +0300 | [diff] [blame] | 1818 | was explicitly signalled using the :meth:`TestCase.assert\*` methods. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1819 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1820 | .. attribute:: skipped |
| 1821 | |
| 1822 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1823 | holding the reason for skipping the test. |
| 1824 | |
Benjamin Peterson | 70e32c8 | 2009-03-24 01:00:11 +0000 | [diff] [blame] | 1825 | .. versionadded:: 3.1 |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1826 | |
| 1827 | .. attribute:: expectedFailures |
| 1828 | |
Georg Brandl | 6faee4e | 2010-09-21 14:48:28 +0000 | [diff] [blame] | 1829 | A list containing 2-tuples of :class:`TestCase` instances and strings |
| 1830 | holding formatted tracebacks. Each tuple represents an expected failure |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1831 | of the test case. |
| 1832 | |
| 1833 | .. attribute:: unexpectedSuccesses |
| 1834 | |
| 1835 | A list containing :class:`TestCase` instances that were marked as expected |
| 1836 | failures, but succeeded. |
| 1837 | |
| 1838 | .. attribute:: shouldStop |
| 1839 | |
| 1840 | Set to ``True`` when the execution of tests should stop by :meth:`stop`. |
| 1841 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1842 | .. attribute:: testsRun |
| 1843 | |
| 1844 | The total number of tests run so far. |
| 1845 | |
Georg Brandl | 1203720 | 2010-12-02 22:35:25 +0000 | [diff] [blame] | 1846 | .. attribute:: buffer |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 1847 | |
| 1848 | If set to true, ``sys.stdout`` and ``sys.stderr`` will be buffered in between |
| 1849 | :meth:`startTest` and :meth:`stopTest` being called. Collected output will |
| 1850 | only be echoed onto the real ``sys.stdout`` and ``sys.stderr`` if the test |
| 1851 | fails or errors. Any output is also attached to the failure / error message. |
| 1852 | |
Ezio Melotti | 7afd3f5 | 2010-04-20 09:32:54 +0000 | [diff] [blame] | 1853 | .. versionadded:: 3.2 |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 1854 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 1855 | .. attribute:: failfast |
| 1856 | |
| 1857 | If set to true :meth:`stop` will be called on the first failure or error, |
| 1858 | halting the test run. |
| 1859 | |
Ezio Melotti | 7afd3f5 | 2010-04-20 09:32:54 +0000 | [diff] [blame] | 1860 | .. versionadded:: 3.2 |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 1861 | |
Robert Collins | f0c819a | 2015-03-06 13:46:35 +1300 | [diff] [blame] | 1862 | .. attribute:: tb_locals |
| 1863 | |
| 1864 | If set to true then local variables will be shown in tracebacks. |
| 1865 | |
| 1866 | .. versionadded:: 3.5 |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 1867 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1868 | .. method:: wasSuccessful() |
| 1869 | |
Ezio Melotti | 75b2a5e | 2010-11-20 10:13:45 +0000 | [diff] [blame] | 1870 | Return ``True`` if all tests run so far have passed, otherwise returns |
| 1871 | ``False``. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1872 | |
Gregory P. Smith | 5a6d4bf | 2014-01-20 01:11:18 -0800 | [diff] [blame] | 1873 | .. versionchanged:: 3.4 |
| 1874 | Returns ``False`` if there were any :attr:`unexpectedSuccesses` |
| 1875 | from tests marked with the :func:`expectedFailure` decorator. |
| 1876 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1877 | .. method:: stop() |
| 1878 | |
| 1879 | This method can be called to signal that the set of tests being run should |
Ezio Melotti | 75b2a5e | 2010-11-20 10:13:45 +0000 | [diff] [blame] | 1880 | be aborted by setting the :attr:`shouldStop` attribute to ``True``. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1881 | :class:`TestRunner` objects should respect this flag and return without |
| 1882 | running any additional tests. |
| 1883 | |
| 1884 | For example, this feature is used by the :class:`TextTestRunner` class to |
| 1885 | stop the test framework when the user signals an interrupt from the |
| 1886 | keyboard. Interactive tools which provide :class:`TestRunner` |
| 1887 | implementations can use this in a similar manner. |
| 1888 | |
| 1889 | The following methods of the :class:`TestResult` class are used to maintain |
| 1890 | the internal data structures, and may be extended in subclasses to support |
| 1891 | additional reporting requirements. This is particularly useful in building |
| 1892 | tools which support interactive reporting while tests are being run. |
| 1893 | |
| 1894 | |
| 1895 | .. method:: startTest(test) |
| 1896 | |
| 1897 | Called when the test case *test* is about to be run. |
| 1898 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1899 | .. method:: stopTest(test) |
| 1900 | |
| 1901 | Called after the test case *test* has been executed, regardless of the |
| 1902 | outcome. |
| 1903 | |
Terry Jan Reedy | f98021c | 2014-04-11 14:11:11 -0400 | [diff] [blame] | 1904 | .. method:: startTestRun() |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1905 | |
| 1906 | Called once before any tests are executed. |
| 1907 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame] | 1908 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1909 | |
| 1910 | |
Terry Jan Reedy | f98021c | 2014-04-11 14:11:11 -0400 | [diff] [blame] | 1911 | .. method:: stopTestRun() |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1912 | |
Ezio Melotti | 176d6c4 | 2010-01-27 20:58:07 +0000 | [diff] [blame] | 1913 | Called once after all tests are executed. |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1914 | |
Ezio Melotti | 2d1e88a | 2011-03-10 12:16:35 +0200 | [diff] [blame] | 1915 | .. versionadded:: 3.1 |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1916 | |
| 1917 | |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1918 | .. method:: addError(test, err) |
| 1919 | |
Ezio Melotti | e64a91a | 2013-09-07 15:23:36 +0300 | [diff] [blame] | 1920 | Called when the test case *test* raises an unexpected exception. *err* is a |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1921 | tuple of the form returned by :func:`sys.exc_info`: ``(type, value, |
| 1922 | traceback)``. |
| 1923 | |
| 1924 | The default implementation appends a tuple ``(test, formatted_err)`` to |
| 1925 | the instance's :attr:`errors` attribute, where *formatted_err* is a |
| 1926 | formatted traceback derived from *err*. |
| 1927 | |
| 1928 | |
| 1929 | .. method:: addFailure(test, err) |
| 1930 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 1931 | Called when the test case *test* signals a failure. *err* is a tuple of |
| 1932 | the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``. |
Benjamin Peterson | 52baa29 | 2009-03-24 00:56:30 +0000 | [diff] [blame] | 1933 | |
| 1934 | The default implementation appends a tuple ``(test, formatted_err)`` to |
| 1935 | the instance's :attr:`failures` attribute, where *formatted_err* is a |
| 1936 | formatted traceback derived from *err*. |
| 1937 | |
| 1938 | |
| 1939 | .. method:: addSuccess(test) |
| 1940 | |
| 1941 | Called when the test case *test* succeeds. |
| 1942 | |
| 1943 | The default implementation does nothing. |
| 1944 | |
| 1945 | |
| 1946 | .. method:: addSkip(test, reason) |
| 1947 | |
| 1948 | Called when the test case *test* is skipped. *reason* is the reason the |
| 1949 | test gave for skipping. |
| 1950 | |
| 1951 | The default implementation appends a tuple ``(test, reason)`` to the |
| 1952 | instance's :attr:`skipped` attribute. |
| 1953 | |
| 1954 | |
| 1955 | .. method:: addExpectedFailure(test, err) |
| 1956 | |
| 1957 | Called when the test case *test* fails, but was marked with the |
| 1958 | :func:`expectedFailure` decorator. |
| 1959 | |
| 1960 | The default implementation appends a tuple ``(test, formatted_err)`` to |
| 1961 | the instance's :attr:`expectedFailures` attribute, where *formatted_err* |
| 1962 | is a formatted traceback derived from *err*. |
| 1963 | |
| 1964 | |
| 1965 | .. method:: addUnexpectedSuccess(test) |
| 1966 | |
| 1967 | Called when the test case *test* was marked with the |
| 1968 | :func:`expectedFailure` decorator, but succeeded. |
| 1969 | |
| 1970 | The default implementation appends the test to the instance's |
| 1971 | :attr:`unexpectedSuccesses` attribute. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1972 | |
Georg Brandl | 67b21b7 | 2010-08-17 15:07:14 +0000 | [diff] [blame] | 1973 | |
Antoine Pitrou | c9b3ef2 | 2013-03-20 20:16:47 +0100 | [diff] [blame] | 1974 | .. method:: addSubTest(test, subtest, outcome) |
| 1975 | |
| 1976 | Called when a subtest finishes. *test* is the test case |
| 1977 | corresponding to the test method. *subtest* is a custom |
| 1978 | :class:`TestCase` instance describing the subtest. |
| 1979 | |
| 1980 | If *outcome* is :const:`None`, the subtest succeeded. Otherwise, |
| 1981 | it failed with an exception where *outcome* is a tuple of the form |
| 1982 | returned by :func:`sys.exc_info`: ``(type, value, traceback)``. |
| 1983 | |
| 1984 | The default implementation does nothing when the outcome is a |
| 1985 | success, and records subtest failures as normal failures. |
| 1986 | |
| 1987 | .. versionadded:: 3.4 |
| 1988 | |
| 1989 | |
Michael Foord | 34c9462 | 2010-02-10 15:51:42 +0000 | [diff] [blame] | 1990 | .. class:: TextTestResult(stream, descriptions, verbosity) |
| 1991 | |
Georg Brandl | 67b21b7 | 2010-08-17 15:07:14 +0000 | [diff] [blame] | 1992 | A concrete implementation of :class:`TestResult` used by the |
| 1993 | :class:`TextTestRunner`. |
Michael Foord | 34c9462 | 2010-02-10 15:51:42 +0000 | [diff] [blame] | 1994 | |
Georg Brandl | 67b21b7 | 2010-08-17 15:07:14 +0000 | [diff] [blame] | 1995 | .. versionadded:: 3.2 |
| 1996 | This class was previously named ``_TextTestResult``. The old name still |
| 1997 | exists as an alias but is deprecated. |
| 1998 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1999 | |
| 2000 | .. data:: defaultTestLoader |
| 2001 | |
| 2002 | Instance of the :class:`TestLoader` class intended to be shared. If no |
| 2003 | customization of the :class:`TestLoader` is needed, this instance can be used |
| 2004 | instead of repeatedly creating new instances. |
| 2005 | |
| 2006 | |
Ezio Melotti | 9c939bc | 2013-05-07 09:46:30 +0300 | [diff] [blame] | 2007 | .. class:: TextTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, \ |
Robert Collins | f0c819a | 2015-03-06 13:46:35 +1300 | [diff] [blame] | 2008 | buffer=False, resultclass=None, warnings=None, *, tb_locals=False) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2009 | |
Michael Foord | d218e95 | 2011-01-03 12:55:11 +0000 | [diff] [blame] | 2010 | A basic test runner implementation that outputs results to a stream. If *stream* |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 2011 | is ``None``, the default, :data:`sys.stderr` is used as the output stream. This class |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2012 | has a few configurable parameters, but is essentially very simple. Graphical |
Robert Collins | f0c819a | 2015-03-06 13:46:35 +1300 | [diff] [blame] | 2013 | applications which run test suites should provide alternate implementations. Such |
| 2014 | implementations should accept ``**kwargs`` as the interface to construct runners |
| 2015 | changes when features are added to unittest. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2016 | |
Ezio Melotti | 6090187 | 2010-12-01 00:56:10 +0000 | [diff] [blame] | 2017 | By default this runner shows :exc:`DeprecationWarning`, |
Senthil Kumaran | 409ea5d | 2014-02-08 14:28:03 -0800 | [diff] [blame] | 2018 | :exc:`PendingDeprecationWarning`, :exc:`ResourceWarning` and |
Larry Hastings | ad88d7a | 2014-02-10 04:26:10 -0800 | [diff] [blame] | 2019 | :exc:`ImportWarning` even if they are :ref:`ignored by default |
| 2020 | <warning-ignored>`. Deprecation warnings caused by :ref:`deprecated unittest |
| 2021 | methods <deprecated-aliases>` are also special-cased and, when the warning |
| 2022 | filters are ``'default'`` or ``'always'``, they will appear only once |
| 2023 | per-module, in order to avoid too many warning messages. This behavior can |
Martin Panter | b8c5f54 | 2016-10-30 04:20:23 +0000 | [diff] [blame] | 2024 | be overridden using Python's :option:`!-Wd` or :option:`!-Wa` options |
| 2025 | (see :ref:`Warning control <using-on-warnings>`) and leaving |
Larry Hastings | ad88d7a | 2014-02-10 04:26:10 -0800 | [diff] [blame] | 2026 | *warnings* to ``None``. |
Ezio Melotti | 6090187 | 2010-12-01 00:56:10 +0000 | [diff] [blame] | 2027 | |
Michael Foord | d218e95 | 2011-01-03 12:55:11 +0000 | [diff] [blame] | 2028 | .. versionchanged:: 3.2 |
| 2029 | Added the ``warnings`` argument. |
| 2030 | |
| 2031 | .. versionchanged:: 3.2 |
Éric Araujo | 941afed | 2011-09-01 02:47:34 +0200 | [diff] [blame] | 2032 | The default stream is set to :data:`sys.stderr` at instantiation time rather |
Michael Foord | d218e95 | 2011-01-03 12:55:11 +0000 | [diff] [blame] | 2033 | than import time. |
| 2034 | |
Robert Collins | f0c819a | 2015-03-06 13:46:35 +1300 | [diff] [blame] | 2035 | .. versionchanged:: 3.5 |
| 2036 | Added the tb_locals parameter. |
| 2037 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 2038 | .. method:: _makeResult() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2039 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 2040 | This method returns the instance of ``TestResult`` used by :meth:`run`. |
| 2041 | It is not intended to be called directly, but can be overridden in |
| 2042 | subclasses to provide a custom ``TestResult``. |
| 2043 | |
Michael Foord | 34c9462 | 2010-02-10 15:51:42 +0000 | [diff] [blame] | 2044 | ``_makeResult()`` instantiates the class or callable passed in the |
| 2045 | ``TextTestRunner`` constructor as the ``resultclass`` argument. It |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2046 | defaults to :class:`TextTestResult` if no ``resultclass`` is provided. |
Michael Foord | 34c9462 | 2010-02-10 15:51:42 +0000 | [diff] [blame] | 2047 | The result class is instantiated with the following arguments:: |
| 2048 | |
| 2049 | stream, descriptions, verbosity |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 2050 | |
Michael Foord | 4d1639f | 2013-12-29 23:38:55 +0000 | [diff] [blame] | 2051 | .. method:: run(test) |
| 2052 | |
| 2053 | This method is the main public interface to the `TextTestRunner`. This |
| 2054 | method takes a :class:`TestSuite` or :class:`TestCase` instance. A |
| 2055 | :class:`TestResult` is created by calling |
| 2056 | :func:`_makeResult` and the test(s) are run and the |
| 2057 | results printed to stdout. |
| 2058 | |
Ezio Melotti | 6090187 | 2010-12-01 00:56:10 +0000 | [diff] [blame] | 2059 | |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 2060 | .. function:: main(module='__main__', defaultTest=None, argv=None, testRunner=None, \ |
Ezio Melotti | 40dcb1d | 2011-03-10 13:46:50 +0200 | [diff] [blame] | 2061 | testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, \ |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 2062 | failfast=None, catchbreak=None, buffer=None, warnings=None) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2063 | |
Ezio Melotti | 3d6d7a5 | 2012-04-30 19:10:28 +0300 | [diff] [blame] | 2064 | A command-line program that loads a set of tests from *module* and runs them; |
| 2065 | this is primarily for making test modules conveniently executable. |
| 2066 | The simplest use for this function is to include the following line at the |
| 2067 | end of a test script:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2068 | |
| 2069 | if __name__ == '__main__': |
| 2070 | unittest.main() |
| 2071 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 2072 | You can run tests with more detailed information by passing in the verbosity |
| 2073 | argument:: |
| 2074 | |
| 2075 | if __name__ == '__main__': |
| 2076 | unittest.main(verbosity=2) |
| 2077 | |
R David Murray | 6e731b0 | 2014-01-02 13:43:02 -0500 | [diff] [blame] | 2078 | The *defaultTest* argument is either the name of a single test or an |
| 2079 | iterable of test names to run if no test names are specified via *argv*. If |
| 2080 | not specified or ``None`` and no test names are provided via *argv*, all |
| 2081 | tests found in *module* are run. |
R David Murray | 12e930f | 2014-01-02 13:37:26 -0500 | [diff] [blame] | 2082 | |
Ezio Melotti | 3d6d7a5 | 2012-04-30 19:10:28 +0300 | [diff] [blame] | 2083 | The *argv* argument can be a list of options passed to the program, with the |
| 2084 | first element being the program name. If not specified or ``None``, |
| 2085 | the values of :data:`sys.argv` are used. |
| 2086 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2087 | The *testRunner* argument can either be a test runner class or an already |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 2088 | created instance of it. By default ``main`` calls :func:`sys.exit` with |
| 2089 | an exit code indicating success or failure of the tests run. |
| 2090 | |
Ezio Melotti | 3d6d7a5 | 2012-04-30 19:10:28 +0300 | [diff] [blame] | 2091 | The *testLoader* argument has to be a :class:`TestLoader` instance, |
| 2092 | and defaults to :data:`defaultTestLoader`. |
| 2093 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 2094 | ``main`` supports being used from the interactive interpreter by passing in the |
| 2095 | argument ``exit=False``. This displays the result on standard output without |
| 2096 | calling :func:`sys.exit`:: |
| 2097 | |
| 2098 | >>> from unittest import main |
| 2099 | >>> main(module='test_module', exit=False) |
| 2100 | |
Ezio Melotti | 3d6d7a5 | 2012-04-30 19:10:28 +0300 | [diff] [blame] | 2101 | The *failfast*, *catchbreak* and *buffer* parameters have the same |
Éric Araujo | 76338ec | 2010-11-26 23:46:18 +0000 | [diff] [blame] | 2102 | effect as the same-name `command-line options`_. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2103 | |
Zachary Ware | f0a71cf | 2016-08-30 00:16:13 -0500 | [diff] [blame] | 2104 | The *warnings* argument specifies the :ref:`warning filter <warning-filter>` |
Ezio Melotti | 6090187 | 2010-12-01 00:56:10 +0000 | [diff] [blame] | 2105 | that should be used while running the tests. If it's not specified, it will |
Martin Panter | b8c5f54 | 2016-10-30 04:20:23 +0000 | [diff] [blame] | 2106 | remain ``None`` if a :option:`!-W` option is passed to :program:`python` |
| 2107 | (see :ref:`Warning control <using-on-warnings>`), |
Ezio Melotti | 6090187 | 2010-12-01 00:56:10 +0000 | [diff] [blame] | 2108 | otherwise it will be set to ``'default'``. |
| 2109 | |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 2110 | Calling ``main`` actually returns an instance of the ``TestProgram`` class. |
| 2111 | This stores the result of the tests run as the ``result`` attribute. |
| 2112 | |
Éric Araujo | 971dc01 | 2010-12-16 03:13:05 +0000 | [diff] [blame] | 2113 | .. versionchanged:: 3.1 |
Ezio Melotti | 3d6d7a5 | 2012-04-30 19:10:28 +0300 | [diff] [blame] | 2114 | The *exit* parameter was added. |
Éric Araujo | 971dc01 | 2010-12-16 03:13:05 +0000 | [diff] [blame] | 2115 | |
Georg Brandl | 853947a | 2010-01-31 18:53:23 +0000 | [diff] [blame] | 2116 | .. versionchanged:: 3.2 |
Ezio Melotti | 3d6d7a5 | 2012-04-30 19:10:28 +0300 | [diff] [blame] | 2117 | The *verbosity*, *failfast*, *catchbreak*, *buffer* |
| 2118 | and *warnings* parameters were added. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 2119 | |
Chris Jerdonek | ccbc26a | 2013-02-23 15:44:46 -0800 | [diff] [blame] | 2120 | .. versionchanged:: 3.4 |
| 2121 | The *defaultTest* parameter was changed to also accept an iterable of |
| 2122 | test names. |
| 2123 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 2124 | |
| 2125 | load_tests Protocol |
| 2126 | ################### |
| 2127 | |
Georg Brandl | 853947a | 2010-01-31 18:53:23 +0000 | [diff] [blame] | 2128 | .. versionadded:: 3.2 |
Benjamin Peterson | 4ac9ce4 | 2009-10-04 14:49:41 +0000 | [diff] [blame] | 2129 | |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 2130 | Modules or packages can customize how tests are loaded from them during normal |
| 2131 | test runs or test discovery by implementing a function called ``load_tests``. |
| 2132 | |
| 2133 | If a test module defines ``load_tests`` it will be called by |
| 2134 | :meth:`TestLoader.loadTestsFromModule` with the following arguments:: |
| 2135 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 2136 | load_tests(loader, standard_tests, pattern) |
| 2137 | |
| 2138 | where *pattern* is passed straight through from ``loadTestsFromModule``. It |
| 2139 | defaults to ``None``. |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 2140 | |
| 2141 | It should return a :class:`TestSuite`. |
| 2142 | |
| 2143 | *loader* is the instance of :class:`TestLoader` doing the loading. |
| 2144 | *standard_tests* are the tests that would be loaded by default from the |
| 2145 | module. It is common for test modules to only want to add or remove tests |
| 2146 | from the standard set of tests. |
| 2147 | The third argument is used when loading packages as part of test discovery. |
| 2148 | |
| 2149 | A typical ``load_tests`` function that loads tests from a specific set of |
| 2150 | :class:`TestCase` classes may look like:: |
| 2151 | |
| 2152 | test_cases = (TestCase1, TestCase2, TestCase3) |
| 2153 | |
| 2154 | def load_tests(loader, tests, pattern): |
| 2155 | suite = TestSuite() |
| 2156 | for test_class in test_cases: |
| 2157 | tests = loader.loadTestsFromTestCase(test_class) |
| 2158 | suite.addTests(tests) |
| 2159 | return suite |
| 2160 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 2161 | If discovery is started in a directory containing a package, either from the |
| 2162 | command line or by calling :meth:`TestLoader.discover`, then the package |
| 2163 | :file:`__init__.py` will be checked for ``load_tests``. If that function does |
| 2164 | not exist, discovery will recurse into the package as though it were just |
| 2165 | another directory. Otherwise, discovery of the package's tests will be left up |
| 2166 | to ``load_tests`` which is called with the following arguments:: |
Benjamin Peterson | d239775 | 2009-06-27 23:45:02 +0000 | [diff] [blame] | 2167 | |
| 2168 | load_tests(loader, standard_tests, pattern) |
| 2169 | |
| 2170 | This should return a :class:`TestSuite` representing all the tests |
| 2171 | from the package. (``standard_tests`` will only contain tests |
| 2172 | collected from :file:`__init__.py`.) |
| 2173 | |
| 2174 | Because the pattern is passed into ``load_tests`` the package is free to |
| 2175 | continue (and potentially modify) test discovery. A 'do nothing' |
| 2176 | ``load_tests`` function for a test package would look like:: |
| 2177 | |
| 2178 | def load_tests(loader, standard_tests, pattern): |
| 2179 | # top level directory cached on loader instance |
| 2180 | this_dir = os.path.dirname(__file__) |
| 2181 | package_tests = loader.discover(start_dir=this_dir, pattern=pattern) |
| 2182 | standard_tests.addTests(package_tests) |
| 2183 | return standard_tests |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2184 | |
Barry Warsaw | d78742a | 2014-09-08 14:21:37 -0400 | [diff] [blame] | 2185 | .. versionchanged:: 3.5 |
| 2186 | Discovery no longer checks package names for matching *pattern* due to the |
| 2187 | impossibility of package names matching the default pattern. |
| 2188 | |
| 2189 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2190 | |
| 2191 | Class and Module Fixtures |
| 2192 | ------------------------- |
| 2193 | |
| 2194 | Class and module level fixtures are implemented in :class:`TestSuite`. When |
| 2195 | the test suite encounters a test from a new class then :meth:`tearDownClass` |
| 2196 | from the previous class (if there is one) is called, followed by |
| 2197 | :meth:`setUpClass` from the new class. |
| 2198 | |
| 2199 | Similarly if a test is from a different module from the previous test then |
| 2200 | ``tearDownModule`` from the previous module is run, followed by |
| 2201 | ``setUpModule`` from the new module. |
| 2202 | |
| 2203 | After all the tests have run the final ``tearDownClass`` and |
| 2204 | ``tearDownModule`` are run. |
| 2205 | |
| 2206 | Note that shared fixtures do not play well with [potential] features like test |
| 2207 | parallelization and they break test isolation. They should be used with care. |
| 2208 | |
| 2209 | The default ordering of tests created by the unittest test loaders is to group |
| 2210 | all tests from the same modules and classes together. This will lead to |
| 2211 | ``setUpClass`` / ``setUpModule`` (etc) being called exactly once per class and |
| 2212 | module. If you randomize the order, so that tests from different modules and |
| 2213 | classes are adjacent to each other, then these shared fixture functions may be |
| 2214 | called multiple times in a single test run. |
| 2215 | |
| 2216 | Shared fixtures are not intended to work with suites with non-standard |
| 2217 | ordering. A ``BaseTestSuite`` still exists for frameworks that don't want to |
| 2218 | support shared fixtures. |
| 2219 | |
| 2220 | If there are any exceptions raised during one of the shared fixture functions |
| 2221 | the test is reported as an error. Because there is no corresponding test |
| 2222 | instance an ``_ErrorHolder`` object (that has the same interface as a |
| 2223 | :class:`TestCase`) is created to represent the error. If you are just using |
| 2224 | the standard unittest test runner then this detail doesn't matter, but if you |
| 2225 | are a framework author it may be relevant. |
| 2226 | |
| 2227 | |
| 2228 | setUpClass and tearDownClass |
| 2229 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2230 | |
| 2231 | These must be implemented as class methods:: |
| 2232 | |
| 2233 | import unittest |
| 2234 | |
| 2235 | class Test(unittest.TestCase): |
| 2236 | @classmethod |
| 2237 | def setUpClass(cls): |
| 2238 | cls._connection = createExpensiveConnectionObject() |
| 2239 | |
| 2240 | @classmethod |
| 2241 | def tearDownClass(cls): |
| 2242 | cls._connection.destroy() |
| 2243 | |
| 2244 | If you want the ``setUpClass`` and ``tearDownClass`` on base classes called |
| 2245 | then you must call up to them yourself. The implementations in |
| 2246 | :class:`TestCase` are empty. |
| 2247 | |
| 2248 | If an exception is raised during a ``setUpClass`` then the tests in the class |
| 2249 | are not run and the ``tearDownClass`` is not run. Skipped classes will not |
Michael Foord | 98b3e76 | 2010-06-05 21:59:55 +0000 | [diff] [blame] | 2250 | have ``setUpClass`` or ``tearDownClass`` run. If the exception is a |
Ezio Melotti | 265281a | 2013-03-27 20:11:55 +0200 | [diff] [blame] | 2251 | :exc:`SkipTest` exception then the class will be reported as having been skipped |
Michael Foord | 98b3e76 | 2010-06-05 21:59:55 +0000 | [diff] [blame] | 2252 | instead of as an error. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2253 | |
| 2254 | |
| 2255 | setUpModule and tearDownModule |
| 2256 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 2257 | |
| 2258 | These should be implemented as functions:: |
| 2259 | |
| 2260 | def setUpModule(): |
| 2261 | createConnection() |
| 2262 | |
| 2263 | def tearDownModule(): |
| 2264 | closeConnection() |
| 2265 | |
| 2266 | If an exception is raised in a ``setUpModule`` then none of the tests in the |
Michael Foord | 98b3e76 | 2010-06-05 21:59:55 +0000 | [diff] [blame] | 2267 | module will be run and the ``tearDownModule`` will not be run. If the exception is a |
Ezio Melotti | 265281a | 2013-03-27 20:11:55 +0200 | [diff] [blame] | 2268 | :exc:`SkipTest` exception then the module will be reported as having been skipped |
Michael Foord | 98b3e76 | 2010-06-05 21:59:55 +0000 | [diff] [blame] | 2269 | instead of as an error. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2270 | |
| 2271 | |
| 2272 | Signal Handling |
| 2273 | --------------- |
| 2274 | |
Georg Brandl | 419e3de | 2010-12-01 15:44:25 +0000 | [diff] [blame] | 2275 | .. versionadded:: 3.2 |
| 2276 | |
Éric Araujo | 8acb67c | 2010-11-26 23:31:07 +0000 | [diff] [blame] | 2277 | The :option:`-c/--catch <unittest -c>` command-line option to unittest, |
Éric Araujo | 76338ec | 2010-11-26 23:46:18 +0000 | [diff] [blame] | 2278 | along with the ``catchbreak`` parameter to :func:`unittest.main()`, provide |
| 2279 | more friendly handling of control-C during a test run. With catch break |
| 2280 | behavior enabled control-C will allow the currently running test to complete, |
| 2281 | and the test run will then end and report all the results so far. A second |
| 2282 | control-c will raise a :exc:`KeyboardInterrupt` in the usual way. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2283 | |
Michael Foord | de4ceab | 2010-04-25 19:53:49 +0000 | [diff] [blame] | 2284 | The control-c handling signal handler attempts to remain compatible with code or |
| 2285 | tests that install their own :const:`signal.SIGINT` handler. If the ``unittest`` |
| 2286 | handler is called but *isn't* the installed :const:`signal.SIGINT` handler, |
| 2287 | i.e. it has been replaced by the system under test and delegated to, then it |
| 2288 | calls the default handler. This will normally be the expected behavior by code |
| 2289 | that replaces an installed handler and delegates to it. For individual tests |
| 2290 | that need ``unittest`` control-c handling disabled the :func:`removeHandler` |
| 2291 | decorator can be used. |
| 2292 | |
| 2293 | There are a few utility functions for framework authors to enable control-c |
| 2294 | handling functionality within test frameworks. |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2295 | |
| 2296 | .. function:: installHandler() |
| 2297 | |
| 2298 | Install the control-c handler. When a :const:`signal.SIGINT` is received |
| 2299 | (usually in response to the user pressing control-c) all registered results |
| 2300 | have :meth:`~TestResult.stop` called. |
| 2301 | |
Michael Foord | 469b1f0 | 2010-04-26 23:41:26 +0000 | [diff] [blame] | 2302 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2303 | .. function:: registerResult(result) |
| 2304 | |
| 2305 | Register a :class:`TestResult` object for control-c handling. Registering a |
| 2306 | result stores a weak reference to it, so it doesn't prevent the result from |
| 2307 | being garbage collected. |
| 2308 | |
Michael Foord | de4ceab | 2010-04-25 19:53:49 +0000 | [diff] [blame] | 2309 | Registering a :class:`TestResult` object has no side-effects if control-c |
| 2310 | handling is not enabled, so test frameworks can unconditionally register |
| 2311 | all results they create independently of whether or not handling is enabled. |
| 2312 | |
Michael Foord | 469b1f0 | 2010-04-26 23:41:26 +0000 | [diff] [blame] | 2313 | |
Benjamin Peterson | b48af54 | 2010-04-11 20:43:16 +0000 | [diff] [blame] | 2314 | .. function:: removeResult(result) |
| 2315 | |
| 2316 | Remove a registered result. Once a result has been removed then |
| 2317 | :meth:`~TestResult.stop` will no longer be called on that result object in |
| 2318 | response to a control-c. |
| 2319 | |
Michael Foord | 469b1f0 | 2010-04-26 23:41:26 +0000 | [diff] [blame] | 2320 | |
Michael Foord | de4ceab | 2010-04-25 19:53:49 +0000 | [diff] [blame] | 2321 | .. function:: removeHandler(function=None) |
| 2322 | |
| 2323 | When called without arguments this function removes the control-c handler |
| 2324 | if it has been installed. This function can also be used as a test decorator |
Mariatta | 98f42aa | 2018-02-23 09:51:11 -0800 | [diff] [blame] | 2325 | to temporarily remove the handler while the test is being executed:: |
Michael Foord | de4ceab | 2010-04-25 19:53:49 +0000 | [diff] [blame] | 2326 | |
| 2327 | @unittest.removeHandler |
| 2328 | def test_signal_handling(self): |
| 2329 | ... |