Benjamin Peterson | 90f5ba5 | 2010-03-11 22:53:45 +0000 | [diff] [blame] | 1 | #! /usr/bin/env python3 |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 2 | |
R. David Murray | 0ba81e0 | 2010-04-26 17:02:32 +0000 | [diff] [blame] | 3 | """ |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 4 | Script to run Python regression tests. |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 5 | |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 6 | Run this script with -h or --help for documentation. |
| 7 | """ |
| 8 | |
| 9 | USAGE = """\ |
Georg Brandl | be41a48 | 2011-01-05 21:47:47 +0000 | [diff] [blame] | 10 | python -m test [options] [test_name1 [test_name2 ...]] |
R. David Murray | 0ba81e0 | 2010-04-26 17:02:32 +0000 | [diff] [blame] | 11 | python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]] |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 12 | """ |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 13 | |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 14 | DESCRIPTION = """\ |
| 15 | Run Python regression tests. |
R. David Murray | 0ba81e0 | 2010-04-26 17:02:32 +0000 | [diff] [blame] | 16 | |
| 17 | If no arguments or options are provided, finds all files matching |
| 18 | the pattern "test_*" in the Lib/test subdirectory and runs |
| 19 | them in alphabetical order (but see -M and -u, below, for exceptions). |
| 20 | |
| 21 | For more rigorous testing, it is useful to use the following |
| 22 | command line: |
| 23 | |
Georg Brandl | be41a48 | 2011-01-05 21:47:47 +0000 | [diff] [blame] | 24 | python -E -Wd -m test [options] [test_name1 ...] |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 25 | """ |
R. David Murray | 0ba81e0 | 2010-04-26 17:02:32 +0000 | [diff] [blame] | 26 | |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 27 | EPILOG = """\ |
| 28 | Additional option details: |
Guido van Rossum | f58ed25 | 1997-03-07 21:04:33 +0000 | [diff] [blame] | 29 | |
Serhiy Storchaka | d65c949 | 2015-11-02 14:10:23 +0200 | [diff] [blame] | 30 | -r randomizes test execution order. You can use --randseed=int to provide an |
Collin Winter | fd12f49 | 2009-03-29 04:05:05 +0000 | [diff] [blame] | 31 | int seed value for the randomizer; this is useful for reproducing troublesome |
| 32 | test orders. |
| 33 | |
R. David Murray | 0ba81e0 | 2010-04-26 17:02:32 +0000 | [diff] [blame] | 34 | -s On the first invocation of regrtest using -s, the first test file found |
| 35 | or the first test file given on the command line is run, and the name of |
| 36 | the next test is recorded in a file named pynexttest. If run from the |
| 37 | Python build directory, pynexttest is located in the 'build' subdirectory, |
| 38 | otherwise it is located in tempfile.gettempdir(). On subsequent runs, |
| 39 | the test in pynexttest is run, and the next test is written to pynexttest. |
| 40 | When the last test has been run, pynexttest is deleted. In this way it |
| 41 | is possible to single step through the test files. This is useful when |
| 42 | doing memory analysis on the Python interpreter, which process tends to |
| 43 | consume too many resources to run the full regression test non-stop. |
Barry Warsaw | e11e3de | 1999-01-28 19:51:51 +0000 | [diff] [blame] | 44 | |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 45 | -S is used to continue running tests after an aborted run. It will |
| 46 | maintain the order a standard run (ie, this assumes -r is not used). |
| 47 | This is useful after the tests have prematurely stopped for some external |
| 48 | reason and you want to start running from where you left off rather |
| 49 | than starting from the beginning. |
| 50 | |
Guido van Rossum | 9e9d4f8 | 2002-06-07 15:17:03 +0000 | [diff] [blame] | 51 | -f reads the names of tests from the file given as f's argument, one |
| 52 | or more test names per line. Whitespace is ignored. Blank lines and |
| 53 | lines beginning with '#' are ignored. This is especially useful for |
| 54 | whittling down failures involving interactions among tests. |
Tim Peters | c5000df | 2002-06-02 21:42:01 +0000 | [diff] [blame] | 55 | |
Skip Montanaro | 0179a18 | 2004-06-06 15:53:18 +0000 | [diff] [blame] | 56 | -L causes the leaks(1) command to be run just before exit if it exists. |
| 57 | leaks(1) is available on Mac OS X and presumably on some other |
| 58 | FreeBSD-derived systems. |
| 59 | |
Michael W. Hudson | 61147f6 | 2004-08-03 11:33:28 +0000 | [diff] [blame] | 60 | -R runs each test several times and examines sys.gettotalrefcount() to |
| 61 | see if the test appears to be leaking references. The argument should |
| 62 | be of the form stab:run:fname where 'stab' is the number of times the |
| 63 | test is run to let gettotalrefcount settle down, 'run' is the number |
| 64 | of times further it is run and 'fname' is the name of the file the |
| 65 | reports are written to. These parameters all have defaults (5, 4 and |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 66 | "reflog.txt" respectively), and the minimal invocation is '-R :'. |
Michael W. Hudson | 61147f6 | 2004-08-03 11:33:28 +0000 | [diff] [blame] | 67 | |
Thomas Wouters | 477c8d5 | 2006-05-27 19:21:47 +0000 | [diff] [blame] | 68 | -M runs tests that require an exorbitant amount of memory. These tests |
| 69 | typically try to ascertain containers keep working when containing more than |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 70 | 2 billion objects, which only works on 64-bit systems. There are also some |
| 71 | tests that try to exhaust the address space of the process, which only makes |
| 72 | sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit, |
Thomas Wouters | 477c8d5 | 2006-05-27 19:21:47 +0000 | [diff] [blame] | 73 | which is a string in the form of '2.5Gb', determines howmuch memory the |
| 74 | tests will limit themselves to (but they may go slightly over.) The number |
| 75 | shouldn't be more memory than the machine has (including swap memory). You |
| 76 | should also keep in mind that swap memory is generally much, much slower |
| 77 | than RAM, and setting memlimit to all available RAM or higher will heavily |
| 78 | tax the machine. On the other hand, it is no use running these tests with a |
| 79 | limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect |
| 80 | to use more than memlimit memory will be skipped. The big-memory tests |
| 81 | generally run very, very long. |
| 82 | |
Guido van Rossum | 9e9d4f8 | 2002-06-07 15:17:03 +0000 | [diff] [blame] | 83 | -u is used to specify which special resource intensive tests to run, |
| 84 | such as those requiring large file support or network connectivity. |
| 85 | The argument is a comma-separated list of words indicating the |
| 86 | resources to test. Currently only the following are defined: |
Barry Warsaw | 08fca52 | 2001-08-20 22:33:46 +0000 | [diff] [blame] | 87 | |
Fred Drake | 3a15dac | 2002-04-11 16:39:16 +0000 | [diff] [blame] | 88 | all - Enable all special resources. |
| 89 | |
Nadeem Vawda | 3c01d16 | 2011-08-01 23:48:26 +0200 | [diff] [blame] | 90 | none - Disable all special resources (this is the default). |
| 91 | |
Guido van Rossum | 315aa36 | 2003-03-11 14:46:48 +0000 | [diff] [blame] | 92 | audio - Tests that use the audio device. (There are known |
| 93 | cases of broken audio drivers that can crash Python or |
| 94 | even the Linux kernel.) |
| 95 | |
Andrew M. Kuchling | 2158df0 | 2001-10-22 15:26:09 +0000 | [diff] [blame] | 96 | curses - Tests that use curses and will modify the terminal's |
| 97 | state and output modes. |
Tim Peters | 1633a2e | 2001-10-30 05:56:40 +0000 | [diff] [blame] | 98 | |
Guido van Rossum | 9e9d4f8 | 2002-06-07 15:17:03 +0000 | [diff] [blame] | 99 | largefile - It is okay to run some test that may create huge |
| 100 | files. These tests can take a long time and may |
| 101 | consume >2GB of disk space temporarily. |
Barry Warsaw | 08fca52 | 2001-08-20 22:33:46 +0000 | [diff] [blame] | 102 | |
Guido van Rossum | 9e9d4f8 | 2002-06-07 15:17:03 +0000 | [diff] [blame] | 103 | network - It is okay to run tests that use external network |
| 104 | resource, e.g. testing SSL support for sockets. |
Martin v. Löwis | 1c6b1a2 | 2002-11-19 17:47:07 +0000 | [diff] [blame] | 105 | |
Raymond Hettinger | 7c85fa4 | 2004-07-01 11:01:35 +0000 | [diff] [blame] | 106 | decimal - Test the decimal module against a large suite that |
| 107 | verifies compliance with standards. |
| 108 | |
Antoine Pitrou | 5bc4fa7 | 2010-10-14 15:34:31 +0000 | [diff] [blame] | 109 | cpu - Used for certain CPU-heavy tests. |
Jeremy Hylton | 4336eda | 2004-08-07 19:25:33 +0000 | [diff] [blame] | 110 | |
Tim Peters | eba28be | 2005-03-28 01:08:02 +0000 | [diff] [blame] | 111 | subprocess Run all tests for the subprocess module. |
Peter Astrand | f7f1bb7 | 2005-03-03 20:47:37 +0000 | [diff] [blame] | 112 | |
Hye-Shik Chang | aaa2f1d | 2005-12-10 17:44:27 +0000 | [diff] [blame] | 113 | urlfetch - It is okay to download files required on testing. |
| 114 | |
Guilherme Polo | 9de29af | 2009-01-28 20:40:48 +0000 | [diff] [blame] | 115 | gui - Run tests that require a running GUI. |
| 116 | |
Fred Drake | 4dd0f7e | 2002-11-26 21:44:56 +0000 | [diff] [blame] | 117 | To enable all resources except one, use '-uall,-<resource>'. For |
Georg Brandl | 1158a33 | 2009-06-04 09:30:30 +0000 | [diff] [blame] | 118 | example, to run all the tests except for the gui tests, give the |
| 119 | option '-uall,-gui'. |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 120 | """ |
| 121 | |
Nick Coghlan | be7e49f | 2012-07-20 23:40:09 +1000 | [diff] [blame] | 122 | # We import importlib *ASAP* in order to test #15386 |
| 123 | import importlib |
| 124 | |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 125 | import argparse |
Brett Cannon | 4507190 | 2010-06-14 22:22:54 +0000 | [diff] [blame] | 126 | import builtins |
Victor Stinner | 024e37a | 2011-03-31 01:31:06 +0200 | [diff] [blame] | 127 | import faulthandler |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 128 | import io |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 129 | import json |
Brett Cannon | 49e835b | 2013-04-01 14:11:37 -0400 | [diff] [blame] | 130 | import locale |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 131 | import logging |
Christian Heimes | b186d00 | 2008-03-18 15:15:01 +0000 | [diff] [blame] | 132 | import os |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 133 | import platform |
Skip Montanaro | ab1c791 | 2000-06-30 16:39:27 +0000 | [diff] [blame] | 134 | import random |
Thomas Wouters | 9ada3d6 | 2006-04-21 09:47:09 +0000 | [diff] [blame] | 135 | import re |
Éric Araujo | ff91306 | 2011-11-29 16:45:07 +0100 | [diff] [blame] | 136 | import shutil |
Victor Stinner | cb41cda | 2011-07-13 23:47:21 +0200 | [diff] [blame] | 137 | import signal |
Christian Heimes | b186d00 | 2008-03-18 15:15:01 +0000 | [diff] [blame] | 138 | import sys |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 139 | import sysconfig |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 140 | import tempfile |
| 141 | import time |
| 142 | import traceback |
| 143 | import unittest |
| 144 | import warnings |
| 145 | from inspect import isabstract |
Ezio Melotti | 184bdfb | 2010-02-18 09:37:05 +0000 | [diff] [blame] | 146 | |
Antoine Pitrou | c081c0c | 2011-07-15 22:12:24 +0200 | [diff] [blame] | 147 | try: |
| 148 | import threading |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 149 | except ImportError: |
Antoine Pitrou | c081c0c | 2011-07-15 22:12:24 +0200 | [diff] [blame] | 150 | threading = None |
| 151 | try: |
Richard Oudkerk | 84ed9a6 | 2013-08-14 15:35:41 +0100 | [diff] [blame] | 152 | import _multiprocessing, multiprocessing.process |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 153 | except ImportError: |
Antoine Pitrou | c081c0c | 2011-07-15 22:12:24 +0200 | [diff] [blame] | 154 | multiprocessing = None |
| 155 | |
Florent Xicluna | 64fb18e | 2010-03-06 14:43:34 +0000 | [diff] [blame] | 156 | |
Ezio Melotti | 184bdfb | 2010-02-18 09:37:05 +0000 | [diff] [blame] | 157 | # Some times __path__ and __file__ are not absolute (e.g. while running from |
| 158 | # Lib/) and, if we change the CWD to run the tests in a temporary dir, some |
| 159 | # imports might fail. This affects only the modules imported before os.chdir(). |
| 160 | # These modules are searched first in sys.path[0] (so '' -- the CWD) and if |
| 161 | # they are found in the CWD their __file__ and __path__ will be relative (this |
| 162 | # happens before the chdir). All the modules imported after the chdir, are |
| 163 | # not found in the CWD, and since the other paths in sys.path[1:] are absolute |
| 164 | # (site.py absolutize them), the __file__ and __path__ will be absolute too. |
| 165 | # Therefore it is necessary to absolutize manually the __file__ and __path__ of |
| 166 | # the packages to prevent later imports to fail when the CWD is different. |
| 167 | for module in sys.modules.values(): |
| 168 | if hasattr(module, '__path__'): |
| 169 | module.__path__ = [os.path.abspath(path) for path in module.__path__] |
| 170 | if hasattr(module, '__file__'): |
| 171 | module.__file__ = os.path.abspath(module.__file__) |
| 172 | |
Guido van Rossum | dc15c27 | 2002-08-12 21:55:51 +0000 | [diff] [blame] | 173 | |
Guido van Rossum | bb48465 | 2002-12-02 09:56:21 +0000 | [diff] [blame] | 174 | # MacOSX (a.k.a. Darwin) has a default stack size that is too small |
| 175 | # for deeply recursive regular expressions. We see this as crashes in |
| 176 | # the Python test suite when running test_re.py and test_sre.py. The |
| 177 | # fix is to set the stack limit to 2048. |
| 178 | # This approach may also be useful for other Unixy platforms that |
| 179 | # suffer from small default stack limits. |
| 180 | if sys.platform == 'darwin': |
| 181 | try: |
| 182 | import resource |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 183 | except ImportError: |
Guido van Rossum | bb48465 | 2002-12-02 09:56:21 +0000 | [diff] [blame] | 184 | pass |
| 185 | else: |
| 186 | soft, hard = resource.getrlimit(resource.RLIMIT_STACK) |
| 187 | newsoft = min(hard, max(soft, 1024*2048)) |
| 188 | resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) |
| 189 | |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 190 | # Test result constants. |
| 191 | PASSED = 1 |
| 192 | FAILED = 0 |
| 193 | ENV_CHANGED = -1 |
| 194 | SKIPPED = -2 |
| 195 | RESOURCE_DENIED = -3 |
| 196 | INTERRUPTED = -4 |
Victor Stinner | 4b73988 | 2011-03-31 18:02:36 +0200 | [diff] [blame] | 197 | CHILD_ERROR = -5 # error in a child process |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 198 | |
Benjamin Peterson | ee8712c | 2008-05-20 21:35:26 +0000 | [diff] [blame] | 199 | from test import support |
Fred Drake | 3a15dac | 2002-04-11 16:39:16 +0000 | [diff] [blame] | 200 | |
Georg Brandl | 1158a33 | 2009-06-04 09:30:30 +0000 | [diff] [blame] | 201 | RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', |
Antoine Pitrou | 5bc4fa7 | 2010-10-14 15:34:31 +0000 | [diff] [blame] | 202 | 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui') |
Fred Drake | 3a15dac | 2002-04-11 16:39:16 +0000 | [diff] [blame] | 203 | |
Chris Jerdonek | 517e925 | 2013-02-27 09:02:53 -0800 | [diff] [blame] | 204 | # When tests are run from the Python build directory, it is best practice |
| 205 | # to keep the test files in a subfolder. This eases the cleanup of leftover |
| 206 | # files using the "make distclean" command. |
| 207 | if sysconfig.is_python_build(): |
| 208 | TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') |
| 209 | else: |
| 210 | TEMPDIR = tempfile.gettempdir() |
| 211 | TEMPDIR = os.path.abspath(TEMPDIR) |
Fred Drake | 3a15dac | 2002-04-11 16:39:16 +0000 | [diff] [blame] | 212 | |
Chris Jerdonek | 1573842 | 2013-01-07 17:07:32 -0800 | [diff] [blame] | 213 | class _ArgParser(argparse.ArgumentParser): |
| 214 | |
| 215 | def error(self, message): |
| 216 | super().error(message + "\nPass -h or --help for complete help.") |
| 217 | |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 218 | def _create_parser(): |
| 219 | # Set prog to prevent the uninformative "__main__.py" from displaying in |
| 220 | # error messages when using "python -m test ...". |
Chris Jerdonek | 1573842 | 2013-01-07 17:07:32 -0800 | [diff] [blame] | 221 | parser = _ArgParser(prog='regrtest.py', |
| 222 | usage=USAGE, |
| 223 | description=DESCRIPTION, |
| 224 | epilog=EPILOG, |
| 225 | add_help=False, |
| 226 | formatter_class=argparse.RawDescriptionHelpFormatter) |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 227 | |
| 228 | # Arguments with this clause added to its help are described further in |
| 229 | # the epilog's "Additional option details" section. |
| 230 | more_details = ' See the section at bottom for more details.' |
| 231 | |
| 232 | group = parser.add_argument_group('General options') |
| 233 | # We add help explicitly to control what argument group it renders under. |
| 234 | group.add_argument('-h', '--help', action='help', |
| 235 | help='show this help message and exit') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 236 | group.add_argument('--timeout', metavar='TIMEOUT', type=float, |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 237 | help='dump the traceback and exit if a test takes ' |
| 238 | 'more than TIMEOUT seconds; disabled if TIMEOUT ' |
| 239 | 'is negative or equals to zero') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 240 | group.add_argument('--wait', action='store_true', |
| 241 | help='wait for user input, e.g., allow a debugger ' |
| 242 | 'to be attached') |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 243 | group.add_argument('--slaveargs', metavar='ARGS') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 244 | group.add_argument('-S', '--start', metavar='START', |
| 245 | help='the name of the test at which to start.' + |
| 246 | more_details) |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 247 | |
| 248 | group = parser.add_argument_group('Verbosity') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 249 | group.add_argument('-v', '--verbose', action='count', |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 250 | help='run tests in verbose mode with output to stdout') |
| 251 | group.add_argument('-w', '--verbose2', action='store_true', |
| 252 | help='re-run failed tests in verbose mode') |
| 253 | group.add_argument('-W', '--verbose3', action='store_true', |
| 254 | help='display test output on failure') |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 255 | group.add_argument('-q', '--quiet', action='store_true', |
| 256 | help='no output unless one or more tests fail') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 257 | group.add_argument('-o', '--slow', action='store_true', dest='print_slow', |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 258 | help='print the slowest 10 tests') |
| 259 | group.add_argument('--header', action='store_true', |
| 260 | help='print header with interpreter info') |
| 261 | |
| 262 | group = parser.add_argument_group('Selecting tests') |
| 263 | group.add_argument('-r', '--randomize', action='store_true', |
| 264 | help='randomize test execution order.' + more_details) |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 265 | group.add_argument('--randseed', metavar='SEED', |
| 266 | dest='random_seed', type=int, |
| 267 | help='pass a random seed to reproduce a previous ' |
| 268 | 'random run') |
| 269 | group.add_argument('-f', '--fromfile', metavar='FILE', |
| 270 | help='read names of tests to run from a file.' + |
| 271 | more_details) |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 272 | group.add_argument('-x', '--exclude', action='store_true', |
| 273 | help='arguments are tests to *exclude*') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 274 | group.add_argument('-s', '--single', action='store_true', |
| 275 | help='single step through a set of tests.' + |
| 276 | more_details) |
| 277 | group.add_argument('-m', '--match', metavar='PAT', |
| 278 | dest='match_tests', |
| 279 | help='match test cases and methods with glob pattern PAT') |
| 280 | group.add_argument('-G', '--failfast', action='store_true', |
| 281 | help='fail as soon as a test fails (only with -v or -W)') |
| 282 | group.add_argument('-u', '--use', metavar='RES1,RES2,...', |
| 283 | action='append', type=resources_list, |
| 284 | help='specify which special resource intensive tests ' |
| 285 | 'to run.' + more_details) |
| 286 | group.add_argument('-M', '--memlimit', metavar='LIMIT', |
| 287 | help='run very large memory-consuming tests.' + |
| 288 | more_details) |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 289 | group.add_argument('--testdir', metavar='DIR', |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 290 | type=relative_filename, |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 291 | help='execute test files in the specified directory ' |
| 292 | '(instead of the Python stdlib test suite)') |
| 293 | |
| 294 | group = parser.add_argument_group('Special runs') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 295 | group.add_argument('-l', '--findleaks', action='store_true', |
| 296 | help='if GC is available detect tests that leak memory') |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 297 | group.add_argument('-L', '--runleaks', action='store_true', |
| 298 | help='run the leaks(1) command just before exit.' + |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 299 | more_details) |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 300 | group.add_argument('-R', '--huntrleaks', metavar='RUNCOUNTS', |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 301 | type=huntrleaks, |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 302 | help='search for reference leaks (needs debug build, ' |
| 303 | 'very slow).' + more_details) |
| 304 | group.add_argument('-j', '--multiprocess', metavar='PROCESSES', |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 305 | dest='use_mp', type=int, |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 306 | help='run PROCESSES processes at once') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 307 | group.add_argument('-T', '--coverage', action='store_true', |
| 308 | dest='trace', |
| 309 | help='turn on code coverage tracing using the trace ' |
| 310 | 'module') |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 311 | group.add_argument('-D', '--coverdir', metavar='DIR', |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 312 | type=relative_filename, |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 313 | help='directory where coverage files are put') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 314 | group.add_argument('-N', '--nocoverdir', |
| 315 | action='store_const', const=None, dest='coverdir', |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 316 | help='put coverage files alongside modules') |
| 317 | group.add_argument('-t', '--threshold', metavar='THRESHOLD', |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 318 | type=int, |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 319 | help='call gc.set_threshold(THRESHOLD)') |
| 320 | group.add_argument('-n', '--nowindows', action='store_true', |
| 321 | help='suppress error message boxes on Windows') |
| 322 | group.add_argument('-F', '--forever', action='store_true', |
| 323 | help='run the specified tests in a loop, until an ' |
| 324 | 'error happens') |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 325 | group.add_argument('-P', '--pgo', dest='pgo', action='store_true', |
| 326 | help='enable Profile Guided Optimization training') |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 327 | |
| 328 | parser.add_argument('args', nargs=argparse.REMAINDER, |
| 329 | help=argparse.SUPPRESS) |
| 330 | |
| 331 | return parser |
| 332 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 333 | def relative_filename(string): |
| 334 | # CWD is replaced with a temporary dir before calling main(), so we |
| 335 | # join it with the saved CWD so it ends up where the user expects. |
| 336 | return os.path.join(support.SAVEDCWD, string) |
Chris Jerdonek | 1573842 | 2013-01-07 17:07:32 -0800 | [diff] [blame] | 337 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 338 | def huntrleaks(string): |
| 339 | args = string.split(':') |
| 340 | if len(args) not in (2, 3): |
| 341 | raise argparse.ArgumentTypeError( |
| 342 | 'needs 2 or 3 colon-separated arguments') |
| 343 | nwarmup = int(args[0]) if args[0] else 5 |
| 344 | ntracked = int(args[1]) if args[1] else 4 |
| 345 | fname = args[2] if len(args) > 2 and args[2] else 'reflog.txt' |
| 346 | return nwarmup, ntracked, fname |
| 347 | |
| 348 | def resources_list(string): |
| 349 | u = [x.lower() for x in string.split(',')] |
| 350 | for r in u: |
| 351 | if r == 'all' or r == 'none': |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 352 | continue |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 353 | if r[0] == '-': |
| 354 | r = r[1:] |
| 355 | if r not in RESOURCE_NAMES: |
| 356 | raise argparse.ArgumentTypeError('invalid resource: ' + r) |
| 357 | return u |
Barry Warsaw | 08fca52 | 2001-08-20 22:33:46 +0000 | [diff] [blame] | 358 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 359 | def _parse_args(args, **kwargs): |
| 360 | # Defaults |
| 361 | ns = argparse.Namespace(testdir=None, verbose=0, quiet=False, |
Barry Warsaw | 3b6d025 | 2004-02-07 22:43:03 +0000 | [diff] [blame] | 362 | exclude=False, single=False, randomize=False, fromfile=None, |
Walter Dörwald | aee4da6 | 2004-11-12 18:51:27 +0000 | [diff] [blame] | 363 | findleaks=False, use_resources=None, trace=False, coverdir='coverage', |
Collin Winter | fd12f49 | 2009-03-29 04:05:05 +0000 | [diff] [blame] | 364 | runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, |
Antoine Pitrou | 3c4402f | 2011-01-03 20:38:52 +0000 | [diff] [blame] | 365 | random_seed=None, use_mp=None, verbose3=False, forever=False, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 366 | header=False, failfast=False, match_tests=None, pgo=False) |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 367 | for k, v in kwargs.items(): |
| 368 | if not hasattr(ns, k): |
| 369 | raise TypeError('%r is an invalid keyword argument ' |
| 370 | 'for this function' % k) |
| 371 | setattr(ns, k, v) |
| 372 | if ns.use_resources is None: |
| 373 | ns.use_resources = [] |
| 374 | |
| 375 | parser = _create_parser() |
| 376 | parser.parse_args(args=args, namespace=ns) |
| 377 | |
| 378 | if ns.single and ns.fromfile: |
| 379 | parser.error("-s and -f don't go together!") |
| 380 | if ns.use_mp and ns.trace: |
| 381 | parser.error("-T and -j don't go together!") |
| 382 | if ns.use_mp and ns.findleaks: |
| 383 | parser.error("-l and -j don't go together!") |
| 384 | if ns.use_mp and ns.memlimit: |
| 385 | parser.error("-M and -j don't go together!") |
| 386 | if ns.failfast and not (ns.verbose or ns.verbose3): |
| 387 | parser.error("-G/--failfast needs either -v or -W") |
| 388 | |
| 389 | if ns.quiet: |
| 390 | ns.verbose = 0 |
| 391 | if ns.timeout is not None: |
| 392 | if hasattr(faulthandler, 'dump_traceback_later'): |
| 393 | if ns.timeout <= 0: |
| 394 | ns.timeout = None |
| 395 | else: |
| 396 | print("Warning: The timeout option requires " |
| 397 | "faulthandler.dump_traceback_later") |
| 398 | ns.timeout = None |
| 399 | if ns.use_mp is not None: |
| 400 | if ns.use_mp <= 0: |
| 401 | # Use all cores + extras for tests that like to sleep |
| 402 | ns.use_mp = 2 + (os.cpu_count() or 1) |
| 403 | if ns.use_mp == 1: |
| 404 | ns.use_mp = None |
| 405 | if ns.use: |
| 406 | for a in ns.use: |
| 407 | for r in a: |
| 408 | if r == 'all': |
| 409 | ns.use_resources[:] = RESOURCE_NAMES |
| 410 | continue |
| 411 | if r == 'none': |
| 412 | del ns.use_resources[:] |
| 413 | continue |
| 414 | remove = False |
| 415 | if r[0] == '-': |
| 416 | remove = True |
| 417 | r = r[1:] |
| 418 | if remove: |
| 419 | if r in ns.use_resources: |
| 420 | ns.use_resources.remove(r) |
| 421 | elif r not in ns.use_resources: |
| 422 | ns.use_resources.append(r) |
| 423 | if ns.random_seed is not None: |
| 424 | ns.randomize = True |
| 425 | |
| 426 | return ns |
| 427 | |
| 428 | |
Eli Bendersky | 7f5c22c | 2013-09-02 08:57:21 -0700 | [diff] [blame] | 429 | def run_test_in_subprocess(testname, ns): |
| 430 | """Run the given test in a subprocess with --slaveargs. |
| 431 | |
| 432 | ns is the option Namespace parsed from command-line arguments. regrtest |
| 433 | is invoked in a subprocess with the --slaveargs argument; when the |
| 434 | subprocess exits, its return code, stdout and stderr are returned as a |
| 435 | 3-tuple. |
| 436 | """ |
| 437 | from subprocess import Popen, PIPE |
| 438 | base_cmd = ([sys.executable] + support.args_from_interpreter_flags() + |
| 439 | ['-X', 'faulthandler', '-m', 'test.regrtest']) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 440 | # required to spawn a new process with PGO flag on/off |
| 441 | if ns.pgo: |
| 442 | base_cmd = base_cmd + ['--pgo'] |
Eli Bendersky | 7f5c22c | 2013-09-02 08:57:21 -0700 | [diff] [blame] | 443 | slaveargs = ( |
| 444 | (testname, ns.verbose, ns.quiet), |
| 445 | dict(huntrleaks=ns.huntrleaks, |
| 446 | use_resources=ns.use_resources, |
Eli Bendersky | e8de296 | 2013-09-02 17:01:10 -0700 | [diff] [blame] | 447 | output_on_failure=ns.verbose3, |
Eli Bendersky | 7f5c22c | 2013-09-02 08:57:21 -0700 | [diff] [blame] | 448 | timeout=ns.timeout, failfast=ns.failfast, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 449 | match_tests=ns.match_tests, pgo=ns.pgo)) |
Eli Bendersky | 7f5c22c | 2013-09-02 08:57:21 -0700 | [diff] [blame] | 450 | # Running the child from the same working directory as regrtest's original |
| 451 | # invocation ensures that TEMPDIR for the child is the same when |
| 452 | # sysconfig.is_python_build() is true. See issue 15300. |
| 453 | popen = Popen(base_cmd + ['--slaveargs', json.dumps(slaveargs)], |
| 454 | stdout=PIPE, stderr=PIPE, |
| 455 | universal_newlines=True, |
| 456 | close_fds=(os.name != 'nt'), |
| 457 | cwd=support.SAVEDCWD) |
| 458 | stdout, stderr = popen.communicate() |
| 459 | retcode = popen.wait() |
| 460 | return retcode, stdout, stderr |
| 461 | |
| 462 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 463 | def main(tests=None, **kwargs): |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 464 | """Execute a test suite. |
| 465 | |
Thomas Wouters | 7e47402 | 2000-07-16 12:04:32 +0000 | [diff] [blame] | 466 | This also parses command-line options and modifies its behavior |
Fred Drake | 004d5e6 | 2000-10-23 17:22:08 +0000 | [diff] [blame] | 467 | accordingly. |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 468 | |
| 469 | tests -- a list of strings containing test names (optional) |
| 470 | testdir -- the directory in which to look for tests (optional) |
| 471 | |
| 472 | Users other than the Python test suite will certainly want to |
| 473 | specify testdir; if it's omitted, the directory containing the |
Fred Drake | 004d5e6 | 2000-10-23 17:22:08 +0000 | [diff] [blame] | 474 | Python test suite is searched for. |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 475 | |
| 476 | If the tests argument is omitted, the tests listed on the |
| 477 | command-line will be used. If that's empty, too, then all *.py |
| 478 | files beginning with test_ will be used. |
Skip Montanaro | ab1c791 | 2000-06-30 16:39:27 +0000 | [diff] [blame] | 479 | |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 480 | The other default arguments (verbose, quiet, exclude, |
Collin Winter | fd12f49 | 2009-03-29 04:05:05 +0000 | [diff] [blame] | 481 | single, randomize, findleaks, use_resources, trace, coverdir, |
| 482 | print_slow, and random_seed) allow programmers calling main() |
| 483 | directly to set the values that would normally be set by flags |
| 484 | on the command line. |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 485 | """ |
Victor Stinner | cb41cda | 2011-07-13 23:47:21 +0200 | [diff] [blame] | 486 | # Display the Python traceback on fatal errors (e.g. segfault) |
Victor Stinner | 3c18f25 | 2011-05-22 15:27:14 +0200 | [diff] [blame] | 487 | faulthandler.enable(all_threads=True) |
| 488 | |
Victor Stinner | cb41cda | 2011-07-13 23:47:21 +0200 | [diff] [blame] | 489 | # Display the Python traceback on SIGALRM or SIGUSR1 signal |
| 490 | signals = [] |
| 491 | if hasattr(signal, 'SIGALRM'): |
| 492 | signals.append(signal.SIGALRM) |
| 493 | if hasattr(signal, 'SIGUSR1'): |
| 494 | signals.append(signal.SIGUSR1) |
| 495 | for signum in signals: |
| 496 | faulthandler.register(signum, chain=True) |
| 497 | |
Victor Stinner | 1802d3f | 2010-05-19 17:11:19 +0000 | [diff] [blame] | 498 | replace_stdout() |
| 499 | |
Benjamin Peterson | ee8712c | 2008-05-20 21:35:26 +0000 | [diff] [blame] | 500 | support.record_original_stdout(sys.stdout) |
Chris Jerdonek | d6c18dc | 2012-12-27 18:53:12 -0800 | [diff] [blame] | 501 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 502 | ns = _parse_args(sys.argv[1:], **kwargs) |
Barry Warsaw | 08fca52 | 2001-08-20 22:33:46 +0000 | [diff] [blame] | 503 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 504 | if ns.huntrleaks: |
| 505 | # Avoid false positives due to various caches |
| 506 | # filling slowly with random data: |
| 507 | warm_caches() |
| 508 | if ns.memlimit is not None: |
| 509 | support.set_memlimit(ns.memlimit) |
| 510 | if ns.threshold is not None: |
| 511 | import gc |
| 512 | gc.set_threshold(ns.threshold) |
| 513 | if ns.nowindows: |
Steve Dower | 12ee744 | 2015-10-08 08:56:06 -0700 | [diff] [blame] | 514 | print('The --nowindows (-n) option is deprecated. ' |
| 515 | 'Use -vv to display assertions in stderr.') |
| 516 | try: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 517 | import msvcrt |
Steve Dower | 12ee744 | 2015-10-08 08:56:06 -0700 | [diff] [blame] | 518 | except ImportError: |
| 519 | pass |
| 520 | else: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 521 | msvcrt.SetErrorMode(msvcrt.SEM_FAILCRITICALERRORS| |
| 522 | msvcrt.SEM_NOALIGNMENTFAULTEXCEPT| |
| 523 | msvcrt.SEM_NOGPFAULTERRORBOX| |
| 524 | msvcrt.SEM_NOOPENFILEERRORBOX) |
| 525 | try: |
| 526 | msvcrt.CrtSetReportMode |
| 527 | except AttributeError: |
| 528 | # release build |
| 529 | pass |
R. David Murray | 35768ad | 2009-11-15 00:23:21 +0000 | [diff] [blame] | 530 | else: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 531 | for m in [msvcrt.CRT_WARN, msvcrt.CRT_ERROR, msvcrt.CRT_ASSERT]: |
Steve Dower | 12ee744 | 2015-10-08 08:56:06 -0700 | [diff] [blame] | 532 | if ns.verbose and ns.verbose >= 2: |
| 533 | msvcrt.CrtSetReportMode(m, msvcrt.CRTDBG_MODE_FILE) |
| 534 | msvcrt.CrtSetReportFile(m, msvcrt.CRTDBG_FILE_STDERR) |
| 535 | else: |
| 536 | msvcrt.CrtSetReportMode(m, 0) |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 537 | if ns.wait: |
| 538 | input("Press any key to continue...") |
| 539 | |
| 540 | if ns.slaveargs is not None: |
| 541 | args, kwargs = json.loads(ns.slaveargs) |
Andrew Svetlov | 8913a6c | 2013-09-01 07:58:41 +0300 | [diff] [blame] | 542 | if kwargs.get('huntrleaks'): |
| 543 | unittest.BaseTestSuite._cleanup = False |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 544 | try: |
| 545 | result = runtest(*args, **kwargs) |
| 546 | except KeyboardInterrupt: |
| 547 | result = INTERRUPTED, '' |
| 548 | except BaseException as e: |
| 549 | traceback.print_exc() |
| 550 | result = CHILD_ERROR, str(e) |
| 551 | sys.stdout.flush() |
| 552 | print() # Force a newline (just in case) |
| 553 | print(json.dumps(result)) |
| 554 | sys.exit(0) |
Barry Warsaw | 08fca52 | 2001-08-20 22:33:46 +0000 | [diff] [blame] | 555 | |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 556 | good = [] |
| 557 | bad = [] |
| 558 | skipped = [] |
Fred Drake | 9a0db07 | 2003-02-03 15:19:30 +0000 | [diff] [blame] | 559 | resource_denieds = [] |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 560 | environment_changed = [] |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 561 | interrupted = False |
Barry Warsaw | e11e3de | 1999-01-28 19:51:51 +0000 | [diff] [blame] | 562 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 563 | if ns.findleaks: |
Barry Warsaw | a873b03 | 2000-08-03 15:50:37 +0000 | [diff] [blame] | 564 | try: |
| 565 | import gc |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 566 | except ImportError: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 567 | print('No GC available, disabling findleaks.') |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 568 | ns.findleaks = False |
Barry Warsaw | a873b03 | 2000-08-03 15:50:37 +0000 | [diff] [blame] | 569 | else: |
Neil Schemenauer | 8a00abc | 2000-10-13 01:32:42 +0000 | [diff] [blame] | 570 | # Uncomment the line below to report garbage that is not |
| 571 | # freeable by reference counting alone. By default only |
| 572 | # garbage that is not collectable by the GC is reported. |
| 573 | #gc.set_debug(gc.DEBUG_SAVEALL) |
Neil Schemenauer | d569f23 | 2000-09-22 15:29:28 +0000 | [diff] [blame] | 574 | found_garbage = [] |
Barry Warsaw | a873b03 | 2000-08-03 15:50:37 +0000 | [diff] [blame] | 575 | |
Andrew Svetlov | 8913a6c | 2013-09-01 07:58:41 +0300 | [diff] [blame] | 576 | if ns.huntrleaks: |
| 577 | unittest.BaseTestSuite._cleanup = False |
| 578 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 579 | if ns.single: |
Florent Xicluna | ec88221 | 2010-08-09 16:56:43 +0000 | [diff] [blame] | 580 | filename = os.path.join(TEMPDIR, 'pynexttest') |
Barry Warsaw | e11e3de | 1999-01-28 19:51:51 +0000 | [diff] [blame] | 581 | try: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 582 | with open(filename, 'r') as fp: |
| 583 | next_test = fp.read().strip() |
| 584 | tests = [next_test] |
Andrew Svetlov | f7a17b4 | 2012-12-25 16:47:37 +0200 | [diff] [blame] | 585 | except OSError: |
Barry Warsaw | e11e3de | 1999-01-28 19:51:51 +0000 | [diff] [blame] | 586 | pass |
Tim Peters | c5000df | 2002-06-02 21:42:01 +0000 | [diff] [blame] | 587 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 588 | if ns.fromfile: |
Tim Peters | c5000df | 2002-06-02 21:42:01 +0000 | [diff] [blame] | 589 | tests = [] |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 590 | with open(os.path.join(support.SAVEDCWD, ns.fromfile)) as fp: |
| 591 | count_pat = re.compile(r'\[\s*\d+/\s*\d+\]') |
| 592 | for line in fp: |
| 593 | line = count_pat.sub('', line) |
| 594 | guts = line.split() # assuming no test has whitespace in its name |
| 595 | if guts and not guts[0].startswith('#'): |
| 596 | tests.extend(guts) |
Tim Peters | c5000df | 2002-06-02 21:42:01 +0000 | [diff] [blame] | 597 | |
| 598 | # Strip .py extensions. |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 599 | removepy(ns.args) |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 600 | removepy(tests) |
Tim Peters | c5000df | 2002-06-02 21:42:01 +0000 | [diff] [blame] | 601 | |
Guido van Rossum | 6c74fea | 1998-08-25 12:29:08 +0000 | [diff] [blame] | 602 | stdtests = STDTESTS[:] |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 603 | nottests = NOTTESTS.copy() |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 604 | if ns.exclude: |
| 605 | for arg in ns.args: |
Guido van Rossum | 6c74fea | 1998-08-25 12:29:08 +0000 | [diff] [blame] | 606 | if arg in stdtests: |
| 607 | stdtests.remove(arg) |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 608 | nottests.add(arg) |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 609 | ns.args = [] |
Florent Xicluna | 0e62a14 | 2010-03-06 17:34:48 +0000 | [diff] [blame] | 610 | |
| 611 | # For a partial run, we do not need to clutter the output. |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 612 | if (ns.verbose or ns.header or |
| 613 | not (ns.pgo or ns.quiet or ns.single or tests or ns.args)): |
Florent Xicluna | 0e62a14 | 2010-03-06 17:34:48 +0000 | [diff] [blame] | 614 | # Print basic platform information |
| 615 | print("==", platform.python_implementation(), *sys.version.split()) |
Florent Xicluna | ec88221 | 2010-08-09 16:56:43 +0000 | [diff] [blame] | 616 | print("== ", platform.platform(aliased=True), |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 617 | "%s-endian" % sys.byteorder) |
Christian Heimes | 985ecdc | 2013-11-20 11:46:18 +0100 | [diff] [blame] | 618 | print("== ", "hash algorithm:", sys.hash_info.algorithm, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 619 | "64bit" if sys.maxsize > 2**32 else "32bit") |
Florent Xicluna | 0e62a14 | 2010-03-06 17:34:48 +0000 | [diff] [blame] | 620 | print("== ", os.getcwd()) |
Antoine Pitrou | 3c4402f | 2011-01-03 20:38:52 +0000 | [diff] [blame] | 621 | print("Testing with flags:", sys.flags) |
Florent Xicluna | 0e62a14 | 2010-03-06 17:34:48 +0000 | [diff] [blame] | 622 | |
R David Murray | b588f8d | 2011-03-24 14:42:58 -0400 | [diff] [blame] | 623 | # if testdir is set, then we are not running the python tests suite, so |
| 624 | # don't add default tests to be executed or skipped (pass empty values) |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 625 | if ns.testdir: |
| 626 | alltests = findtests(ns.testdir, list(), set()) |
R David Murray | b588f8d | 2011-03-24 14:42:58 -0400 | [diff] [blame] | 627 | else: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 628 | alltests = findtests(ns.testdir, stdtests, nottests) |
R David Murray | b588f8d | 2011-03-24 14:42:58 -0400 | [diff] [blame] | 629 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 630 | selected = tests or ns.args or alltests |
| 631 | if ns.single: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 632 | selected = selected[:1] |
R. David Murray | ef1992b | 2009-12-16 15:19:27 +0000 | [diff] [blame] | 633 | try: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 634 | next_single_test = alltests[alltests.index(selected[0])+1] |
R. David Murray | ef1992b | 2009-12-16 15:19:27 +0000 | [diff] [blame] | 635 | except IndexError: |
| 636 | next_single_test = None |
R David Murray | c3bf78a | 2012-10-27 17:07:05 -0400 | [diff] [blame] | 637 | # Remove all the selected tests that precede start if it's set. |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 638 | if ns.start: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 639 | try: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 640 | del selected[:selected.index(ns.start)] |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 641 | except ValueError: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 642 | print("Couldn't find starting test (%s), using all tests" % ns.start) |
| 643 | if ns.randomize: |
| 644 | if ns.random_seed is None: |
| 645 | ns.random_seed = random.randrange(10000000) |
| 646 | random.seed(ns.random_seed) |
| 647 | print("Using random seed", ns.random_seed) |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 648 | random.shuffle(selected) |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 649 | if ns.trace: |
Georg Brandl | 33c2881 | 2009-04-01 23:07:29 +0000 | [diff] [blame] | 650 | import trace, tempfile |
Vinay Sajip | 7ded1f0 | 2012-05-26 03:45:29 +0100 | [diff] [blame] | 651 | tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix, |
Georg Brandl | 33c2881 | 2009-04-01 23:07:29 +0000 | [diff] [blame] | 652 | tempfile.gettempdir()], |
Barry Warsaw | 3b6d025 | 2004-02-07 22:43:03 +0000 | [diff] [blame] | 653 | trace=False, count=True) |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 654 | |
Christian Heimes | b186d00 | 2008-03-18 15:15:01 +0000 | [diff] [blame] | 655 | test_times = [] |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 656 | support.verbose = ns.verbose # Tell tests to be moderately quiet |
| 657 | support.use_resources = ns.use_resources |
Guido van Rossum | 5796d26 | 2000-04-21 21:35:06 +0000 | [diff] [blame] | 658 | save_modules = sys.modules.keys() |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 659 | |
| 660 | def accumulate_result(test, result): |
| 661 | ok, test_time = result |
Victor Stinner | 0c886f7 | 2015-10-11 10:39:56 +0200 | [diff] [blame] | 662 | if ok not in (CHILD_ERROR, INTERRUPTED): |
| 663 | test_times.append((test_time, test)) |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 664 | if ok == PASSED: |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 665 | good.append(test) |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 666 | elif ok == FAILED: |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 667 | bad.append(test) |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 668 | elif ok == ENV_CHANGED: |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 669 | environment_changed.append(test) |
| 670 | elif ok == SKIPPED: |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 671 | skipped.append(test) |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 672 | elif ok == RESOURCE_DENIED: |
| 673 | skipped.append(test) |
| 674 | resource_denieds.append(test) |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 675 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 676 | if ns.forever: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 677 | def test_forever(tests=list(selected)): |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 678 | while True: |
| 679 | for test in tests: |
| 680 | yield test |
| 681 | if bad: |
| 682 | return |
| 683 | tests = test_forever() |
Georg Brandl | e8e02e3 | 2010-08-03 07:56:50 +0000 | [diff] [blame] | 684 | test_count = '' |
| 685 | test_count_width = 3 |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 686 | else: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 687 | tests = iter(selected) |
Georg Brandl | e8e02e3 | 2010-08-03 07:56:50 +0000 | [diff] [blame] | 688 | test_count = '/{}'.format(len(selected)) |
| 689 | test_count_width = len(test_count) - 1 |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 690 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 691 | if ns.use_mp: |
Victor Stinner | 45df820 | 2010-04-28 22:31:17 +0000 | [diff] [blame] | 692 | try: |
| 693 | from threading import Thread |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 694 | except ImportError: |
Victor Stinner | 45df820 | 2010-04-28 22:31:17 +0000 | [diff] [blame] | 695 | print("Multiprocess option requires thread support") |
| 696 | sys.exit(2) |
Georg Brandl | 1b37e87 | 2010-03-14 10:45:50 +0000 | [diff] [blame] | 697 | from queue import Queue |
Antoine Pitrou | f9d0b12 | 2012-12-09 14:28:26 +0100 | [diff] [blame] | 698 | debug_output_pat = re.compile(r"\[\d+ refs, \d+ blocks\]$") |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 699 | output = Queue() |
Antoine Pitrou | 09f2e6f | 2012-07-26 00:45:19 +0200 | [diff] [blame] | 700 | pending = MultiprocessTests(tests) |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 701 | def work(): |
| 702 | # A worker thread. |
Neal Norwitz | 14ca327 | 2006-02-28 18:05:43 +0000 | [diff] [blame] | 703 | try: |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 704 | while True: |
| 705 | try: |
Antoine Pitrou | 09f2e6f | 2012-07-26 00:45:19 +0200 | [diff] [blame] | 706 | test = next(pending) |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 707 | except StopIteration: |
R. David Murray | 2714460 | 2009-10-19 15:26:16 +0000 | [diff] [blame] | 708 | output.put((None, None, None, None)) |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 709 | return |
Eli Bendersky | 7f5c22c | 2013-09-02 08:57:21 -0700 | [diff] [blame] | 710 | retcode, stdout, stderr = run_test_in_subprocess(test, ns) |
R. David Murray | 2714460 | 2009-10-19 15:26:16 +0000 | [diff] [blame] | 711 | # Strip last refcount output line if it exists, since it |
| 712 | # comes from the shutdown of the interpreter in the subcommand. |
| 713 | stderr = debug_output_pat.sub("", stderr) |
| 714 | stdout, _, result = stdout.strip().rpartition("\n") |
Victor Stinner | 4b73988 | 2011-03-31 18:02:36 +0200 | [diff] [blame] | 715 | if retcode != 0: |
| 716 | result = (CHILD_ERROR, "Exit code %s" % retcode) |
| 717 | output.put((test, stdout.rstrip(), stderr.rstrip(), result)) |
| 718 | return |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 719 | if not result: |
| 720 | output.put((None, None, None, None)) |
| 721 | return |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 722 | result = json.loads(result) |
R. David Murray | 2714460 | 2009-10-19 15:26:16 +0000 | [diff] [blame] | 723 | output.put((test, stdout.rstrip(), stderr.rstrip(), result)) |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 724 | except BaseException: |
R. David Murray | 2714460 | 2009-10-19 15:26:16 +0000 | [diff] [blame] | 725 | output.put((None, None, None, None)) |
Neal Norwitz | 14ca327 | 2006-02-28 18:05:43 +0000 | [diff] [blame] | 726 | raise |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 727 | workers = [Thread(target=work) for i in range(ns.use_mp)] |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 728 | for worker in workers: |
| 729 | worker.start() |
| 730 | finished = 0 |
Georg Brandl | dee7b85 | 2010-08-02 18:59:52 +0000 | [diff] [blame] | 731 | test_index = 1 |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 732 | try: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 733 | while finished < ns.use_mp: |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 734 | test, stdout, stderr, result = output.get() |
| 735 | if test is None: |
| 736 | finished += 1 |
| 737 | continue |
Victor Stinner | a2a895c | 2011-05-23 23:14:05 +0200 | [diff] [blame] | 738 | accumulate_result(test, result) |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 739 | if not ns.quiet: |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 740 | if bad and not ns.pgo: |
| 741 | fmt = "[{1:{0}}{2}/{3}] {4}" |
| 742 | else: |
| 743 | fmt = "[{1:{0}}{2}] {4}" |
Ezio Melotti | 84f75c6 | 2011-05-24 01:00:10 +0300 | [diff] [blame] | 744 | print(fmt.format( |
Victor Stinner | a2a895c | 2011-05-23 23:14:05 +0200 | [diff] [blame] | 745 | test_count_width, test_index, test_count, |
| 746 | len(bad), test)) |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 747 | if stdout: |
| 748 | print(stdout) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 749 | if stderr and not ns.pgo: |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 750 | print(stderr, file=sys.stderr) |
Antoine Pitrou | 8237258 | 2012-06-27 17:41:07 +0200 | [diff] [blame] | 751 | sys.stdout.flush() |
| 752 | sys.stderr.flush() |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 753 | if result[0] == INTERRUPTED: |
Victor Stinner | 2965011 | 2012-08-08 22:37:26 +0200 | [diff] [blame] | 754 | raise KeyboardInterrupt |
Victor Stinner | 4b73988 | 2011-03-31 18:02:36 +0200 | [diff] [blame] | 755 | if result[0] == CHILD_ERROR: |
Victor Stinner | 571e8fd | 2011-05-01 22:57:43 +0200 | [diff] [blame] | 756 | raise Exception("Child error on {}: {}".format(test, result[1])) |
Georg Brandl | dee7b85 | 2010-08-02 18:59:52 +0000 | [diff] [blame] | 757 | test_index += 1 |
R. David Murray | 7dc72cc | 2009-11-14 16:13:02 +0000 | [diff] [blame] | 758 | except KeyboardInterrupt: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 759 | interrupted = True |
Antoine Pitrou | 09f2e6f | 2012-07-26 00:45:19 +0200 | [diff] [blame] | 760 | pending.interrupted = True |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 761 | for worker in workers: |
| 762 | worker.join() |
| 763 | else: |
Georg Brandl | dee7b85 | 2010-08-02 18:59:52 +0000 | [diff] [blame] | 764 | for test_index, test in enumerate(tests, 1): |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 765 | if not ns.quiet: |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 766 | if bad and not ns.pgo: |
| 767 | fmt = "[{1:{0}}{2}/{3}] {4}" |
| 768 | else: |
| 769 | fmt = "[{1:{0}}{2}] {4}" |
Ezio Melotti | 84f75c6 | 2011-05-24 01:00:10 +0300 | [diff] [blame] | 770 | print(fmt.format( |
Victor Stinner | a2a895c | 2011-05-23 23:14:05 +0200 | [diff] [blame] | 771 | test_count_width, test_index, test_count, len(bad), test)) |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 772 | sys.stdout.flush() |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 773 | if ns.trace: |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 774 | # If we're tracing code coverage, then we don't exit with status |
| 775 | # if on a false return value from main. |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 776 | tracer.runctx('runtest(test, ns.verbose, ns.quiet, timeout=ns.timeout)', |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 777 | globals=globals(), locals=vars()) |
Barry Warsaw | 3b6d025 | 2004-02-07 22:43:03 +0000 | [diff] [blame] | 778 | else: |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 779 | try: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 780 | result = runtest(test, ns.verbose, ns.quiet, |
Eli Bendersky | e8de296 | 2013-09-02 17:01:10 -0700 | [diff] [blame] | 781 | ns.huntrleaks, |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 782 | output_on_failure=ns.verbose3, |
| 783 | timeout=ns.timeout, failfast=ns.failfast, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 784 | match_tests=ns.match_tests, pgo=ns.pgo) |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 785 | accumulate_result(test, result) |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 786 | except KeyboardInterrupt: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 787 | interrupted = True |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 788 | break |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 789 | if ns.findleaks: |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 790 | gc.collect() |
| 791 | if gc.garbage: |
| 792 | print("Warning: test created", len(gc.garbage), end=' ') |
| 793 | print("uncollectable object(s).") |
| 794 | # move the uncollectable objects somewhere so we don't see |
| 795 | # them again |
| 796 | found_garbage.extend(gc.garbage) |
| 797 | del gc.garbage[:] |
| 798 | # Unload the newly imported modules (best effort finalization) |
| 799 | for module in sys.modules.keys(): |
| 800 | if module not in save_modules and module.startswith("test."): |
| 801 | support.unload(module) |
Jeremy Hylton | 7a1ea0e | 2001-10-17 13:45:28 +0000 | [diff] [blame] | 802 | |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 803 | if interrupted and not ns.pgo: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 804 | # print a newline after ^C |
| 805 | print() |
| 806 | print("Test suite interrupted by signal SIGINT.") |
| 807 | omitted = set(selected) - set(good) - set(bad) - set(skipped) |
| 808 | print(count(len(omitted), "test"), "omitted:") |
| 809 | printlist(omitted) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 810 | if good and not ns.quiet and not ns.pgo: |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 811 | if not bad and not skipped and not interrupted and len(good) > 1: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 812 | print("All", end=' ') |
| 813 | print(count(len(good), "test"), "OK.") |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 814 | if ns.print_slow: |
Christian Heimes | b186d00 | 2008-03-18 15:15:01 +0000 | [diff] [blame] | 815 | test_times.sort(reverse=True) |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 816 | print("10 slowest tests:") |
Christian Heimes | b186d00 | 2008-03-18 15:15:01 +0000 | [diff] [blame] | 817 | for time, test in test_times[:10]: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 818 | print("%s: %.1fs" % (test, time)) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 819 | if bad and not ns.pgo: |
Zachary Ware | 9a47ed8 | 2015-08-08 22:03:27 -0500 | [diff] [blame] | 820 | print(count(len(bad), "test"), "failed:") |
| 821 | printlist(bad) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 822 | if environment_changed and not ns.pgo: |
Vinay Sajip | f959618 | 2012-03-02 01:01:13 +0000 | [diff] [blame] | 823 | print("{} altered the execution environment:".format( |
| 824 | count(len(environment_changed), "test"))) |
| 825 | printlist(environment_changed) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 826 | if skipped and not ns.quiet and not ns.pgo: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 827 | print(count(len(skipped), "test"), "skipped:") |
Tim Peters | a45da92 | 2001-08-12 03:45:50 +0000 | [diff] [blame] | 828 | printlist(skipped) |
Barry Warsaw | e11e3de | 1999-01-28 19:51:51 +0000 | [diff] [blame] | 829 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 830 | if ns.verbose2 and bad: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 831 | print("Re-running failed tests in verbose mode") |
Zachary Ware | 920a335 | 2015-08-04 21:54:54 -0500 | [diff] [blame] | 832 | for test in bad[:]: |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 833 | if not ns.pgo: |
| 834 | print("Re-running test %r in verbose mode" % test) |
Tim Peters | 922dd7d | 2006-03-10 23:37:10 +0000 | [diff] [blame] | 835 | sys.stdout.flush() |
Martin v. Löwis | 04824ce | 2006-03-10 21:26:16 +0000 | [diff] [blame] | 836 | try: |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 837 | ns.verbose = True |
Eli Bendersky | e8de296 | 2013-09-02 17:01:10 -0700 | [diff] [blame] | 838 | ok = runtest(test, True, ns.quiet, ns.huntrleaks, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 839 | timeout=ns.timeout, pgo=ns.pgo) |
Martin v. Löwis | 04824ce | 2006-03-10 21:26:16 +0000 | [diff] [blame] | 840 | except KeyboardInterrupt: |
| 841 | # print a newline separate from the ^C |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 842 | print() |
Martin v. Löwis | 04824ce | 2006-03-10 21:26:16 +0000 | [diff] [blame] | 843 | break |
Zachary Ware | 920a335 | 2015-08-04 21:54:54 -0500 | [diff] [blame] | 844 | else: |
| 845 | if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: |
| 846 | bad.remove(test) |
| 847 | else: |
| 848 | if bad: |
| 849 | print(count(len(bad), 'test'), "failed again:") |
| 850 | printlist(bad) |
Martin v. Löwis | 04824ce | 2006-03-10 21:26:16 +0000 | [diff] [blame] | 851 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 852 | if ns.single: |
R. David Murray | ef1992b | 2009-12-16 15:19:27 +0000 | [diff] [blame] | 853 | if next_single_test: |
| 854 | with open(filename, 'w') as fp: |
| 855 | fp.write(next_single_test + '\n') |
Barry Warsaw | e11e3de | 1999-01-28 19:51:51 +0000 | [diff] [blame] | 856 | else: |
| 857 | os.unlink(filename) |
| 858 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 859 | if ns.trace: |
Barry Warsaw | 3b6d025 | 2004-02-07 22:43:03 +0000 | [diff] [blame] | 860 | r = tracer.results() |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 861 | r.write_results(show_missing=True, summary=True, coverdir=ns.coverdir) |
Barry Warsaw | 3b6d025 | 2004-02-07 22:43:03 +0000 | [diff] [blame] | 862 | |
Serhiy Storchaka | 64f7c4e | 2013-08-29 12:26:23 +0300 | [diff] [blame] | 863 | if ns.runleaks: |
Skip Montanaro | 0179a18 | 2004-06-06 15:53:18 +0000 | [diff] [blame] | 864 | os.system("leaks %d" % os.getpid()) |
| 865 | |
Florent Xicluna | d6995eb | 2010-03-30 19:43:09 +0000 | [diff] [blame] | 866 | sys.exit(len(bad) > 0 or interrupted) |
Barry Warsaw | 08fca52 | 2001-08-20 22:33:46 +0000 | [diff] [blame] | 867 | |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 868 | |
R David Murray | b588f8d | 2011-03-24 14:42:58 -0400 | [diff] [blame] | 869 | # small set of tests to determine if we have a basically functioning interpreter |
| 870 | # (i.e. if any of these fail, then anything else is likely to follow) |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 871 | STDTESTS = [ |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 872 | 'test_grammar', |
| 873 | 'test_opcodes', |
Guido van Rossum | d8faa36 | 2007-04-27 19:54:29 +0000 | [diff] [blame] | 874 | 'test_dict', |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 875 | 'test_builtin', |
| 876 | 'test_exceptions', |
| 877 | 'test_types', |
Collin Winter | 7afaa88 | 2007-03-08 19:54:43 +0000 | [diff] [blame] | 878 | 'test_unittest', |
| 879 | 'test_doctest', |
| 880 | 'test_doctest2', |
Eli Bendersky | d18a047 | 2011-07-27 20:21:45 +0300 | [diff] [blame] | 881 | 'test_support' |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 882 | ] |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 883 | |
R David Murray | b588f8d | 2011-03-24 14:42:58 -0400 | [diff] [blame] | 884 | # set of tests that we don't want to be executed when using regrtest |
R David Murray | 5764830 | 2011-03-24 14:57:05 -0400 | [diff] [blame] | 885 | NOTTESTS = set() |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 886 | |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 887 | def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 888 | """Return a list of all applicable test modules.""" |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 889 | testdir = findtestdir(testdir) |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 890 | names = os.listdir(testdir) |
| 891 | tests = [] |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 892 | others = set(stdtests) | nottests |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 893 | for name in names: |
R David Murray | 661720e | 2011-03-21 15:14:34 -0400 | [diff] [blame] | 894 | mod, ext = os.path.splitext(name) |
| 895 | if mod[:5] == "test_" and ext in (".py", "") and mod not in others: |
| 896 | tests.append(mod) |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 897 | return stdtests + sorted(tests) |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 898 | |
Antoine Pitrou | 09f2e6f | 2012-07-26 00:45:19 +0200 | [diff] [blame] | 899 | # We do not use a generator so multiple threads can call next(). |
| 900 | class MultiprocessTests(object): |
| 901 | |
| 902 | """A thread-safe iterator over tests for multiprocess mode.""" |
| 903 | |
| 904 | def __init__(self, tests): |
| 905 | self.interrupted = False |
| 906 | self.lock = threading.Lock() |
| 907 | self.tests = tests |
| 908 | |
| 909 | def __iter__(self): |
| 910 | return self |
| 911 | |
| 912 | def __next__(self): |
| 913 | with self.lock: |
| 914 | if self.interrupted: |
| 915 | raise StopIteration('tests interrupted') |
| 916 | return next(self.tests) |
| 917 | |
Victor Stinner | f58087b | 2010-05-02 17:24:51 +0000 | [diff] [blame] | 918 | def replace_stdout(): |
| 919 | """Set stdout encoder error handler to backslashreplace (as stderr error |
| 920 | handler) to avoid UnicodeEncodeError when printing a traceback""" |
Victor Stinner | 4b2b43d | 2011-01-05 03:54:26 +0000 | [diff] [blame] | 921 | import atexit |
| 922 | |
Victor Stinner | f58087b | 2010-05-02 17:24:51 +0000 | [diff] [blame] | 923 | stdout = sys.stdout |
| 924 | sys.stdout = open(stdout.fileno(), 'w', |
| 925 | encoding=stdout.encoding, |
Victor Stinner | 4b2b43d | 2011-01-05 03:54:26 +0000 | [diff] [blame] | 926 | errors="backslashreplace", |
Victor Stinner | be62103 | 2011-05-25 02:01:55 +0200 | [diff] [blame] | 927 | closefd=False, |
| 928 | newline='\n') |
Victor Stinner | 4b2b43d | 2011-01-05 03:54:26 +0000 | [diff] [blame] | 929 | |
| 930 | def restore_stdout(): |
| 931 | sys.stdout.close() |
| 932 | sys.stdout = stdout |
| 933 | atexit.register(restore_stdout) |
Victor Stinner | f58087b | 2010-05-02 17:24:51 +0000 | [diff] [blame] | 934 | |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 935 | def runtest(test, verbose, quiet, |
Eli Bendersky | e8de296 | 2013-09-02 17:01:10 -0700 | [diff] [blame] | 936 | huntrleaks=False, use_resources=None, |
Antoine Pitrou | f83e4ac | 2011-07-29 23:57:10 +0200 | [diff] [blame] | 937 | output_on_failure=False, failfast=False, match_tests=None, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 938 | timeout=None, *, pgo=False): |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 939 | """Run a single test. |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 940 | |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 941 | test -- the name of the test |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 942 | verbose -- if true, print more messages |
Trent Mick | f29f47b | 2000-08-11 19:02:59 +0000 | [diff] [blame] | 943 | quiet -- if true, don't print 'skipped' messages (probably redundant) |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 944 | huntrleaks -- run multiple times to test for leaks; requires a debug |
| 945 | build; a triple corresponding to -R's three arguments |
Eli Bendersky | e5eebed | 2013-09-02 16:52:25 -0700 | [diff] [blame] | 946 | use_resources -- list of extra resources to use |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 947 | output_on_failure -- if true, display test output on failure |
Victor Stinner | 0cc8d59 | 2011-03-31 18:10:13 +0200 | [diff] [blame] | 948 | timeout -- dump the traceback and exit if a test takes more than |
| 949 | timeout seconds |
Eli Bendersky | e5eebed | 2013-09-02 16:52:25 -0700 | [diff] [blame] | 950 | failfast, match_tests -- See regrtest command-line flags for these. |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 951 | pgo -- if true, do not print unnecessary info when running the test |
| 952 | for Profile Guided Optimization build |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 953 | |
Eli Bendersky | e5eebed | 2013-09-02 16:52:25 -0700 | [diff] [blame] | 954 | Returns the tuple result, test_time, where result is one of the constants: |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 955 | INTERRUPTED KeyboardInterrupt when run under -j |
| 956 | RESOURCE_DENIED test skipped because resource denied |
| 957 | SKIPPED test skipped for some other reason |
| 958 | ENV_CHANGED test failed because it changed the execution environment |
| 959 | FAILED test failed |
| 960 | PASSED test passed |
Guido van Rossum | 6fd83b7 | 1998-08-01 17:04:08 +0000 | [diff] [blame] | 961 | """ |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 962 | if use_resources is not None: |
| 963 | support.use_resources = use_resources |
Victor Stinner | 3019688 | 2011-06-03 12:53:26 +0200 | [diff] [blame] | 964 | use_timeout = (timeout is not None) |
Victor Stinner | 7d648a0 | 2011-03-31 18:27:50 +0200 | [diff] [blame] | 965 | if use_timeout: |
Victor Stinner | 4de701b | 2013-06-17 20:27:10 +0200 | [diff] [blame] | 966 | faulthandler.dump_traceback_later(timeout, exit=True) |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 967 | try: |
Antoine Pitrou | b9c73e8 | 2011-07-29 23:53:38 +0200 | [diff] [blame] | 968 | support.match_tests = match_tests |
Antoine Pitrou | 216a3bc | 2011-07-23 22:33:39 +0200 | [diff] [blame] | 969 | if failfast: |
| 970 | support.failfast = True |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 971 | if output_on_failure: |
Victor Stinner | ea95de7 | 2011-06-29 15:34:48 +0200 | [diff] [blame] | 972 | support.verbose = True |
| 973 | |
| 974 | # Reuse the same instance to all calls to runtest(). Some |
| 975 | # tests keep a reference to sys.stdout or sys.stderr |
| 976 | # (eg. test_argparse). |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 977 | if runtest.stringio is None: |
Victor Stinner | fcc2a21 | 2011-06-29 20:01:29 +0200 | [diff] [blame] | 978 | stream = io.StringIO() |
| 979 | runtest.stringio = stream |
| 980 | else: |
| 981 | stream = runtest.stringio |
| 982 | stream.seek(0) |
| 983 | stream.truncate() |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 984 | |
| 985 | orig_stdout = sys.stdout |
Victor Stinner | a7c33e5 | 2011-06-29 13:00:54 +0200 | [diff] [blame] | 986 | orig_stderr = sys.stderr |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 987 | try: |
Victor Stinner | ea95de7 | 2011-06-29 15:34:48 +0200 | [diff] [blame] | 988 | sys.stdout = stream |
| 989 | sys.stderr = stream |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 990 | result = runtest_inner(test, verbose, quiet, huntrleaks, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 991 | display_failure=False, pgo=pgo) |
| 992 | if result[0] == FAILED and not pgo: |
Victor Stinner | ea95de7 | 2011-06-29 15:34:48 +0200 | [diff] [blame] | 993 | output = stream.getvalue() |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 994 | orig_stderr.write(output) |
| 995 | orig_stderr.flush() |
| 996 | finally: |
| 997 | sys.stdout = orig_stdout |
| 998 | sys.stderr = orig_stderr |
Victor Stinner | a7c33e5 | 2011-06-29 13:00:54 +0200 | [diff] [blame] | 999 | else: |
Victor Stinner | ea95de7 | 2011-06-29 15:34:48 +0200 | [diff] [blame] | 1000 | support.verbose = verbose # Tell tests to be moderately quiet |
Eli Bendersky | e8de296 | 2013-09-02 17:01:10 -0700 | [diff] [blame] | 1001 | result = runtest_inner(test, verbose, quiet, huntrleaks, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1002 | display_failure=not verbose, pgo=pgo) |
Antoine Pitrou | 293954d | 2011-03-23 23:01:49 +0100 | [diff] [blame] | 1003 | return result |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1004 | finally: |
Victor Stinner | 7d648a0 | 2011-03-31 18:27:50 +0200 | [diff] [blame] | 1005 | if use_timeout: |
Victor Stinner | 934676a | 2013-06-17 20:35:08 +0200 | [diff] [blame] | 1006 | faulthandler.cancel_dump_traceback_later() |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1007 | cleanup_test_droppings(test, verbose) |
Victor Stinner | 8313d6a | 2011-06-29 15:22:26 +0200 | [diff] [blame] | 1008 | runtest.stringio = None |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1009 | |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1010 | # Unit tests are supposed to leave the execution environment unchanged |
| 1011 | # once they complete. But sometimes tests have bugs, especially when |
| 1012 | # tests fail, and the changes to environment go on to mess up other |
| 1013 | # tests. This can cause issues with buildbot stability, since tests |
| 1014 | # are run in random order and so problems may appear to come and go. |
| 1015 | # There are a few things we can save and restore to mitigate this, and |
| 1016 | # the following context manager handles this task. |
| 1017 | |
| 1018 | class saved_test_environment: |
| 1019 | """Save bits of the test environment and restore them at block exit. |
| 1020 | |
| 1021 | with saved_test_environment(testname, verbose, quiet): |
| 1022 | #stuff |
| 1023 | |
| 1024 | Unless quiet is True, a warning is printed to stderr if any of |
| 1025 | the saved items was changed by the test. The attribute 'changed' |
| 1026 | is initially False, but is set to True if a change is detected. |
| 1027 | |
| 1028 | If verbose is more than 1, the before and after state of changed |
| 1029 | items is also printed. |
| 1030 | """ |
| 1031 | |
| 1032 | changed = False |
| 1033 | |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1034 | def __init__(self, testname, verbose=0, quiet=False, *, pgo=False): |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1035 | self.testname = testname |
| 1036 | self.verbose = verbose |
| 1037 | self.quiet = quiet |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1038 | self.pgo = pgo |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1039 | |
| 1040 | # To add things to save and restore, add a name XXX to the resources list |
| 1041 | # and add corresponding get_XXX/restore_XXX functions. get_XXX should |
| 1042 | # return the value to be saved and compared against a second call to the |
| 1043 | # get function when test execution completes. restore_XXX should accept |
| 1044 | # the saved value and restore the resource using it. It will be called if |
| 1045 | # and only if a change in the value is detected. |
| 1046 | # |
| 1047 | # Note: XXX will have any '.' replaced with '_' characters when determining |
| 1048 | # the corresponding method names. |
| 1049 | |
| 1050 | resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', |
Brett Cannon | 29c0e4f | 2010-03-20 22:22:57 +0000 | [diff] [blame] | 1051 | 'os.environ', 'sys.path', 'sys.path_hooks', '__import__', |
Nick Coghlan | 7bd5dbe | 2010-12-05 07:17:25 +0000 | [diff] [blame] | 1052 | 'warnings.filters', 'asyncore.socket_map', |
Ezio Melotti | 45763d0 | 2011-03-20 15:34:28 +0200 | [diff] [blame] | 1053 | 'logging._handlers', 'logging._handlerList', 'sys.gettrace', |
Richard Oudkerk | 83d7dea | 2013-08-29 12:51:11 +0100 | [diff] [blame] | 1054 | 'sys.warnoptions', |
| 1055 | # multiprocessing.process._cleanup() may release ref |
| 1056 | # to a thread, so check processes first. |
| 1057 | 'multiprocessing.process._dangling', 'threading._dangling', |
Éric Araujo | ec177c1 | 2012-06-24 03:27:43 -0400 | [diff] [blame] | 1058 | 'sysconfig._CONFIG_VARS', 'sysconfig._INSTALL_SCHEMES', |
Serhiy Storchaka | a3a100b | 2015-03-30 01:28:02 +0300 | [diff] [blame] | 1059 | 'files', 'locale', 'warnings.showwarning', |
Victor Stinner | 8ba2083 | 2016-03-29 13:33:35 +0200 | [diff] [blame] | 1060 | 'shutil_archive_formats', 'shutil_unpack_formats', |
Éric Araujo | 28df8de | 2011-09-19 05:10:45 +0200 | [diff] [blame] | 1061 | ) |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1062 | |
| 1063 | def get_sys_argv(self): |
| 1064 | return id(sys.argv), sys.argv, sys.argv[:] |
| 1065 | def restore_sys_argv(self, saved_argv): |
| 1066 | sys.argv = saved_argv[1] |
| 1067 | sys.argv[:] = saved_argv[2] |
| 1068 | |
| 1069 | def get_cwd(self): |
| 1070 | return os.getcwd() |
| 1071 | def restore_cwd(self, saved_cwd): |
| 1072 | os.chdir(saved_cwd) |
| 1073 | |
| 1074 | def get_sys_stdout(self): |
| 1075 | return sys.stdout |
| 1076 | def restore_sys_stdout(self, saved_stdout): |
| 1077 | sys.stdout = saved_stdout |
| 1078 | |
| 1079 | def get_sys_stderr(self): |
| 1080 | return sys.stderr |
| 1081 | def restore_sys_stderr(self, saved_stderr): |
| 1082 | sys.stderr = saved_stderr |
| 1083 | |
| 1084 | def get_sys_stdin(self): |
| 1085 | return sys.stdin |
| 1086 | def restore_sys_stdin(self, saved_stdin): |
| 1087 | sys.stdin = saved_stdin |
| 1088 | |
| 1089 | def get_os_environ(self): |
| 1090 | return id(os.environ), os.environ, dict(os.environ) |
| 1091 | def restore_os_environ(self, saved_environ): |
| 1092 | os.environ = saved_environ[1] |
| 1093 | os.environ.clear() |
| 1094 | os.environ.update(saved_environ[2]) |
| 1095 | |
| 1096 | def get_sys_path(self): |
| 1097 | return id(sys.path), sys.path, sys.path[:] |
| 1098 | def restore_sys_path(self, saved_path): |
| 1099 | sys.path = saved_path[1] |
| 1100 | sys.path[:] = saved_path[2] |
| 1101 | |
Brett Cannon | 055470a | 2010-02-19 15:57:10 +0000 | [diff] [blame] | 1102 | def get_sys_path_hooks(self): |
| 1103 | return id(sys.path_hooks), sys.path_hooks, sys.path_hooks[:] |
| 1104 | def restore_sys_path_hooks(self, saved_hooks): |
| 1105 | sys.path_hooks = saved_hooks[1] |
| 1106 | sys.path_hooks[:] = saved_hooks[2] |
| 1107 | |
Brett Cannon | 31f5929 | 2011-02-21 19:29:56 +0000 | [diff] [blame] | 1108 | def get_sys_gettrace(self): |
| 1109 | return sys.gettrace() |
| 1110 | def restore_sys_gettrace(self, trace_fxn): |
| 1111 | sys.settrace(trace_fxn) |
| 1112 | |
Brett Cannon | 055470a | 2010-02-19 15:57:10 +0000 | [diff] [blame] | 1113 | def get___import__(self): |
Brett Cannon | 4507190 | 2010-06-14 22:22:54 +0000 | [diff] [blame] | 1114 | return builtins.__import__ |
Brett Cannon | 055470a | 2010-02-19 15:57:10 +0000 | [diff] [blame] | 1115 | def restore___import__(self, import_): |
Brett Cannon | 4507190 | 2010-06-14 22:22:54 +0000 | [diff] [blame] | 1116 | builtins.__import__ = import_ |
Brett Cannon | 055470a | 2010-02-19 15:57:10 +0000 | [diff] [blame] | 1117 | |
Brett Cannon | 29c0e4f | 2010-03-20 22:22:57 +0000 | [diff] [blame] | 1118 | def get_warnings_filters(self): |
| 1119 | return id(warnings.filters), warnings.filters, warnings.filters[:] |
| 1120 | def restore_warnings_filters(self, saved_filters): |
| 1121 | warnings.filters = saved_filters[1] |
| 1122 | warnings.filters[:] = saved_filters[2] |
| 1123 | |
Antoine Pitrou | b14ac8c | 2010-08-16 00:28:05 +0000 | [diff] [blame] | 1124 | def get_asyncore_socket_map(self): |
| 1125 | asyncore = sys.modules.get('asyncore') |
Antoine Pitrou | aa87965 | 2010-10-29 11:54:38 +0000 | [diff] [blame] | 1126 | # XXX Making a copy keeps objects alive until __exit__ gets called. |
| 1127 | return asyncore and asyncore.socket_map.copy() or {} |
Antoine Pitrou | b14ac8c | 2010-08-16 00:28:05 +0000 | [diff] [blame] | 1128 | def restore_asyncore_socket_map(self, saved_map): |
| 1129 | asyncore = sys.modules.get('asyncore') |
| 1130 | if asyncore is not None: |
Antoine Pitrou | aa87965 | 2010-10-29 11:54:38 +0000 | [diff] [blame] | 1131 | asyncore.close_all(ignore_all=True) |
Antoine Pitrou | b14ac8c | 2010-08-16 00:28:05 +0000 | [diff] [blame] | 1132 | asyncore.socket_map.update(saved_map) |
| 1133 | |
Éric Araujo | ff91306 | 2011-11-29 16:45:07 +0100 | [diff] [blame] | 1134 | def get_shutil_archive_formats(self): |
| 1135 | # we could call get_archives_formats() but that only returns the |
| 1136 | # registry keys; we want to check the values too (the functions that |
| 1137 | # are registered) |
| 1138 | return shutil._ARCHIVE_FORMATS, shutil._ARCHIVE_FORMATS.copy() |
| 1139 | def restore_shutil_archive_formats(self, saved): |
| 1140 | shutil._ARCHIVE_FORMATS = saved[0] |
| 1141 | shutil._ARCHIVE_FORMATS.clear() |
| 1142 | shutil._ARCHIVE_FORMATS.update(saved[1]) |
| 1143 | |
| 1144 | def get_shutil_unpack_formats(self): |
| 1145 | return shutil._UNPACK_FORMATS, shutil._UNPACK_FORMATS.copy() |
| 1146 | def restore_shutil_unpack_formats(self, saved): |
| 1147 | shutil._UNPACK_FORMATS = saved[0] |
| 1148 | shutil._UNPACK_FORMATS.clear() |
| 1149 | shutil._UNPACK_FORMATS.update(saved[1]) |
| 1150 | |
Nick Coghlan | 7d81975 | 2010-12-05 06:45:03 +0000 | [diff] [blame] | 1151 | def get_logging__handlers(self): |
| 1152 | # _handlers is a WeakValueDictionary |
Nick Coghlan | 7bd5dbe | 2010-12-05 07:17:25 +0000 | [diff] [blame] | 1153 | return id(logging._handlers), logging._handlers, logging._handlers.copy() |
Nick Coghlan | 7d81975 | 2010-12-05 06:45:03 +0000 | [diff] [blame] | 1154 | def restore_logging__handlers(self, saved_handlers): |
| 1155 | # Can't easily revert the logging state |
| 1156 | pass |
| 1157 | |
Nick Coghlan | 7bd5dbe | 2010-12-05 07:17:25 +0000 | [diff] [blame] | 1158 | def get_logging__handlerList(self): |
| 1159 | # _handlerList is a list of weakrefs to handlers |
| 1160 | return id(logging._handlerList), logging._handlerList, logging._handlerList[:] |
| 1161 | def restore_logging__handlerList(self, saved_handlerList): |
| 1162 | # Can't easily revert the logging state |
| 1163 | pass |
| 1164 | |
Ezio Melotti | 0123e05 | 2011-03-20 15:09:26 +0200 | [diff] [blame] | 1165 | def get_sys_warnoptions(self): |
| 1166 | return id(sys.warnoptions), sys.warnoptions, sys.warnoptions[:] |
| 1167 | def restore_sys_warnoptions(self, saved_options): |
| 1168 | sys.warnoptions = saved_options[1] |
| 1169 | sys.warnoptions[:] = saved_options[2] |
| 1170 | |
Antoine Pitrou | c081c0c | 2011-07-15 22:12:24 +0200 | [diff] [blame] | 1171 | # Controlling dangling references to Thread objects can make it easier |
| 1172 | # to track reference leaks. |
| 1173 | def get_threading__dangling(self): |
| 1174 | if not threading: |
| 1175 | return None |
| 1176 | # This copies the weakrefs without making any strong reference |
| 1177 | return threading._dangling.copy() |
| 1178 | def restore_threading__dangling(self, saved): |
| 1179 | if not threading: |
| 1180 | return |
| 1181 | threading._dangling.clear() |
| 1182 | threading._dangling.update(saved) |
| 1183 | |
| 1184 | # Same for Process objects |
| 1185 | def get_multiprocessing_process__dangling(self): |
| 1186 | if not multiprocessing: |
| 1187 | return None |
Richard Oudkerk | 83d7dea | 2013-08-29 12:51:11 +0100 | [diff] [blame] | 1188 | # Unjoined process objects can survive after process exits |
| 1189 | multiprocessing.process._cleanup() |
Antoine Pitrou | c081c0c | 2011-07-15 22:12:24 +0200 | [diff] [blame] | 1190 | # This copies the weakrefs without making any strong reference |
| 1191 | return multiprocessing.process._dangling.copy() |
| 1192 | def restore_multiprocessing_process__dangling(self, saved): |
| 1193 | if not multiprocessing: |
| 1194 | return |
| 1195 | multiprocessing.process._dangling.clear() |
| 1196 | multiprocessing.process._dangling.update(saved) |
| 1197 | |
Éric Araujo | 28df8de | 2011-09-19 05:10:45 +0200 | [diff] [blame] | 1198 | def get_sysconfig__CONFIG_VARS(self): |
| 1199 | # make sure the dict is initialized |
| 1200 | sysconfig.get_config_var('prefix') |
| 1201 | return (id(sysconfig._CONFIG_VARS), sysconfig._CONFIG_VARS, |
| 1202 | dict(sysconfig._CONFIG_VARS)) |
| 1203 | def restore_sysconfig__CONFIG_VARS(self, saved): |
| 1204 | sysconfig._CONFIG_VARS = saved[1] |
| 1205 | sysconfig._CONFIG_VARS.clear() |
| 1206 | sysconfig._CONFIG_VARS.update(saved[2]) |
| 1207 | |
Éric Araujo | ec177c1 | 2012-06-24 03:27:43 -0400 | [diff] [blame] | 1208 | def get_sysconfig__INSTALL_SCHEMES(self): |
| 1209 | return (id(sysconfig._INSTALL_SCHEMES), sysconfig._INSTALL_SCHEMES, |
| 1210 | sysconfig._INSTALL_SCHEMES.copy()) |
| 1211 | def restore_sysconfig__INSTALL_SCHEMES(self, saved): |
| 1212 | sysconfig._INSTALL_SCHEMES = saved[1] |
| 1213 | sysconfig._INSTALL_SCHEMES.clear() |
| 1214 | sysconfig._INSTALL_SCHEMES.update(saved[2]) |
Éric Araujo | 28df8de | 2011-09-19 05:10:45 +0200 | [diff] [blame] | 1215 | |
Serhiy Storchaka | a3a100b | 2015-03-30 01:28:02 +0300 | [diff] [blame] | 1216 | def get_files(self): |
| 1217 | return sorted(fn + ('/' if os.path.isdir(fn) else '') |
| 1218 | for fn in os.listdir()) |
| 1219 | def restore_files(self, saved_value): |
| 1220 | fn = support.TESTFN |
| 1221 | if fn not in saved_value and (fn + '/') not in saved_value: |
| 1222 | if os.path.isfile(fn): |
| 1223 | support.unlink(fn) |
| 1224 | elif os.path.isdir(fn): |
| 1225 | support.rmtree(fn) |
Éric Araujo | 28df8de | 2011-09-19 05:10:45 +0200 | [diff] [blame] | 1226 | |
Victor Stinner | d9ccf7f | 2013-06-17 20:40:05 +0200 | [diff] [blame] | 1227 | _lc = [getattr(locale, lc) for lc in dir(locale) |
Victor Stinner | 546ccf0 | 2013-06-17 21:28:14 +0200 | [diff] [blame] | 1228 | if lc.startswith('LC_')] |
Brett Cannon | 49e835b | 2013-04-01 14:11:37 -0400 | [diff] [blame] | 1229 | def get_locale(self): |
| 1230 | pairings = [] |
| 1231 | for lc in self._lc: |
| 1232 | try: |
Victor Stinner | 546ccf0 | 2013-06-17 21:28:14 +0200 | [diff] [blame] | 1233 | pairings.append((lc, locale.setlocale(lc, None))) |
Victor Stinner | d9ccf7f | 2013-06-17 20:40:05 +0200 | [diff] [blame] | 1234 | except (TypeError, ValueError): |
Brett Cannon | 49e835b | 2013-04-01 14:11:37 -0400 | [diff] [blame] | 1235 | continue |
| 1236 | return pairings |
| 1237 | def restore_locale(self, saved): |
| 1238 | for lc, setting in saved: |
| 1239 | locale.setlocale(lc, setting) |
| 1240 | |
Brett Cannon | 6d26eba | 2013-06-16 15:20:48 -0400 | [diff] [blame] | 1241 | def get_warnings_showwarning(self): |
| 1242 | return warnings.showwarning |
| 1243 | def restore_warnings_showwarning(self, fxn): |
| 1244 | warnings.showwarning = fxn |
| 1245 | |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1246 | def resource_info(self): |
| 1247 | for name in self.resources: |
| 1248 | method_suffix = name.replace('.', '_') |
| 1249 | get_name = 'get_' + method_suffix |
| 1250 | restore_name = 'restore_' + method_suffix |
| 1251 | yield name, getattr(self, get_name), getattr(self, restore_name) |
| 1252 | |
| 1253 | def __enter__(self): |
| 1254 | self.saved_values = dict((name, get()) for name, get, restore |
| 1255 | in self.resource_info()) |
| 1256 | return self |
| 1257 | |
| 1258 | def __exit__(self, exc_type, exc_val, exc_tb): |
Antoine Pitrou | aa87965 | 2010-10-29 11:54:38 +0000 | [diff] [blame] | 1259 | saved_values = self.saved_values |
| 1260 | del self.saved_values |
Martin Panter | 68c1f1e | 2016-08-20 06:50:14 +0000 | [diff] [blame] | 1261 | support.gc_collect() # Some resources use weak references |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1262 | for name, get, restore in self.resource_info(): |
| 1263 | current = get() |
Antoine Pitrou | aa87965 | 2010-10-29 11:54:38 +0000 | [diff] [blame] | 1264 | original = saved_values.pop(name) |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1265 | # Check for changes to the resource's value |
| 1266 | if current != original: |
| 1267 | self.changed = True |
| 1268 | restore(original) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1269 | if not self.quiet and not self.pgo: |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1270 | print("Warning -- {} was modified by {}".format( |
| 1271 | name, self.testname), |
| 1272 | file=sys.stderr) |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1273 | if self.verbose > 1 and not self.pgo: |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1274 | print(" Before: {}\n After: {} ".format( |
| 1275 | original, current), |
| 1276 | file=sys.stderr) |
| 1277 | return False |
| 1278 | |
| 1279 | |
Victor Stinner | a7c33e5 | 2011-06-29 13:00:54 +0200 | [diff] [blame] | 1280 | def runtest_inner(test, verbose, quiet, |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1281 | huntrleaks=False, display_failure=True, pgo=False): |
Benjamin Peterson | ee8712c | 2008-05-20 21:35:26 +0000 | [diff] [blame] | 1282 | support.unload(test) |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1283 | |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 1284 | test_time = 0.0 |
Collin Winter | a5503d5 | 2009-05-15 01:20:21 +0000 | [diff] [blame] | 1285 | refleak = False # True if the test leaked references. |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 1286 | try: |
R. David Murray | 0bc11ae | 2009-10-18 22:18:17 +0000 | [diff] [blame] | 1287 | if test.startswith('test.'): |
| 1288 | abstest = test |
| 1289 | else: |
| 1290 | # Always import it from the test package |
| 1291 | abstest = 'test.' + test |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1292 | with saved_test_environment(test, verbose, quiet, pgo=pgo) as environment: |
R. David Murray | 0bc11ae | 2009-10-18 22:18:17 +0000 | [diff] [blame] | 1293 | start_time = time.time() |
Brett Cannon | 613cf25 | 2012-11-14 13:42:51 -0500 | [diff] [blame] | 1294 | the_module = importlib.import_module(abstest) |
R David Murray | 78fc25c | 2012-04-09 08:55:42 -0400 | [diff] [blame] | 1295 | # If the test has a test_main, that will run the appropriate |
| 1296 | # tests. If not, use normal unittest test loading. |
| 1297 | test_runner = getattr(the_module, "test_main", None) |
| 1298 | if test_runner is None: |
Zachary Ware | 69fb6a4 | 2014-08-04 11:15:10 -0500 | [diff] [blame] | 1299 | def test_runner(): |
| 1300 | loader = unittest.TestLoader() |
| 1301 | tests = loader.loadTestsFromModule(the_module) |
Victor Stinner | 5d57539 | 2015-01-06 14:05:03 +0100 | [diff] [blame] | 1302 | for error in loader.errors: |
| 1303 | print(error, file=sys.stderr) |
| 1304 | if loader.errors: |
| 1305 | raise Exception("errors while loading tests") |
Zachary Ware | 69fb6a4 | 2014-08-04 11:15:10 -0500 | [diff] [blame] | 1306 | support.run_unittest(tests) |
R David Murray | 78fc25c | 2012-04-09 08:55:42 -0400 | [diff] [blame] | 1307 | test_runner() |
R. David Murray | 0bc11ae | 2009-10-18 22:18:17 +0000 | [diff] [blame] | 1308 | if huntrleaks: |
Eli Bendersky | e5eebed | 2013-09-02 16:52:25 -0700 | [diff] [blame] | 1309 | refleak = dash_R(the_module, test, test_runner, huntrleaks) |
R. David Murray | 0bc11ae | 2009-10-18 22:18:17 +0000 | [diff] [blame] | 1310 | test_time = time.time() - start_time |
Benjamin Peterson | ee8712c | 2008-05-20 21:35:26 +0000 | [diff] [blame] | 1311 | except support.ResourceDenied as msg: |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1312 | if not quiet and not pgo: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1313 | print(test, "skipped --", msg) |
Fred Drake | 9a0db07 | 2003-02-03 15:19:30 +0000 | [diff] [blame] | 1314 | sys.stdout.flush() |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 1315 | return RESOURCE_DENIED, test_time |
R. David Murray | a21e4ca | 2009-03-31 23:16:50 +0000 | [diff] [blame] | 1316 | except unittest.SkipTest as msg: |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1317 | if not quiet and not pgo: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1318 | print(test, "skipped --", msg) |
Guido van Rossum | 3cda93e | 2002-09-13 21:28:03 +0000 | [diff] [blame] | 1319 | sys.stdout.flush() |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 1320 | return SKIPPED, test_time |
Fred Drake | fe5c22a | 2000-08-18 16:04:05 +0000 | [diff] [blame] | 1321 | except KeyboardInterrupt: |
| 1322 | raise |
Benjamin Peterson | ee8712c | 2008-05-20 21:35:26 +0000 | [diff] [blame] | 1323 | except support.TestFailed as msg: |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1324 | if not pgo: |
| 1325 | if display_failure: |
| 1326 | print("test", test, "failed --", msg, file=sys.stderr) |
| 1327 | else: |
| 1328 | print("test", test, "failed", file=sys.stderr) |
R. David Murray | 11cabcf | 2010-09-29 01:08:05 +0000 | [diff] [blame] | 1329 | sys.stderr.flush() |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 1330 | return FAILED, test_time |
Guido van Rossum | 9e48b27 | 1997-07-16 01:56:13 +0000 | [diff] [blame] | 1331 | except: |
Antoine Pitrou | 779a5b0 | 2011-03-21 19:55:16 +0100 | [diff] [blame] | 1332 | msg = traceback.format_exc() |
Brett Cannon | be7c163 | 2015-10-02 16:16:44 -0700 | [diff] [blame] | 1333 | if not pgo: |
| 1334 | print("test", test, "crashed --", msg, file=sys.stderr) |
R. David Murray | 11cabcf | 2010-09-29 01:08:05 +0000 | [diff] [blame] | 1335 | sys.stderr.flush() |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 1336 | return FAILED, test_time |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 1337 | else: |
Collin Winter | a5503d5 | 2009-05-15 01:20:21 +0000 | [diff] [blame] | 1338 | if refleak: |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 1339 | return FAILED, test_time |
Nick Coghlan | 6ead552 | 2009-10-18 13:19:33 +0000 | [diff] [blame] | 1340 | if environment.changed: |
Benjamin Peterson | a0dfa82 | 2009-11-13 02:25:08 +0000 | [diff] [blame] | 1341 | return ENV_CHANGED, test_time |
| 1342 | return PASSED, test_time |
Guido van Rossum | 0fcca4e | 2001-09-21 20:31:52 +0000 | [diff] [blame] | 1343 | |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1344 | def cleanup_test_droppings(testname, verbose): |
| 1345 | import shutil |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1346 | import stat |
Antoine Pitrou | c14efc4 | 2010-10-29 19:34:45 +0000 | [diff] [blame] | 1347 | import gc |
| 1348 | |
| 1349 | # First kill any dangling references to open files etc. |
| 1350 | # This can also issue some ResourceWarnings which would otherwise get |
Antoine Pitrou | 2b40efd | 2010-10-29 19:36:37 +0000 | [diff] [blame] | 1351 | # triggered during the following test run, and possibly produce failures. |
Antoine Pitrou | c14efc4 | 2010-10-29 19:34:45 +0000 | [diff] [blame] | 1352 | gc.collect() |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1353 | |
| 1354 | # Try to clean up junk commonly left behind. While tests shouldn't leave |
| 1355 | # any files or directories behind, when a test fails that can be tedious |
| 1356 | # for it to arrange. The consequences can be especially nasty on Windows, |
| 1357 | # since if a test leaves a file open, it cannot be deleted by name (while |
| 1358 | # there's nothing we can do about that here either, we can display the |
| 1359 | # name of the offending test, which is a real help). |
Benjamin Peterson | ee8712c | 2008-05-20 21:35:26 +0000 | [diff] [blame] | 1360 | for name in (support.TESTFN, |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1361 | "db_home", |
| 1362 | ): |
| 1363 | if not os.path.exists(name): |
| 1364 | continue |
| 1365 | |
| 1366 | if os.path.isdir(name): |
| 1367 | kind, nuker = "directory", shutil.rmtree |
| 1368 | elif os.path.isfile(name): |
| 1369 | kind, nuker = "file", os.unlink |
| 1370 | else: |
| 1371 | raise SystemError("os.path says %r exists but is neither " |
| 1372 | "directory nor file" % name) |
| 1373 | |
| 1374 | if verbose: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1375 | print("%r left behind %s %r" % (testname, kind, name)) |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1376 | try: |
Benjamin Peterson | 25c95f1 | 2009-05-08 20:42:26 +0000 | [diff] [blame] | 1377 | # if we have chmod, fix possible permissions problems |
| 1378 | # that might prevent cleanup |
| 1379 | if (hasattr(os, 'chmod')): |
| 1380 | os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1381 | nuker(name) |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1382 | except Exception as msg: |
| 1383 | print(("%r left behind %s %r and it couldn't be " |
| 1384 | "removed: %s" % (testname, kind, name, msg)), file=sys.stderr) |
Thomas Wouters | 0e3f591 | 2006-08-11 14:57:12 +0000 | [diff] [blame] | 1385 | |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1386 | def dash_R(the_module, test, indirect_test, huntrleaks): |
Collin Winter | a5503d5 | 2009-05-15 01:20:21 +0000 | [diff] [blame] | 1387 | """Run a test multiple times, looking for reference leaks. |
| 1388 | |
| 1389 | Returns: |
| 1390 | False if the test didn't leak references; True if we detected refleaks. |
| 1391 | """ |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1392 | # This code is hackish and inelegant, but it seems to do the job. |
Raymond Hettinger | 158c9c2 | 2011-02-22 00:41:50 +0000 | [diff] [blame] | 1393 | import copyreg |
| 1394 | import collections.abc |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1395 | |
| 1396 | if not hasattr(sys, 'gettotalrefcount'): |
| 1397 | raise Exception("Tracking reference leaks requires a debug build " |
| 1398 | "of Python") |
| 1399 | |
| 1400 | # Save current values for dash_R_cleanup() to restore. |
| 1401 | fs = warnings.filters[:] |
Alexandre Vassalotti | f7fa63d | 2008-05-11 08:55:36 +0000 | [diff] [blame] | 1402 | ps = copyreg.dispatch_table.copy() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1403 | pic = sys.path_importer_cache.copy() |
Nick Coghlan | 260bd3e | 2009-11-16 06:49:25 +0000 | [diff] [blame] | 1404 | try: |
| 1405 | import zipimport |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 1406 | except ImportError: |
Benjamin Peterson | cf62603 | 2014-02-16 14:52:01 -0500 | [diff] [blame] | 1407 | zdc = None # Run unmodified on platforms without zipimport support |
Nick Coghlan | 260bd3e | 2009-11-16 06:49:25 +0000 | [diff] [blame] | 1408 | else: |
| 1409 | zdc = zipimport._zip_directory_cache.copy() |
Christian Heimes | 9385266 | 2007-12-01 12:22:32 +0000 | [diff] [blame] | 1410 | abcs = {} |
Raymond Hettinger | 158c9c2 | 2011-02-22 00:41:50 +0000 | [diff] [blame] | 1411 | for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: |
Christian Heimes | be5b30b | 2008-03-03 19:18:51 +0000 | [diff] [blame] | 1412 | if not isabstract(abc): |
Christian Heimes | 9385266 | 2007-12-01 12:22:32 +0000 | [diff] [blame] | 1413 | continue |
| 1414 | for obj in abc.__subclasses__() + [abc]: |
| 1415 | abcs[obj] = obj._abc_registry.copy() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1416 | |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1417 | nwarmup, ntracked, fname = huntrleaks |
Ezio Melotti | 184bdfb | 2010-02-18 09:37:05 +0000 | [diff] [blame] | 1418 | fname = os.path.join(support.SAVEDCWD, fname) |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1419 | repcount = nwarmup + ntracked |
Antoine Pitrou | f9d0b12 | 2012-12-09 14:28:26 +0100 | [diff] [blame] | 1420 | rc_deltas = [0] * repcount |
| 1421 | alloc_deltas = [0] * repcount |
| 1422 | |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1423 | print("beginning", repcount, "repetitions", file=sys.stderr) |
| 1424 | print(("1234567890"*(repcount//10 + 1))[:repcount], file=sys.stderr) |
Antoine Pitrou | 8890954 | 2009-06-29 13:54:42 +0000 | [diff] [blame] | 1425 | sys.stderr.flush() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1426 | for i in range(repcount): |
R David Murray | 14d080e | 2013-01-12 11:34:38 -0500 | [diff] [blame] | 1427 | indirect_test() |
Benjamin Peterson | f617fa8 | 2014-02-16 14:53:55 -0500 | [diff] [blame] | 1428 | alloc_after, rc_after = dash_R_cleanup(fs, ps, pic, zdc, abcs) |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1429 | sys.stderr.write('.') |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1430 | sys.stderr.flush() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1431 | if i >= nwarmup: |
Antoine Pitrou | f9d0b12 | 2012-12-09 14:28:26 +0100 | [diff] [blame] | 1432 | rc_deltas[i] = rc_after - rc_before |
| 1433 | alloc_deltas[i] = alloc_after - alloc_before |
| 1434 | alloc_before, rc_before = alloc_after, rc_after |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1435 | print(file=sys.stderr) |
Antoine Pitrou | f9d0b12 | 2012-12-09 14:28:26 +0100 | [diff] [blame] | 1436 | # These checkers return False on success, True on failure |
| 1437 | def check_rc_deltas(deltas): |
| 1438 | return any(deltas) |
| 1439 | def check_alloc_deltas(deltas): |
| 1440 | # At least 1/3rd of 0s |
| 1441 | if 3 * deltas.count(0) < len(deltas): |
| 1442 | return True |
| 1443 | # Nothing else than 1s, 0s and -1s |
| 1444 | if not set(deltas) <= {1,0,-1}: |
| 1445 | return True |
| 1446 | return False |
| 1447 | failed = False |
| 1448 | for deltas, item_name, checker in [ |
| 1449 | (rc_deltas, 'references', check_rc_deltas), |
| 1450 | (alloc_deltas, 'memory blocks', check_alloc_deltas)]: |
| 1451 | if checker(deltas): |
| 1452 | msg = '%s leaked %s %s, sum=%s' % ( |
| 1453 | test, deltas[nwarmup:], item_name, sum(deltas)) |
| 1454 | print(msg, file=sys.stderr) |
| 1455 | sys.stderr.flush() |
| 1456 | with open(fname, "a") as refrep: |
| 1457 | print(msg, file=refrep) |
| 1458 | refrep.flush() |
| 1459 | failed = True |
| 1460 | return failed |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1461 | |
Benjamin Peterson | cf62603 | 2014-02-16 14:52:01 -0500 | [diff] [blame] | 1462 | def dash_R_cleanup(fs, ps, pic, zdc, abcs): |
Alexandre Vassalotti | f7fa63d | 2008-05-11 08:55:36 +0000 | [diff] [blame] | 1463 | import gc, copyreg |
Brett Cannon | f4fd993 | 2008-05-10 21:11:46 +0000 | [diff] [blame] | 1464 | import _strptime, linecache |
Jeremy Hylton | 1afc169 | 2008-06-18 20:49:58 +0000 | [diff] [blame] | 1465 | import urllib.parse, urllib.request, mimetypes, doctest |
Raymond Hettinger | 158c9c2 | 2011-02-22 00:41:50 +0000 | [diff] [blame] | 1466 | import struct, filecmp, collections.abc |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1467 | from distutils.dir_util import _path_created |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1468 | from weakref import WeakSet |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1469 | |
Christian Heimes | dae2a89 | 2008-04-19 00:55:37 +0000 | [diff] [blame] | 1470 | # Clear the warnings registry, so they can be displayed again |
| 1471 | for mod in sys.modules.values(): |
| 1472 | if hasattr(mod, '__warningregistry__'): |
| 1473 | del mod.__warningregistry__ |
| 1474 | |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1475 | # Restore some original values. |
| 1476 | warnings.filters[:] = fs |
Alexandre Vassalotti | f7fa63d | 2008-05-11 08:55:36 +0000 | [diff] [blame] | 1477 | copyreg.dispatch_table.clear() |
| 1478 | copyreg.dispatch_table.update(ps) |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1479 | sys.path_importer_cache.clear() |
| 1480 | sys.path_importer_cache.update(pic) |
Nick Coghlan | 260bd3e | 2009-11-16 06:49:25 +0000 | [diff] [blame] | 1481 | try: |
| 1482 | import zipimport |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 1483 | except ImportError: |
Nick Coghlan | 260bd3e | 2009-11-16 06:49:25 +0000 | [diff] [blame] | 1484 | pass # Run unmodified on platforms without zipimport support |
| 1485 | else: |
| 1486 | zipimport._zip_directory_cache.clear() |
| 1487 | zipimport._zip_directory_cache.update(zdc) |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1488 | |
Christian Heimes | 2685563 | 2008-01-27 23:50:43 +0000 | [diff] [blame] | 1489 | # clear type cache |
Christian Heimes | 15ebc88 | 2008-02-04 18:48:49 +0000 | [diff] [blame] | 1490 | sys._clear_type_cache() |
Christian Heimes | 2685563 | 2008-01-27 23:50:43 +0000 | [diff] [blame] | 1491 | |
Guido van Rossum | 3de862d | 2007-08-18 00:10:33 +0000 | [diff] [blame] | 1492 | # Clear ABC registries, restoring previously saved ABC registries. |
Raymond Hettinger | 158c9c2 | 2011-02-22 00:41:50 +0000 | [diff] [blame] | 1493 | for abc in [getattr(collections.abc, a) for a in collections.abc.__all__]: |
Christian Heimes | be5b30b | 2008-03-03 19:18:51 +0000 | [diff] [blame] | 1494 | if not isabstract(abc): |
Christian Heimes | 941973a | 2007-11-30 21:53:03 +0000 | [diff] [blame] | 1495 | continue |
Guido van Rossum | 7eaf822 | 2007-06-18 17:58:50 +0000 | [diff] [blame] | 1496 | for obj in abc.__subclasses__() + [abc]: |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1497 | obj._abc_registry = abcs.get(obj, WeakSet()).copy() |
Guido van Rossum | c1e315d | 2007-08-20 19:29:24 +0000 | [diff] [blame] | 1498 | obj._abc_cache.clear() |
| 1499 | obj._abc_negative_cache.clear() |
Guido van Rossum | cd16bf6 | 2007-06-13 18:07:49 +0000 | [diff] [blame] | 1500 | |
Antoine Pitrou | 046467c | 2009-10-30 18:30:35 +0000 | [diff] [blame] | 1501 | # Flush standard output, so that buffered data is sent to the OS and |
| 1502 | # associated Python objects are reclaimed. |
| 1503 | for stream in (sys.stdout, sys.stderr, sys.__stdout__, sys.__stderr__): |
| 1504 | if stream is not None: |
| 1505 | stream.flush() |
| 1506 | |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1507 | # Clear assorted module caches. |
| 1508 | _path_created.clear() |
| 1509 | re.purge() |
| 1510 | _strptime._regex_cache.clear() |
Jeremy Hylton | 1afc169 | 2008-06-18 20:49:58 +0000 | [diff] [blame] | 1511 | urllib.parse.clear_cache() |
| 1512 | urllib.request.urlcleanup() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1513 | linecache.clearcache() |
| 1514 | mimetypes._default_mime_types() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1515 | filecmp._cache.clear() |
Christian Heimes | a34706f | 2008-01-04 03:06:10 +0000 | [diff] [blame] | 1516 | struct._clearcache() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1517 | doctest.master = None |
Meador Inge | 11e3813 | 2011-11-25 22:33:32 -0600 | [diff] [blame] | 1518 | try: |
| 1519 | import ctypes |
Brett Cannon | 260fbe8 | 2013-07-04 18:16:15 -0400 | [diff] [blame] | 1520 | except ImportError: |
Meador Inge | 11e3813 | 2011-11-25 22:33:32 -0600 | [diff] [blame] | 1521 | # Don't worry about resetting the cache if ctypes is not supported |
| 1522 | pass |
| 1523 | else: |
| 1524 | ctypes._reset_cache() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1525 | |
Antoine Pitrou | f9d0b12 | 2012-12-09 14:28:26 +0100 | [diff] [blame] | 1526 | # Collect cyclic trash and read memory statistics immediately after. |
| 1527 | func1 = sys.getallocatedblocks |
| 1528 | func2 = sys.gettotalrefcount |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1529 | gc.collect() |
Antoine Pitrou | f9d0b12 | 2012-12-09 14:28:26 +0100 | [diff] [blame] | 1530 | return func1(), func2() |
Thomas Wouters | 73e5a5b | 2006-06-08 15:35:45 +0000 | [diff] [blame] | 1531 | |
Stefan Krah | 5c3ddc8 | 2012-08-17 23:09:48 +0200 | [diff] [blame] | 1532 | def warm_caches(): |
| 1533 | # char cache |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1534 | s = bytes(range(256)) |
| 1535 | for i in range(256): |
| 1536 | s[i:i+1] |
Stefan Krah | 5c3ddc8 | 2012-08-17 23:09:48 +0200 | [diff] [blame] | 1537 | # unicode cache |
| 1538 | x = [chr(i) for i in range(256)] |
| 1539 | # int cache |
| 1540 | x = list(range(-5, 257)) |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1541 | |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 1542 | def findtestdir(path=None): |
| 1543 | return path or os.path.dirname(__file__) or os.curdir |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 1544 | |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 1545 | def removepy(names): |
| 1546 | if not names: |
| 1547 | return |
| 1548 | for idx, name in enumerate(names): |
| 1549 | basename, ext = os.path.splitext(name) |
| 1550 | if ext == '.py': |
| 1551 | names[idx] = basename |
Tim Peters | c5000df | 2002-06-02 21:42:01 +0000 | [diff] [blame] | 1552 | |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 1553 | def count(n, word): |
| 1554 | if n == 1: |
Guido van Rossum | 41360a4 | 1998-03-26 19:42:58 +0000 | [diff] [blame] | 1555 | return "%d %s" % (n, word) |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 1556 | else: |
Guido van Rossum | 41360a4 | 1998-03-26 19:42:58 +0000 | [diff] [blame] | 1557 | return "%d %ss" % (n, word) |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 1558 | |
Tim Peters | a45da92 | 2001-08-12 03:45:50 +0000 | [diff] [blame] | 1559 | def printlist(x, width=70, indent=4): |
Tim Peters | 7c7efe9 | 2002-08-23 17:55:54 +0000 | [diff] [blame] | 1560 | """Print the elements of iterable x to stdout. |
Tim Peters | a45da92 | 2001-08-12 03:45:50 +0000 | [diff] [blame] | 1561 | |
| 1562 | Optional arg width (default 70) is the maximum line length. |
| 1563 | Optional arg indent (default 4) is the number of blanks with which to |
| 1564 | begin each line. |
| 1565 | """ |
| 1566 | |
Tim Peters | ba78bc4 | 2002-07-04 19:45:06 +0000 | [diff] [blame] | 1567 | from textwrap import fill |
| 1568 | blanks = ' ' * indent |
Florent Xicluna | fd1b093 | 2010-03-28 00:25:02 +0000 | [diff] [blame] | 1569 | # Print the sorted list: 'x' may be a '--random' list or a set() |
| 1570 | print(fill(' '.join(str(elt) for elt in sorted(x)), width, |
Neal Norwitz | 94fa2ee | 2008-03-31 02:55:15 +0000 | [diff] [blame] | 1571 | initial_indent=blanks, subsequent_indent=blanks)) |
Tim Peters | a45da92 | 2001-08-12 03:45:50 +0000 | [diff] [blame] | 1572 | |
Tim Peters | b5b7b78 | 2001-08-12 01:20:39 +0000 | [diff] [blame] | 1573 | |
Chris Jerdonek | 517e925 | 2013-02-27 09:02:53 -0800 | [diff] [blame] | 1574 | def main_in_temp_cwd(): |
| 1575 | """Run main() in a temporary working directory.""" |
Michael Foord | 3ab34cc | 2010-12-03 12:27:40 +0000 | [diff] [blame] | 1576 | if sysconfig.is_python_build(): |
Antoine Pitrou | ee42934 | 2011-04-16 18:53:59 +0200 | [diff] [blame] | 1577 | try: |
Michael Foord | 3ab34cc | 2010-12-03 12:27:40 +0000 | [diff] [blame] | 1578 | os.mkdir(TEMPDIR) |
Florent Xicluna | 68f71a3 | 2011-10-28 16:06:23 +0200 | [diff] [blame] | 1579 | except FileExistsError: |
| 1580 | pass |
Michael Foord | 3ab34cc | 2010-12-03 12:27:40 +0000 | [diff] [blame] | 1581 | |
| 1582 | # Define a writable temp dir that will be used as cwd while running |
| 1583 | # the tests. The name of the dir includes the pid to allow parallel |
| 1584 | # testing (see the -j option). |
Chris Jerdonek | 517e925 | 2013-02-27 09:02:53 -0800 | [diff] [blame] | 1585 | test_cwd = 'test_python_{}'.format(os.getpid()) |
| 1586 | test_cwd = os.path.join(TEMPDIR, test_cwd) |
Michael Foord | 3ab34cc | 2010-12-03 12:27:40 +0000 | [diff] [blame] | 1587 | |
Chris Jerdonek | 517e925 | 2013-02-27 09:02:53 -0800 | [diff] [blame] | 1588 | # Run the tests in a context manager that temporarily changes the CWD to a |
| 1589 | # temporary and writable directory. If it's not possible to create or |
| 1590 | # change the CWD, the original CWD will be used. The original CWD is |
| 1591 | # available from support.SAVEDCWD. |
| 1592 | with support.temp_cwd(test_cwd, quiet=True): |
| 1593 | main() |
| 1594 | |
Nick Coghlan | 4c4c0f2 | 2010-12-03 07:44:33 +0000 | [diff] [blame] | 1595 | |
Guido van Rossum | 152494a | 1996-12-20 03:12:20 +0000 | [diff] [blame] | 1596 | if __name__ == '__main__': |
Nick Coghlan | 4c4c0f2 | 2010-12-03 07:44:33 +0000 | [diff] [blame] | 1597 | # Remove regrtest.py's own directory from the module search path. Despite |
| 1598 | # the elimination of implicit relative imports, this is still needed to |
| 1599 | # ensure that submodules of the test package do not inappropriately appear |
| 1600 | # as top-level modules even when people (or buildbots!) invoke regrtest.py |
| 1601 | # directly instead of using the -m switch |
| 1602 | mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) |
| 1603 | i = len(sys.path) |
| 1604 | while i >= 0: |
| 1605 | i -= 1 |
| 1606 | if os.path.abspath(os.path.normpath(sys.path[i])) == mydir: |
| 1607 | del sys.path[i] |
| 1608 | |
Florent Xicluna | dc69e72 | 2010-09-13 16:35:02 +0000 | [diff] [blame] | 1609 | # findtestdir() gets the dirname out of __file__, so we have to make it |
| 1610 | # absolute before changing the working directory. |
| 1611 | # For example __file__ may be relative when running trace or profile. |
| 1612 | # See issue #9323. |
| 1613 | __file__ = os.path.abspath(__file__) |
| 1614 | |
| 1615 | # sanity check |
Florent Xicluna | da7bfd5 | 2010-03-06 11:43:55 +0000 | [diff] [blame] | 1616 | assert __file__ == os.path.abspath(sys.argv[0]) |
Ezio Melotti | 184bdfb | 2010-02-18 09:37:05 +0000 | [diff] [blame] | 1617 | |
Chris Jerdonek | 517e925 | 2013-02-27 09:02:53 -0800 | [diff] [blame] | 1618 | main_in_temp_cwd() |