blob: 9a8172cc20337b2072ab8fdf87fbade8e5d2c5b2 [file] [log] [blame]
Guido van Rossum152494a1996-12-20 03:12:20 +00001#! /usr/bin/env python
2
R. David Murray3bfa8832010-04-26 16:54:57 +00003"""
4Usage:
Guido van Rossum152494a1996-12-20 03:12:20 +00005
R. David Murray3bfa8832010-04-26 16:54:57 +00006python -m test.regrtest [options] [test_name1 [test_name2 ...]]
7python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]
Guido van Rossum152494a1996-12-20 03:12:20 +00008
R. David Murray3bfa8832010-04-26 16:54:57 +00009
10If no arguments or options are provided, finds all files matching
11the pattern "test_*" in the Lib/test subdirectory and runs
12them in alphabetical order (but see -M and -u, below, for exceptions).
13
14For more rigorous testing, it is useful to use the following
15command line:
16
17python -E -tt -Wd -3 -m test.regrtest [options] [test_name1 ...]
18
19
20Options:
Guido van Rossum152494a1996-12-20 03:12:20 +000021
Georg Brandl83191702009-10-27 20:55:44 +000022-h/--help -- print this text and exit
23
24Verbosity
25
26-v/--verbose -- run tests in verbose mode with output to stdout
27-w/--verbose2 -- re-run failed tests in verbose mode
28-W/--verbose3 -- re-run failed tests in verbose mode immediately
R. David Murray3bfa8832010-04-26 16:54:57 +000029-q/--quiet -- no output unless one or more tests fail
Georg Brandl83191702009-10-27 20:55:44 +000030-S/--slow -- print the slowest 10 tests
Antoine Pitrou8c688be2011-01-03 20:49:01 +000031 --header -- print header with interpreter info
Georg Brandl83191702009-10-27 20:55:44 +000032
33Selecting tests
34
Chris Jerdonek3684c792012-12-27 18:14:01 -080035-r/--randomize -- randomize test execution order (see below)
Antoine Pitrou8c688be2011-01-03 20:49:01 +000036 --randseed -- pass a random seed to reproduce a previous random run
Georg Brandl83191702009-10-27 20:55:44 +000037-f/--fromfile -- read names of tests to run from a file (see below)
38-x/--exclude -- arguments are tests to *exclude*
R. David Murray3bfa8832010-04-26 16:54:57 +000039-s/--single -- single step through a set of tests (see below)
Georg Brandl83191702009-10-27 20:55:44 +000040-u/--use RES1,RES2,...
41 -- specify which special resource intensive tests to run
42-M/--memlimit LIMIT
43 -- run very large memory-consuming tests
44
45Special runs
46
47-l/--findleaks -- if GC is available detect tests that leak memory
48-L/--runleaks -- run the leaks(1) command just before exit
49-R/--huntrleaks RUNCOUNTS
50 -- search for reference leaks (needs debug build, v. slow)
51-j/--multiprocess PROCESSES
52 -- run PROCESSES processes at once
R. David Murray3bfa8832010-04-26 16:54:57 +000053-T/--coverage -- turn on code coverage tracing using the trace module
Georg Brandl83191702009-10-27 20:55:44 +000054-D/--coverdir DIRECTORY
55 -- Directory where coverage files are put
56-N/--nocoverdir -- Put coverage files alongside modules
57-t/--threshold THRESHOLD
58 -- call gc.set_threshold(THRESHOLD)
R. David Murray3bfa8832010-04-26 16:54:57 +000059-F/--forever -- run the specified tests in a loop, until an error happens
Guido van Rossum152494a1996-12-20 03:12:20 +000060
R. David Murray3bfa8832010-04-26 16:54:57 +000061
62Additional Option Details:
Guido van Rossumf58ed251997-03-07 21:04:33 +000063
Collin Wintera199f012009-03-29 03:44:19 +000064-r randomizes test execution order. You can use --randseed=int to provide a
65int seed value for the randomizer; this is useful for reproducing troublesome
66test orders.
67
R. David Murray3bfa8832010-04-26 16:54:57 +000068-s On the first invocation of regrtest using -s, the first test file found
69or the first test file given on the command line is run, and the name of
70the next test is recorded in a file named pynexttest. If run from the
71Python build directory, pynexttest is located in the 'build' subdirectory,
72otherwise it is located in tempfile.gettempdir(). On subsequent runs,
73the test in pynexttest is run, and the next test is written to pynexttest.
74When the last test has been run, pynexttest is deleted. In this way it
75is possible to single step through the test files. This is useful when
76doing memory analysis on the Python interpreter, which process tends to
77consume too many resources to run the full regression test non-stop.
Barry Warsawe11e3de1999-01-28 19:51:51 +000078
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000079-f reads the names of tests from the file given as f's argument, one
80or more test names per line. Whitespace is ignored. Blank lines and
81lines beginning with '#' are ignored. This is especially useful for
82whittling down failures involving interactions among tests.
Tim Petersc5000df2002-06-02 21:42:01 +000083
Skip Montanaro0179a182004-06-06 15:53:18 +000084-L causes the leaks(1) command to be run just before exit if it exists.
85leaks(1) is available on Mac OS X and presumably on some other
86FreeBSD-derived systems.
87
Michael W. Hudson61147f62004-08-03 11:33:28 +000088-R runs each test several times and examines sys.gettotalrefcount() to
89see if the test appears to be leaking references. The argument should
90be of the form stab:run:fname where 'stab' is the number of times the
91test is run to let gettotalrefcount settle down, 'run' is the number
92of times further it is run and 'fname' is the name of the file the
93reports are written to. These parameters all have defaults (5, 4 and
Florent Xicluna12d750d2010-03-06 14:38:09 +000094"reflog.txt" respectively), and the minimal invocation is '-R :'.
Michael W. Hudson61147f62004-08-03 11:33:28 +000095
Thomas Woutersabd08882006-04-26 15:53:30 +000096-M runs tests that require an exorbitant amount of memory. These tests
97typically try to ascertain containers keep working when containing more than
Armin Rigo97ff0472006-08-09 15:37:26 +0000982 billion objects, which only works on 64-bit systems. There are also some
99tests that try to exhaust the address space of the process, which only makes
100sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,
Thomas Woutersabd08882006-04-26 15:53:30 +0000101which is a string in the form of '2.5Gb', determines howmuch memory the
102tests will limit themselves to (but they may go slightly over.) The number
103shouldn't be more memory than the machine has (including swap memory). You
104should also keep in mind that swap memory is generally much, much slower
105than RAM, and setting memlimit to all available RAM or higher will heavily
106tax the machine. On the other hand, it is no use running these tests with a
107limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect
108to use more than memlimit memory will be skipped. The big-memory tests
109generally run very, very long.
110
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000111-u is used to specify which special resource intensive tests to run,
112such as those requiring large file support or network connectivity.
113The argument is a comma-separated list of words indicating the
114resources to test. Currently only the following are defined:
Barry Warsaw08fca522001-08-20 22:33:46 +0000115
Fred Drake3a15dac2002-04-11 16:39:16 +0000116 all - Enable all special resources.
117
Guido van Rossum315aa362003-03-11 14:46:48 +0000118 audio - Tests that use the audio device. (There are known
119 cases of broken audio drivers that can crash Python or
120 even the Linux kernel.)
121
Andrew M. Kuchling2158df02001-10-22 15:26:09 +0000122 curses - Tests that use curses and will modify the terminal's
123 state and output modes.
Tim Peters1633a2e2001-10-30 05:56:40 +0000124
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000125 largefile - It is okay to run some test that may create huge
126 files. These tests can take a long time and may
127 consume >2GB of disk space temporarily.
Barry Warsaw08fca522001-08-20 22:33:46 +0000128
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000129 network - It is okay to run tests that use external network
130 resource, e.g. testing SSL support for sockets.
Martin v. Löwis1c6b1a22002-11-19 17:47:07 +0000131
132 bsddb - It is okay to run the bsddb testsuite, which takes
133 a long time to complete.
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000134
Raymond Hettinger7c85fa42004-07-01 11:01:35 +0000135 decimal - Test the decimal module against a large suite that
136 verifies compliance with standards.
137
Antoine Pitroud989f822010-10-14 15:43:25 +0000138 cpu - Used for certain CPU-heavy tests.
Jeremy Hylton4336eda2004-08-07 19:25:33 +0000139
Tim Peterseba28be2005-03-28 01:08:02 +0000140 subprocess Run all tests for the subprocess module.
Peter Astrandf7f1bb72005-03-03 20:47:37 +0000141
Hye-Shik Changaaa2f1d2005-12-10 17:44:27 +0000142 urlfetch - It is okay to download files required on testing.
143
Guilherme Polob1a98de2009-01-28 20:03:26 +0000144 gui - Run tests that require a running GUI.
145
Collin Winterf8089c72009-04-09 16:46:46 +0000146 xpickle - Test pickle and cPickle against Python 2.4, 2.5 and 2.6 to
147 test backwards compatibility. These tests take a long time
148 to run.
149
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000150To enable all resources except one, use '-uall,-<resource>'. For
151example, to run all the tests except for the bsddb tests, give the
152option '-uall,-bsddb'.
Guido van Rossum152494a1996-12-20 03:12:20 +0000153"""
154
Michael Foord91a2c892010-04-08 00:04:24 +0000155import StringIO
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000156import getopt
Senthil Kumarance8e33a2010-01-08 19:04:16 +0000157import json
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000158import os
159import random
160import re
Serhiy Storchakabb801312013-02-10 12:01:31 +0200161import shutil
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000162import sys
163import time
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000164import traceback
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000165import warnings
Benjamin Peterson888a39b2009-03-26 20:48:25 +0000166import unittest
Ezio Melotti8f6a2872010-02-10 21:40:33 +0000167import tempfile
Ezio Melottidde5b942010-02-03 05:37:26 +0000168import imp
Florent Xiclunab7c20022010-03-06 09:11:55 +0000169import platform
Florent Xicluna0932dc52010-03-06 08:07:44 +0000170import sysconfig
Guido van Rossumdc15c272002-08-12 21:55:51 +0000171
Guido van Rossum152494a1996-12-20 03:12:20 +0000172
Ezio Melotti8f6a2872010-02-10 21:40:33 +0000173# Some times __path__ and __file__ are not absolute (e.g. while running from
174# Lib/) and, if we change the CWD to run the tests in a temporary dir, some
175# imports might fail. This affects only the modules imported before os.chdir().
176# These modules are searched first in sys.path[0] (so '' -- the CWD) and if
177# they are found in the CWD their __file__ and __path__ will be relative (this
178# happens before the chdir). All the modules imported after the chdir, are
179# not found in the CWD, and since the other paths in sys.path[1:] are absolute
180# (site.py absolutize them), the __file__ and __path__ will be absolute too.
181# Therefore it is necessary to absolutize manually the __file__ and __path__ of
182# the packages to prevent later imports to fail when the CWD is different.
183for module in sys.modules.itervalues():
184 if hasattr(module, '__path__'):
185 module.__path__ = [os.path.abspath(path) for path in module.__path__]
186 if hasattr(module, '__file__'):
187 module.__file__ = os.path.abspath(module.__file__)
188
189
Guido van Rossumbb484652002-12-02 09:56:21 +0000190# MacOSX (a.k.a. Darwin) has a default stack size that is too small
191# for deeply recursive regular expressions. We see this as crashes in
192# the Python test suite when running test_re.py and test_sre.py. The
193# fix is to set the stack limit to 2048.
194# This approach may also be useful for other Unixy platforms that
195# suffer from small default stack limits.
196if sys.platform == 'darwin':
197 try:
198 import resource
199 except ImportError:
200 pass
201 else:
202 soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
203 newsoft = min(hard, max(soft, 1024*2048))
204 resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
205
Zachary Ware47343722015-07-16 00:24:48 -0500206# Windows, Tkinter, and resetting the environment after each test don't
207# mix well. To alleviate test failures due to Tcl/Tk not being able to
208# find its library, get the necessary environment massage done once early.
209if sys.platform == 'win32':
210 try:
211 import FixTk
212 except Exception:
213 pass
214
Georg Brandl4045e882009-10-29 20:53:00 +0000215# Test result constants.
216PASSED = 1
217FAILED = 0
218ENV_CHANGED = -1
219SKIPPED = -2
220RESOURCE_DENIED = -3
221INTERRUPTED = -4
222
Barry Warsaw04f357c2002-07-23 19:04:11 +0000223from test import test_support
Fred Drake3a15dac2002-04-11 16:39:16 +0000224
Benjamin Petersone2886fd2008-07-18 14:26:35 +0000225RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
Antoine Pitroud989f822010-10-14 15:43:25 +0000226 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui',
Collin Winterf8089c72009-04-09 16:46:46 +0000227 'xpickle')
Fred Drake3a15dac2002-04-11 16:39:16 +0000228
Florent Xicluna12735662010-04-23 18:10:12 +0000229TEMPDIR = os.path.abspath(tempfile.gettempdir())
230
Fred Drake3a15dac2002-04-11 16:39:16 +0000231
Barry Warsaw08fca522001-08-20 22:33:46 +0000232def usage(code, msg=''):
233 print __doc__
234 if msg: print msg
235 sys.exit(code)
236
237
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000238def main(tests=None, testdir=None, verbose=0, quiet=False,
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000239 exclude=False, single=False, randomize=False, fromfile=None,
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000240 findleaks=False, use_resources=None, trace=False, coverdir='coverage',
Collin Wintera199f012009-03-29 03:44:19 +0000241 runleaks=False, huntrleaks=False, verbose2=False, print_slow=False,
Antoine Pitrou8c688be2011-01-03 20:49:01 +0000242 random_seed=None, use_mp=None, verbose3=False, forever=False,
243 header=False):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000244 """Execute a test suite.
245
Thomas Wouters7e474022000-07-16 12:04:32 +0000246 This also parses command-line options and modifies its behavior
Fred Drake004d5e62000-10-23 17:22:08 +0000247 accordingly.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000248
249 tests -- a list of strings containing test names (optional)
250 testdir -- the directory in which to look for tests (optional)
251
252 Users other than the Python test suite will certainly want to
253 specify testdir; if it's omitted, the directory containing the
Fred Drake004d5e62000-10-23 17:22:08 +0000254 Python test suite is searched for.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000255
256 If the tests argument is omitted, the tests listed on the
257 command-line will be used. If that's empty, too, then all *.py
258 files beginning with test_ will be used.
Skip Montanaroab1c7912000-06-30 16:39:27 +0000259
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000260 The other default arguments (verbose, quiet, exclude,
Florent Xicluna12d750d2010-03-06 14:38:09 +0000261 single, randomize, findleaks, use_resources, trace, coverdir,
262 print_slow, and random_seed) allow programmers calling main()
263 directly to set the values that would normally be set by flags
264 on the command line.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000265 """
Fred Drake004d5e62000-10-23 17:22:08 +0000266
Tim Peters8dee8092001-09-25 20:05:11 +0000267 test_support.record_original_stdout(sys.stdout)
Guido van Rossum152494a1996-12-20 03:12:20 +0000268 try:
R. David Murray4653b8c2009-11-15 00:07:00 +0000269 opts, args = getopt.getopt(sys.argv[1:], 'hvqxsSrf:lu:t:TD:NLR:FwWM:j:',
Georg Brandl71a015a2009-10-27 20:52:02 +0000270 ['help', 'verbose', 'verbose2', 'verbose3', 'quiet',
Chris Jerdonek3684c792012-12-27 18:14:01 -0800271 'exclude', 'single', 'slow', 'randomize', 'fromfile=', 'findleaks',
Georg Brandl71a015a2009-10-27 20:52:02 +0000272 'use=', 'threshold=', 'trace', 'coverdir=', 'nocoverdir',
273 'runleaks', 'huntrleaks=', 'memlimit=', 'randseed=',
Antoine Pitrou8c688be2011-01-03 20:49:01 +0000274 'multiprocess=', 'slaveargs=', 'forever', 'header'])
Guido van Rossum152494a1996-12-20 03:12:20 +0000275 except getopt.error, msg:
Barry Warsaw08fca522001-08-20 22:33:46 +0000276 usage(2, msg)
277
278 # Defaults
Collin Wintera199f012009-03-29 03:44:19 +0000279 if random_seed is None:
280 random_seed = random.randrange(10000000)
Barry Warsaw08fca522001-08-20 22:33:46 +0000281 if use_resources is None:
282 use_resources = []
Guido van Rossum152494a1996-12-20 03:12:20 +0000283 for o, a in opts:
Barry Warsaw08fca522001-08-20 22:33:46 +0000284 if o in ('-h', '--help'):
285 usage(0)
286 elif o in ('-v', '--verbose'):
287 verbose += 1
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000288 elif o in ('-w', '--verbose2'):
289 verbose2 = True
Georg Brandl71a015a2009-10-27 20:52:02 +0000290 elif o in ('-W', '--verbose3'):
291 verbose3 = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000292 elif o in ('-q', '--quiet'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000293 quiet = True;
Barry Warsaw08fca522001-08-20 22:33:46 +0000294 verbose = 0
Barry Warsaw08fca522001-08-20 22:33:46 +0000295 elif o in ('-x', '--exclude'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000296 exclude = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000297 elif o in ('-s', '--single'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000298 single = True
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000299 elif o in ('-S', '--slow'):
300 print_slow = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000301 elif o in ('-r', '--randomize'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000302 randomize = True
Collin Wintera199f012009-03-29 03:44:19 +0000303 elif o == '--randseed':
304 random_seed = int(a)
Tim Petersc5000df2002-06-02 21:42:01 +0000305 elif o in ('-f', '--fromfile'):
306 fromfile = a
Barry Warsaw08fca522001-08-20 22:33:46 +0000307 elif o in ('-l', '--findleaks'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000308 findleaks = True
Skip Montanaro0179a182004-06-06 15:53:18 +0000309 elif o in ('-L', '--runleaks'):
310 runleaks = True
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000311 elif o in ('-t', '--threshold'):
312 import gc
313 gc.set_threshold(int(a))
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000314 elif o in ('-T', '--coverage'):
315 trace = True
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000316 elif o in ('-D', '--coverdir'):
317 coverdir = os.path.join(os.getcwd(), a)
318 elif o in ('-N', '--nocoverdir'):
319 coverdir = None
Michael W. Hudson61147f62004-08-03 11:33:28 +0000320 elif o in ('-R', '--huntrleaks'):
321 huntrleaks = a.split(':')
Florent Xicluna12d750d2010-03-06 14:38:09 +0000322 if len(huntrleaks) not in (2, 3):
Michael W. Hudson61147f62004-08-03 11:33:28 +0000323 print a, huntrleaks
Florent Xicluna12d750d2010-03-06 14:38:09 +0000324 usage(2, '-R takes 2 or 3 colon-separated arguments')
325 if not huntrleaks[0]:
Michael W. Hudson61147f62004-08-03 11:33:28 +0000326 huntrleaks[0] = 5
327 else:
328 huntrleaks[0] = int(huntrleaks[0])
Florent Xicluna12d750d2010-03-06 14:38:09 +0000329 if not huntrleaks[1]:
Michael W. Hudson61147f62004-08-03 11:33:28 +0000330 huntrleaks[1] = 4
331 else:
332 huntrleaks[1] = int(huntrleaks[1])
Florent Xicluna12d750d2010-03-06 14:38:09 +0000333 if len(huntrleaks) == 2 or not huntrleaks[2]:
334 huntrleaks[2:] = ["reflog.txt"]
Thomas Woutersabd08882006-04-26 15:53:30 +0000335 elif o in ('-M', '--memlimit'):
336 test_support.set_memlimit(a)
Barry Warsaw08fca522001-08-20 22:33:46 +0000337 elif o in ('-u', '--use'):
Guido van Rossumfe3f6962001-09-06 16:09:41 +0000338 u = [x.lower() for x in a.split(',')]
339 for r in u:
Fred Drake3a15dac2002-04-11 16:39:16 +0000340 if r == 'all':
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000341 use_resources[:] = RESOURCE_NAMES
342 continue
343 remove = False
344 if r[0] == '-':
345 remove = True
346 r = r[1:]
Fred Drake3a15dac2002-04-11 16:39:16 +0000347 if r not in RESOURCE_NAMES:
348 usage(1, 'Invalid -u/--use option: ' + a)
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000349 if remove:
350 if r in use_resources:
351 use_resources.remove(r)
352 elif r not in use_resources:
Andrew MacIntyree41abab2002-04-30 12:11:04 +0000353 use_resources.append(r)
R. David Murray7f7eea62009-11-14 15:18:22 +0000354 elif o in ('-F', '--forever'):
355 forever = True
Antoine Pitrou4698d992009-05-31 14:20:14 +0000356 elif o in ('-j', '--multiprocess'):
357 use_mp = int(a)
Antoine Pitrou8c688be2011-01-03 20:49:01 +0000358 elif o == '--header':
359 header = True
Antoine Pitrou4698d992009-05-31 14:20:14 +0000360 elif o == '--slaveargs':
361 args, kwargs = json.loads(a)
362 try:
363 result = runtest(*args, **kwargs)
364 except BaseException, e:
Georg Brandl4045e882009-10-29 20:53:00 +0000365 result = INTERRUPTED, e.__class__.__name__
Antoine Pitrou4698d992009-05-31 14:20:14 +0000366 print # Force a newline (just in case)
367 print json.dumps(result)
368 sys.exit(0)
R. David Murrayda826892009-11-15 00:04:32 +0000369 else:
370 print >>sys.stderr, ("No handler for option {}. Please "
371 "report this as a bug at http://bugs.python.org.").format(o)
372 sys.exit(1)
Tim Petersc5000df2002-06-02 21:42:01 +0000373 if single and fromfile:
374 usage(2, "-s and -f don't go together!")
Antoine Pitrou4698d992009-05-31 14:20:14 +0000375 if use_mp and trace:
376 usage(2, "-T and -j don't go together!")
377 if use_mp and findleaks:
378 usage(2, "-l and -j don't go together!")
Barry Warsaw08fca522001-08-20 22:33:46 +0000379
Guido van Rossum152494a1996-12-20 03:12:20 +0000380 good = []
381 bad = []
382 skipped = []
Fred Drake9a0db072003-02-03 15:19:30 +0000383 resource_denieds = []
R. David Murray14dcd432009-10-14 13:58:07 +0000384 environment_changed = []
Florent Xicluna75c66762010-03-30 16:31:14 +0000385 interrupted = False
Barry Warsawe11e3de1999-01-28 19:51:51 +0000386
Neil Schemenauerd569f232000-09-22 15:29:28 +0000387 if findleaks:
Barry Warsawa873b032000-08-03 15:50:37 +0000388 try:
389 import gc
390 except ImportError:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000391 print 'No GC available, disabling findleaks.'
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000392 findleaks = False
Barry Warsawa873b032000-08-03 15:50:37 +0000393 else:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000394 # Uncomment the line below to report garbage that is not
395 # freeable by reference counting alone. By default only
396 # garbage that is not collectable by the GC is reported.
397 #gc.set_debug(gc.DEBUG_SAVEALL)
Neil Schemenauerd569f232000-09-22 15:29:28 +0000398 found_garbage = []
Barry Warsawa873b032000-08-03 15:50:37 +0000399
Barry Warsawe11e3de1999-01-28 19:51:51 +0000400 if single:
Florent Xicluna12735662010-04-23 18:10:12 +0000401 filename = os.path.join(TEMPDIR, 'pynexttest')
Barry Warsawe11e3de1999-01-28 19:51:51 +0000402 try:
403 fp = open(filename, 'r')
R. David Murray7f7eea62009-11-14 15:18:22 +0000404 next_test = fp.read().strip()
405 tests = [next_test]
Barry Warsawe11e3de1999-01-28 19:51:51 +0000406 fp.close()
407 except IOError:
408 pass
Tim Petersc5000df2002-06-02 21:42:01 +0000409
410 if fromfile:
411 tests = []
Ezio Melotti8f6a2872010-02-10 21:40:33 +0000412 fp = open(os.path.join(test_support.SAVEDCWD, fromfile))
Tim Petersc5000df2002-06-02 21:42:01 +0000413 for line in fp:
414 guts = line.split() # assuming no test has whitespace in its name
415 if guts and not guts[0].startswith('#'):
416 tests.extend(guts)
417 fp.close()
418
419 # Strip .py extensions.
Florent Xicluna0932dc52010-03-06 08:07:44 +0000420 removepy(args)
421 removepy(tests)
Tim Petersc5000df2002-06-02 21:42:01 +0000422
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000423 stdtests = STDTESTS[:]
Florent Xicluna12d750d2010-03-06 14:38:09 +0000424 nottests = NOTTESTS.copy()
Guido van Rossum152494a1996-12-20 03:12:20 +0000425 if exclude:
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000426 for arg in args:
427 if arg in stdtests:
428 stdtests.remove(arg)
Florent Xicluna12d750d2010-03-06 14:38:09 +0000429 nottests.add(arg)
Guido van Rossum41360a41998-03-26 19:42:58 +0000430 args = []
Florent Xicluna58526412010-03-06 17:24:36 +0000431
432 # For a partial run, we do not need to clutter the output.
Antoine Pitrou8c688be2011-01-03 20:49:01 +0000433 if verbose or header or not (quiet or single or tests or args):
Florent Xicluna58526412010-03-06 17:24:36 +0000434 # Print basic platform information
435 print "==", platform.python_implementation(), \
436 " ".join(sys.version.split())
Florent Xiclunab7d413f2010-05-13 17:05:29 +0000437 print "== ", platform.platform(aliased=True), \
Florent Xicluna8652f2e2010-05-13 18:16:06 +0000438 "%s-endian" % sys.byteorder
Florent Xicluna58526412010-03-06 17:24:36 +0000439 print "== ", os.getcwd()
Antoine Pitrou8c688be2011-01-03 20:49:01 +0000440 print "Testing with flags:", sys.flags
Florent Xicluna58526412010-03-06 17:24:36 +0000441
R. David Murray98e3df32009-12-16 11:49:46 +0000442 alltests = findtests(testdir, stdtests, nottests)
Florent Xicluna75c66762010-03-30 16:31:14 +0000443 selected = tests or args or alltests
Barry Warsawe11e3de1999-01-28 19:51:51 +0000444 if single:
Florent Xicluna75c66762010-03-30 16:31:14 +0000445 selected = selected[:1]
R. David Murray98e3df32009-12-16 11:49:46 +0000446 try:
Florent Xicluna75c66762010-03-30 16:31:14 +0000447 next_single_test = alltests[alltests.index(selected[0])+1]
R. David Murray98e3df32009-12-16 11:49:46 +0000448 except IndexError:
449 next_single_test = None
Skip Montanaroab1c7912000-06-30 16:39:27 +0000450 if randomize:
Collin Wintera199f012009-03-29 03:44:19 +0000451 random.seed(random_seed)
452 print "Using random seed", random_seed
Florent Xicluna75c66762010-03-30 16:31:14 +0000453 random.shuffle(selected)
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000454 if trace:
455 import trace
456 tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],
457 trace=False, count=True)
R. David Murray7f7eea62009-11-14 15:18:22 +0000458
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000459 test_times = []
Barry Warsaw08fca522001-08-20 22:33:46 +0000460 test_support.use_resources = use_resources
Guido van Rossum5796d262000-04-21 21:35:06 +0000461 save_modules = sys.modules.keys()
Antoine Pitrou4698d992009-05-31 14:20:14 +0000462
463 def accumulate_result(test, result):
464 ok, test_time = result
465 test_times.append((test_time, test))
Georg Brandl4045e882009-10-29 20:53:00 +0000466 if ok == PASSED:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000467 good.append(test)
Georg Brandl4045e882009-10-29 20:53:00 +0000468 elif ok == FAILED:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000469 bad.append(test)
Georg Brandl4045e882009-10-29 20:53:00 +0000470 elif ok == ENV_CHANGED:
Georg Brandl4045e882009-10-29 20:53:00 +0000471 environment_changed.append(test)
472 elif ok == SKIPPED:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000473 skipped.append(test)
Georg Brandl4045e882009-10-29 20:53:00 +0000474 elif ok == RESOURCE_DENIED:
475 skipped.append(test)
476 resource_denieds.append(test)
Antoine Pitrou4698d992009-05-31 14:20:14 +0000477
R. David Murray7f7eea62009-11-14 15:18:22 +0000478 if forever:
Florent Xicluna75c66762010-03-30 16:31:14 +0000479 def test_forever(tests=list(selected)):
R. David Murray7f7eea62009-11-14 15:18:22 +0000480 while True:
481 for test in tests:
482 yield test
483 if bad:
484 return
485 tests = test_forever()
Victor Stinner32b1ff92014-09-05 12:12:11 +0200486 test_count = ''
487 test_count_width = 3
R. David Murray7f7eea62009-11-14 15:18:22 +0000488 else:
Florent Xicluna75c66762010-03-30 16:31:14 +0000489 tests = iter(selected)
Victor Stinner32b1ff92014-09-05 12:12:11 +0200490 test_count = '/{}'.format(len(selected))
491 test_count_width = len(test_count) - 1
R. David Murray7f7eea62009-11-14 15:18:22 +0000492
Antoine Pitrou4698d992009-05-31 14:20:14 +0000493 if use_mp:
Victor Stinner1b4a69d2010-04-27 21:47:01 +0000494 try:
495 from threading import Thread
496 except ImportError:
497 print "Multiprocess option requires thread support"
498 sys.exit(2)
Georg Brandla4f46e12010-02-07 17:03:15 +0000499 from Queue import Queue
500 from subprocess import Popen, PIPE
Antoine Pitrou4698d992009-05-31 14:20:14 +0000501 debug_output_pat = re.compile(r"\[\d+ refs\]$")
Antoine Pitrou4698d992009-05-31 14:20:14 +0000502 output = Queue()
R. David Murray7f7eea62009-11-14 15:18:22 +0000503 def tests_and_args():
504 for test in tests:
505 args_tuple = (
R. David Murray561b96f2011-02-11 17:25:54 +0000506 (test, verbose, quiet),
R. David Murray7f7eea62009-11-14 15:18:22 +0000507 dict(huntrleaks=huntrleaks, use_resources=use_resources)
508 )
509 yield (test, args_tuple)
510 pending = tests_and_args()
Antoine Pitroua226c912010-10-14 11:15:50 +0000511 opt_args = test_support.args_from_interpreter_flags()
512 base_cmd = [sys.executable] + opt_args + ['-m', 'test.regrtest']
Antoine Pitrou4698d992009-05-31 14:20:14 +0000513 def work():
514 # A worker thread.
Neal Norwitz14ca3272006-02-28 18:05:43 +0000515 try:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000516 while True:
517 try:
R. David Murray7f7eea62009-11-14 15:18:22 +0000518 test, args_tuple = next(pending)
519 except StopIteration:
R. David Murray282396f2009-10-18 21:12:37 +0000520 output.put((None, None, None, None))
Antoine Pitrou4698d992009-05-31 14:20:14 +0000521 return
Antoine Pitrou79c3bd82009-06-29 14:14:56 +0000522 # -E is needed by some tests, e.g. test_import
Antoine Pitroua226c912010-10-14 11:15:50 +0000523 popen = Popen(base_cmd + ['--slaveargs', json.dumps(args_tuple)],
R. David Murray282396f2009-10-18 21:12:37 +0000524 stdout=PIPE, stderr=PIPE,
Antoine Pitrou0dd8e1d2010-08-18 20:45:10 +0000525 universal_newlines=True,
526 close_fds=(os.name != 'nt'))
R. David Murray282396f2009-10-18 21:12:37 +0000527 stdout, stderr = popen.communicate()
528 # Strip last refcount output line if it exists, since it
529 # comes from the shutdown of the interpreter in the subcommand.
530 stderr = debug_output_pat.sub("", stderr)
531 stdout, _, result = stdout.strip().rpartition("\n")
R. David Murray7f7eea62009-11-14 15:18:22 +0000532 if not result:
533 output.put((None, None, None, None))
534 return
Antoine Pitrou4698d992009-05-31 14:20:14 +0000535 result = json.loads(result)
R. David Murray282396f2009-10-18 21:12:37 +0000536 output.put((test, stdout.rstrip(), stderr.rstrip(), result))
Antoine Pitrou4698d992009-05-31 14:20:14 +0000537 except BaseException:
R. David Murray282396f2009-10-18 21:12:37 +0000538 output.put((None, None, None, None))
Neal Norwitz14ca3272006-02-28 18:05:43 +0000539 raise
Antoine Pitrou4698d992009-05-31 14:20:14 +0000540 workers = [Thread(target=work) for i in range(use_mp)]
541 for worker in workers:
542 worker.start()
543 finished = 0
Victor Stinner32b1ff92014-09-05 12:12:11 +0200544 test_index = 1
R. David Murray7f7eea62009-11-14 15:18:22 +0000545 try:
546 while finished < use_mp:
547 test, stdout, stderr, result = output.get()
548 if test is None:
549 finished += 1
550 continue
551 if stdout:
552 print stdout
553 if stderr:
554 print >>sys.stderr, stderr
Antoine Pitroucf807c02012-06-27 17:41:07 +0200555 sys.stdout.flush()
556 sys.stderr.flush()
R. David Murray7f7eea62009-11-14 15:18:22 +0000557 if result[0] == INTERRUPTED:
558 assert result[1] == 'KeyboardInterrupt'
559 raise KeyboardInterrupt # What else?
560 accumulate_result(test, result)
Victor Stinner32b1ff92014-09-05 12:12:11 +0200561 if not quiet:
562 fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}"
563 print(fmt.format(
564 test_count_width, test_index, test_count,
565 len(bad), test))
566 test_index += 1
R. David Murray7f7eea62009-11-14 15:18:22 +0000567 except KeyboardInterrupt:
Florent Xicluna75c66762010-03-30 16:31:14 +0000568 interrupted = True
R. David Murray7f7eea62009-11-14 15:18:22 +0000569 pending.close()
Antoine Pitrou4698d992009-05-31 14:20:14 +0000570 for worker in workers:
571 worker.join()
572 else:
Victor Stinner32b1ff92014-09-05 12:12:11 +0200573 for test_index, test in enumerate(tests, 1):
Antoine Pitrou4698d992009-05-31 14:20:14 +0000574 if not quiet:
Victor Stinner32b1ff92014-09-05 12:12:11 +0200575 fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}"
576 print(fmt.format(
577 test_count_width, test_index, test_count, len(bad), test))
Antoine Pitrou4698d992009-05-31 14:20:14 +0000578 sys.stdout.flush()
579 if trace:
580 # If we're tracing code coverage, then we don't exit with status
581 # if on a false return value from main.
R. David Murray561b96f2011-02-11 17:25:54 +0000582 tracer.runctx('runtest(test, verbose, quiet)',
Antoine Pitrou4698d992009-05-31 14:20:14 +0000583 globals=globals(), locals=vars())
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000584 else:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000585 try:
R. David Murray561b96f2011-02-11 17:25:54 +0000586 result = runtest(test, verbose, quiet, huntrleaks)
Georg Brandl4045e882009-10-29 20:53:00 +0000587 accumulate_result(test, result)
588 if verbose3 and result[0] == FAILED:
Georg Brandl71a015a2009-10-27 20:52:02 +0000589 print "Re-running test %r in verbose mode" % test
R. David Murray561b96f2011-02-11 17:25:54 +0000590 runtest(test, True, quiet, huntrleaks)
Antoine Pitrou4698d992009-05-31 14:20:14 +0000591 except KeyboardInterrupt:
Florent Xicluna75c66762010-03-30 16:31:14 +0000592 interrupted = True
Antoine Pitrou4698d992009-05-31 14:20:14 +0000593 break
594 except:
595 raise
596 if findleaks:
597 gc.collect()
598 if gc.garbage:
599 print "Warning: test created", len(gc.garbage),
600 print "uncollectable object(s)."
601 # move the uncollectable objects somewhere so we don't see
602 # them again
603 found_garbage.extend(gc.garbage)
604 del gc.garbage[:]
605 # Unload the newly imported modules (best effort finalization)
606 for module in sys.modules.keys():
607 if module not in save_modules and module.startswith("test."):
608 test_support.unload(module)
Jeremy Hylton7a1ea0e2001-10-17 13:45:28 +0000609
Florent Xicluna75c66762010-03-30 16:31:14 +0000610 if interrupted:
611 # print a newline after ^C
612 print
613 print "Test suite interrupted by signal SIGINT."
614 omitted = set(selected) - set(good) - set(bad) - set(skipped)
615 print count(len(omitted), "test"), "omitted:"
616 printlist(omitted)
Guido van Rossum152494a1996-12-20 03:12:20 +0000617 if good and not quiet:
Florent Xicluna75c66762010-03-30 16:31:14 +0000618 if not bad and not skipped and not interrupted and len(good) > 1:
Guido van Rossum41360a41998-03-26 19:42:58 +0000619 print "All",
620 print count(len(good), "test"), "OK."
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000621 if print_slow:
622 test_times.sort(reverse=True)
623 print "10 slowest tests:"
624 for time, test in test_times[:10]:
625 print "%s: %.1fs" % (test, time)
Guido van Rossum152494a1996-12-20 03:12:20 +0000626 if bad:
Zachary Warece698f32015-08-08 22:01:20 -0500627 print count(len(bad), "test"), "failed:"
628 printlist(bad)
629 if environment_changed:
630 print "{} altered the execution environment:".format(
631 count(len(environment_changed), "test"))
632 printlist(environment_changed)
Guido van Rossum152494a1996-12-20 03:12:20 +0000633 if skipped and not quiet:
Tim Petersa45da922001-08-12 03:45:50 +0000634 print count(len(skipped), "test"), "skipped:"
635 printlist(skipped)
Barry Warsawe11e3de1999-01-28 19:51:51 +0000636
Tim Petersb5b7b782001-08-12 01:20:39 +0000637 e = _ExpectedSkips()
Tim Petersa2be2d62001-08-12 02:01:09 +0000638 plat = sys.platform
Tim Petersb5b7b782001-08-12 01:20:39 +0000639 if e.isvalid():
Raymond Hettingera690a992003-11-16 16:17:49 +0000640 surprise = set(skipped) - e.getexpected() - set(resource_denieds)
Tim Petersb5b7b782001-08-12 01:20:39 +0000641 if surprise:
642 print count(len(surprise), "skip"), \
Tim Petersa45da922001-08-12 03:45:50 +0000643 "unexpected on", plat + ":"
644 printlist(surprise)
Tim Petersb5b7b782001-08-12 01:20:39 +0000645 else:
646 print "Those skips are all expected on", plat + "."
647 else:
648 print "Ask someone to teach regrtest.py about which tests are"
649 print "expected to get skipped on", plat + "."
650
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000651 if verbose2 and bad:
652 print "Re-running failed tests in verbose mode"
Zachary Warea4777f12015-08-04 21:53:21 -0500653 for test in bad[:]:
Tim Peters922dd7d2006-03-10 23:37:10 +0000654 print "Re-running test %r in verbose mode" % test
655 sys.stdout.flush()
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000656 try:
Neal Norwitz5fe5cf62008-03-18 17:58:02 +0000657 test_support.verbose = True
R. David Murray561b96f2011-02-11 17:25:54 +0000658 ok = runtest(test, True, quiet, huntrleaks)
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000659 except KeyboardInterrupt:
660 # print a newline separate from the ^C
661 print
662 break
Zachary Warea4777f12015-08-04 21:53:21 -0500663 else:
664 if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}:
665 bad.remove(test)
666 else:
667 if bad:
668 print count(len(bad), "test"), "failed again:"
669 printlist(bad)
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000670
Barry Warsawe11e3de1999-01-28 19:51:51 +0000671 if single:
R. David Murray98e3df32009-12-16 11:49:46 +0000672 if next_single_test:
673 with open(filename, 'w') as fp:
674 fp.write(next_single_test + '\n')
Barry Warsawe11e3de1999-01-28 19:51:51 +0000675 else:
676 os.unlink(filename)
677
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000678 if trace:
679 r = tracer.results()
680 r.write_results(show_missing=True, summary=True, coverdir=coverdir)
681
Skip Montanaro0179a182004-06-06 15:53:18 +0000682 if runleaks:
683 os.system("leaks %d" % os.getpid())
684
Florent Xicluna75c66762010-03-30 16:31:14 +0000685 sys.exit(len(bad) > 0 or interrupted)
Barry Warsaw08fca522001-08-20 22:33:46 +0000686
Guido van Rossum152494a1996-12-20 03:12:20 +0000687
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000688STDTESTS = [
Guido van Rossum152494a1996-12-20 03:12:20 +0000689 'test_grammar',
690 'test_opcodes',
Collin Winterf567ca32007-03-12 15:57:19 +0000691 'test_dict',
Guido van Rossum152494a1996-12-20 03:12:20 +0000692 'test_builtin',
693 'test_exceptions',
694 'test_types',
Collin Winter55453142007-03-08 19:58:14 +0000695 'test_unittest',
696 'test_doctest',
697 'test_doctest2',
Florent Xicluna12d750d2010-03-06 14:38:09 +0000698]
Guido van Rossum152494a1996-12-20 03:12:20 +0000699
Florent Xicluna12d750d2010-03-06 14:38:09 +0000700NOTTESTS = {
Guido van Rossum152494a1996-12-20 03:12:20 +0000701 'test_support',
Jeremy Hylton62e2c7e2001-02-28 17:48:06 +0000702 'test_future1',
703 'test_future2',
Florent Xicluna12d750d2010-03-06 14:38:09 +0000704}
Guido van Rossum152494a1996-12-20 03:12:20 +0000705
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000706def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):
Guido van Rossum152494a1996-12-20 03:12:20 +0000707 """Return a list of all applicable test modules."""
Florent Xicluna0932dc52010-03-06 08:07:44 +0000708 testdir = findtestdir(testdir)
Guido van Rossum152494a1996-12-20 03:12:20 +0000709 names = os.listdir(testdir)
710 tests = []
Florent Xicluna12d750d2010-03-06 14:38:09 +0000711 others = set(stdtests) | nottests
Guido van Rossum152494a1996-12-20 03:12:20 +0000712 for name in names:
Florent Xicluna0932dc52010-03-06 08:07:44 +0000713 modname, ext = os.path.splitext(name)
714 if modname[:5] == "test_" and ext == ".py" and modname not in others:
715 tests.append(modname)
716 return stdtests + sorted(tests)
Guido van Rossum152494a1996-12-20 03:12:20 +0000717
Antoine Pitrou4698d992009-05-31 14:20:14 +0000718def runtest(test, verbose, quiet,
R. David Murray561b96f2011-02-11 17:25:54 +0000719 huntrleaks=False, use_resources=None):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000720 """Run a single test.
Tim Peterse5584862006-06-09 19:24:44 +0000721
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000722 test -- the name of the test
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000723 verbose -- if true, print more messages
Trent Mickf29f47b2000-08-11 19:02:59 +0000724 quiet -- if true, don't print 'skipped' messages (probably redundant)
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000725 test_times -- a list of (time, test_name) pairs
Tim Peterse5584862006-06-09 19:24:44 +0000726 huntrleaks -- run multiple times to test for leaks; requires a debug
727 build; a triple corresponding to -R's three arguments
Georg Brandl4045e882009-10-29 20:53:00 +0000728 Returns one of the test result constants:
729 INTERRUPTED KeyboardInterrupt when run under -j
730 RESOURCE_DENIED test skipped because resource denied
731 SKIPPED test skipped for some other reason
732 ENV_CHANGED test failed because it changed the execution environment
733 FAILED test failed
734 PASSED test passed
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000735 """
Tim Peters06c5c002006-06-05 00:55:26 +0000736
Antoine Pitrou4698d992009-05-31 14:20:14 +0000737 test_support.verbose = verbose # Tell tests to be moderately quiet
738 if use_resources is not None:
739 test_support.use_resources = use_resources
Tim Peterse5584862006-06-09 19:24:44 +0000740 try:
R. David Murray561b96f2011-02-11 17:25:54 +0000741 return runtest_inner(test, verbose, quiet, huntrleaks)
Tim Peterse5584862006-06-09 19:24:44 +0000742 finally:
743 cleanup_test_droppings(test, verbose)
744
R. David Murray14dcd432009-10-14 13:58:07 +0000745
746# Unit tests are supposed to leave the execution environment unchanged
747# once they complete. But sometimes tests have bugs, especially when
748# tests fail, and the changes to environment go on to mess up other
749# tests. This can cause issues with buildbot stability, since tests
750# are run in random order and so problems may appear to come and go.
751# There are a few things we can save and restore to mitigate this, and
752# the following context manager handles this task.
753
754class saved_test_environment:
755 """Save bits of the test environment and restore them at block exit.
756
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000757 with saved_test_environment(testname, verbose, quiet):
R. David Murray14dcd432009-10-14 13:58:07 +0000758 #stuff
759
760 Unless quiet is True, a warning is printed to stderr if any of
761 the saved items was changed by the test. The attribute 'changed'
762 is initially False, but is set to True if a change is detected.
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000763
764 If verbose is more than 1, the before and after state of changed
765 items is also printed.
R. David Murray14dcd432009-10-14 13:58:07 +0000766 """
767
768 changed = False
769
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000770 def __init__(self, testname, verbose=0, quiet=False):
R. David Murray14dcd432009-10-14 13:58:07 +0000771 self.testname = testname
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000772 self.verbose = verbose
R. David Murray14dcd432009-10-14 13:58:07 +0000773 self.quiet = quiet
774
775 # To add things to save and restore, add a name XXX to the resources list
776 # and add corresponding get_XXX/restore_XXX functions. get_XXX should
777 # return the value to be saved and compared against a second call to the
778 # get function when test execution completes. restore_XXX should accept
779 # the saved value and restore the resource using it. It will be called if
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000780 # and only if a change in the value is detected.
781 #
782 # Note: XXX will have any '.' replaced with '_' characters when determining
783 # the corresponding method names.
R. David Murray14dcd432009-10-14 13:58:07 +0000784
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000785 resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr',
Vinay Sajipda563bf2012-03-06 20:07:15 +0000786 'os.environ', 'sys.path', 'asyncore.socket_map',
Serhiy Storchakac0a2f2b2015-03-30 01:27:44 +0300787 'files',
Vinay Sajipda563bf2012-03-06 20:07:15 +0000788 )
R. David Murray14dcd432009-10-14 13:58:07 +0000789
790 def get_sys_argv(self):
Nick Coghlan8157e192009-10-17 15:48:16 +0000791 return id(sys.argv), sys.argv, sys.argv[:]
R. David Murray14dcd432009-10-14 13:58:07 +0000792 def restore_sys_argv(self, saved_argv):
Nick Coghlan8157e192009-10-17 15:48:16 +0000793 sys.argv = saved_argv[1]
794 sys.argv[:] = saved_argv[2]
R. David Murray14dcd432009-10-14 13:58:07 +0000795
796 def get_cwd(self):
797 return os.getcwd()
798 def restore_cwd(self, saved_cwd):
799 os.chdir(saved_cwd)
800
801 def get_sys_stdout(self):
802 return sys.stdout
803 def restore_sys_stdout(self, saved_stdout):
804 sys.stdout = saved_stdout
805
806 def get_sys_stderr(self):
807 return sys.stderr
808 def restore_sys_stderr(self, saved_stderr):
809 sys.stderr = saved_stderr
810
811 def get_sys_stdin(self):
812 return sys.stdin
813 def restore_sys_stdin(self, saved_stdin):
814 sys.stdin = saved_stdin
815
816 def get_os_environ(self):
Nick Coghlan8157e192009-10-17 15:48:16 +0000817 return id(os.environ), os.environ, dict(os.environ)
R. David Murray14dcd432009-10-14 13:58:07 +0000818 def restore_os_environ(self, saved_environ):
Nick Coghlan8157e192009-10-17 15:48:16 +0000819 os.environ = saved_environ[1]
R. David Murray14dcd432009-10-14 13:58:07 +0000820 os.environ.clear()
Nick Coghlan8157e192009-10-17 15:48:16 +0000821 os.environ.update(saved_environ[2])
R. David Murray14dcd432009-10-14 13:58:07 +0000822
823 def get_sys_path(self):
Nick Coghlan8157e192009-10-17 15:48:16 +0000824 return id(sys.path), sys.path, sys.path[:]
R. David Murray14dcd432009-10-14 13:58:07 +0000825 def restore_sys_path(self, saved_path):
Nick Coghlan8157e192009-10-17 15:48:16 +0000826 sys.path = saved_path[1]
827 sys.path[:] = saved_path[2]
R. David Murray14dcd432009-10-14 13:58:07 +0000828
Antoine Pitrou1e8a7542010-08-16 00:32:47 +0000829 def get_asyncore_socket_map(self):
830 asyncore = sys.modules.get('asyncore')
Antoine Pitroua71bfc42010-10-31 13:17:46 +0000831 # XXX Making a copy keeps objects alive until __exit__ gets called.
832 return asyncore and asyncore.socket_map.copy() or {}
Antoine Pitrou1e8a7542010-08-16 00:32:47 +0000833 def restore_asyncore_socket_map(self, saved_map):
834 asyncore = sys.modules.get('asyncore')
835 if asyncore is not None:
Antoine Pitroua71bfc42010-10-31 13:17:46 +0000836 asyncore.close_all(ignore_all=True)
Antoine Pitrou1e8a7542010-08-16 00:32:47 +0000837 asyncore.socket_map.update(saved_map)
838
Vinay Sajipda563bf2012-03-06 20:07:15 +0000839 def get_test_support_TESTFN(self):
840 if os.path.isfile(test_support.TESTFN):
841 result = 'f'
842 elif os.path.isdir(test_support.TESTFN):
843 result = 'd'
844 else:
845 result = None
846 return result
847 def restore_test_support_TESTFN(self, saved_value):
848 if saved_value is None:
849 if os.path.isfile(test_support.TESTFN):
850 os.unlink(test_support.TESTFN)
851 elif os.path.isdir(test_support.TESTFN):
852 shutil.rmtree(test_support.TESTFN)
853
Serhiy Storchakac0a2f2b2015-03-30 01:27:44 +0300854 def get_files(self):
855 return sorted(fn + ('/' if os.path.isdir(fn) else '')
856 for fn in os.listdir(os.curdir))
857 def restore_files(self, saved_value):
Victor Stinnerac0d1892015-03-30 01:07:29 +0200858 fn = test_support.TESTFN
Serhiy Storchakac0a2f2b2015-03-30 01:27:44 +0300859 if fn not in saved_value and (fn + '/') not in saved_value:
860 if os.path.isfile(fn):
Victor Stinnerac0d1892015-03-30 01:07:29 +0200861 test_support.unlink(fn)
Serhiy Storchakac0a2f2b2015-03-30 01:27:44 +0300862 elif os.path.isdir(fn):
Victor Stinnerac0d1892015-03-30 01:07:29 +0200863 test_support.rmtree(fn)
Serhiy Storchakac0a2f2b2015-03-30 01:27:44 +0300864
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000865 def resource_info(self):
866 for name in self.resources:
867 method_suffix = name.replace('.', '_')
868 get_name = 'get_' + method_suffix
869 restore_name = 'restore_' + method_suffix
870 yield name, getattr(self, get_name), getattr(self, restore_name)
871
R. David Murray14dcd432009-10-14 13:58:07 +0000872 def __enter__(self):
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000873 self.saved_values = dict((name, get()) for name, get, restore
874 in self.resource_info())
R. David Murray14dcd432009-10-14 13:58:07 +0000875 return self
876
877 def __exit__(self, exc_type, exc_val, exc_tb):
Antoine Pitroua71bfc42010-10-31 13:17:46 +0000878 saved_values = self.saved_values
879 del self.saved_values
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000880 for name, get, restore in self.resource_info():
881 current = get()
Antoine Pitroua71bfc42010-10-31 13:17:46 +0000882 original = saved_values.pop(name)
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000883 # Check for changes to the resource's value
884 if current != original:
R. David Murray14dcd432009-10-14 13:58:07 +0000885 self.changed = True
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000886 restore(original)
R. David Murray14dcd432009-10-14 13:58:07 +0000887 if not self.quiet:
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000888 print >>sys.stderr, (
889 "Warning -- {} was modified by {}".format(
890 name, self.testname))
891 if self.verbose > 1:
892 print >>sys.stderr, (
Nick Coghlana3e97ad2009-10-17 07:34:27 +0000893 " Before: {}\n After: {} ".format(
894 original, current))
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000895 # XXX (ncoghlan): for most resources (e.g. sys.path) identity
896 # matters at least as much as value. For others (e.g. cwd),
897 # identity is irrelevant. Should we add a mechanism to check
898 # for substitution in the cases where it matters?
R. David Murray14dcd432009-10-14 13:58:07 +0000899 return False
900
901
R. David Murray561b96f2011-02-11 17:25:54 +0000902def runtest_inner(test, verbose, quiet, huntrleaks=False):
Guido van Rossum152494a1996-12-20 03:12:20 +0000903 test_support.unload(test)
Tim Peters9390cc12001-09-28 20:14:46 +0000904 if verbose:
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000905 capture_stdout = None
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000906 else:
Michael Foord91a2c892010-04-08 00:04:24 +0000907 capture_stdout = StringIO.StringIO()
Tim Peters06c5c002006-06-05 00:55:26 +0000908
Antoine Pitrou4698d992009-05-31 14:20:14 +0000909 test_time = 0.0
Collin Winter0f489742009-05-14 23:26:30 +0000910 refleak = False # True if the test leaked references.
Guido van Rossum152494a1996-12-20 03:12:20 +0000911 try:
Tim Peters342ca752001-09-25 19:13:20 +0000912 save_stdout = sys.stdout
Guido van Rossum41360a41998-03-26 19:42:58 +0000913 try:
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000914 if capture_stdout:
915 sys.stdout = capture_stdout
Barry Warsaw408b6d32002-07-30 23:27:12 +0000916 if test.startswith('test.'):
917 abstest = test
918 else:
919 # Always import it from the test package
920 abstest = 'test.' + test
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000921 with saved_test_environment(test, verbose, quiet) as environment:
R. David Murray14dcd432009-10-14 13:58:07 +0000922 start_time = time.time()
923 the_package = __import__(abstest, globals(), locals(), [])
924 the_module = getattr(the_package, test)
925 # Old tests run to completion simply as a side-effect of
926 # being imported. For tests based on unittest or doctest,
927 # explicitly invoke their test_main() function (if it exists).
928 indirect_test = getattr(the_module, "test_main", None)
929 if indirect_test is not None:
930 indirect_test()
931 if huntrleaks:
932 refleak = dash_R(the_module, test, indirect_test,
933 huntrleaks)
934 test_time = time.time() - start_time
Guido van Rossum41360a41998-03-26 19:42:58 +0000935 finally:
Tim Peters342ca752001-09-25 19:13:20 +0000936 sys.stdout = save_stdout
Fred Drake9a0db072003-02-03 15:19:30 +0000937 except test_support.ResourceDenied, msg:
938 if not quiet:
939 print test, "skipped --", msg
940 sys.stdout.flush()
Georg Brandl4045e882009-10-29 20:53:00 +0000941 return RESOURCE_DENIED, test_time
R. David Murray612f1d52009-03-31 19:49:15 +0000942 except unittest.SkipTest, msg:
Trent Mickf29f47b2000-08-11 19:02:59 +0000943 if not quiet:
Fred Drakede4742b2002-10-17 20:36:08 +0000944 print test, "skipped --", msg
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000945 sys.stdout.flush()
Georg Brandl4045e882009-10-29 20:53:00 +0000946 return SKIPPED, test_time
Fred Drakefe5c22a2000-08-18 16:04:05 +0000947 except KeyboardInterrupt:
948 raise
Guido van Rossum152494a1996-12-20 03:12:20 +0000949 except test_support.TestFailed, msg:
R. David Murray0960bc52010-12-04 17:24:46 +0000950 print >>sys.stderr, "test", test, "failed --", msg
951 sys.stderr.flush()
Georg Brandl4045e882009-10-29 20:53:00 +0000952 return FAILED, test_time
Guido van Rossum9e48b271997-07-16 01:56:13 +0000953 except:
Guido van Rossum41360a41998-03-26 19:42:58 +0000954 type, value = sys.exc_info()[:2]
R. David Murray0960bc52010-12-04 17:24:46 +0000955 print >>sys.stderr, "test", test, "crashed --", str(type) + ":", value
956 sys.stderr.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000957 if verbose:
R. David Murray0960bc52010-12-04 17:24:46 +0000958 traceback.print_exc(file=sys.stderr)
959 sys.stderr.flush()
Georg Brandl4045e882009-10-29 20:53:00 +0000960 return FAILED, test_time
Guido van Rossum152494a1996-12-20 03:12:20 +0000961 else:
Collin Winter0f489742009-05-14 23:26:30 +0000962 if refleak:
Georg Brandl4045e882009-10-29 20:53:00 +0000963 return FAILED, test_time
R. David Murray14dcd432009-10-14 13:58:07 +0000964 if environment.changed:
Georg Brandl93c21712009-10-29 21:44:56 +0000965 return ENV_CHANGED, test_time
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000966 # Except in verbose mode, tests should not print anything
967 if verbose or huntrleaks:
Georg Brandl4045e882009-10-29 20:53:00 +0000968 return PASSED, test_time
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000969 output = capture_stdout.getvalue()
970 if not output:
Georg Brandl4045e882009-10-29 20:53:00 +0000971 return PASSED, test_time
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000972 print "test", test, "produced unexpected output:"
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000973 print "*" * 70
974 print output
975 print "*" * 70
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000976 sys.stdout.flush()
Georg Brandl4045e882009-10-29 20:53:00 +0000977 return FAILED, test_time
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000978
Tim Peterse5584862006-06-09 19:24:44 +0000979def cleanup_test_droppings(testname, verbose):
Benjamin Petersona9b4d472009-04-29 22:44:15 +0000980 import stat
Antoine Pitrou0d759572010-10-31 13:15:24 +0000981 import gc
982
983 # First kill any dangling references to open files etc.
984 gc.collect()
Tim Peterse5584862006-06-09 19:24:44 +0000985
986 # Try to clean up junk commonly left behind. While tests shouldn't leave
987 # any files or directories behind, when a test fails that can be tedious
988 # for it to arrange. The consequences can be especially nasty on Windows,
989 # since if a test leaves a file open, it cannot be deleted by name (while
990 # there's nothing we can do about that here either, we can display the
991 # name of the offending test, which is a real help).
992 for name in (test_support.TESTFN,
993 "db_home",
994 ):
995 if not os.path.exists(name):
996 continue
997
998 if os.path.isdir(name):
999 kind, nuker = "directory", shutil.rmtree
1000 elif os.path.isfile(name):
1001 kind, nuker = "file", os.unlink
1002 else:
1003 raise SystemError("os.path says %r exists but is neither "
1004 "directory nor file" % name)
1005
1006 if verbose:
1007 print "%r left behind %s %r" % (testname, kind, name)
1008 try:
Benjamin Petersona9b4d472009-04-29 22:44:15 +00001009 # if we have chmod, fix possible permissions problems
1010 # that might prevent cleanup
1011 if (hasattr(os, 'chmod')):
1012 os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO)
Tim Peterse5584862006-06-09 19:24:44 +00001013 nuker(name)
1014 except Exception, msg:
1015 print >> sys.stderr, ("%r left behind %s %r and it couldn't be "
1016 "removed: %s" % (testname, kind, name, msg))
1017
Tim Peters06c5c002006-06-05 00:55:26 +00001018def dash_R(the_module, test, indirect_test, huntrleaks):
Collin Winter0f489742009-05-14 23:26:30 +00001019 """Run a test multiple times, looking for reference leaks.
1020
1021 Returns:
1022 False if the test didn't leak references; True if we detected refleaks.
1023 """
Tim Peters06c5c002006-06-05 00:55:26 +00001024 # This code is hackish and inelegant, but it seems to do the job.
Amaury Forgeot d'Arce55df1f2009-07-28 20:47:55 +00001025 import copy_reg, _abcoll, _pyio
Tim Peters06c5c002006-06-05 00:55:26 +00001026
1027 if not hasattr(sys, 'gettotalrefcount'):
1028 raise Exception("Tracking reference leaks requires a debug build "
1029 "of Python")
1030
1031 # Save current values for dash_R_cleanup() to restore.
1032 fs = warnings.filters[:]
Georg Brandldffbf5f2008-05-20 07:49:57 +00001033 ps = copy_reg.dispatch_table.copy()
Tim Peters06c5c002006-06-05 00:55:26 +00001034 pic = sys.path_importer_cache.copy()
Nick Coghlan4976e322009-11-16 03:55:51 +00001035 try:
1036 import zipimport
1037 except ImportError:
Benjamin Peterson55035792014-02-16 14:51:17 -05001038 zdc = None # Run unmodified on platforms without zipimport support
Nick Coghlan4976e322009-11-16 03:55:51 +00001039 else:
1040 zdc = zipimport._zip_directory_cache.copy()
Guido van Rossum64c06e32007-11-22 00:55:51 +00001041 abcs = {}
Amaury Forgeot d'Arce55df1f2009-07-28 20:47:55 +00001042 modules = _abcoll, _pyio
Amaury Forgeot d'Arc4d0c1172008-04-02 00:25:14 +00001043 for abc in [getattr(mod, a) for mod in modules for a in mod.__all__]:
1044 # XXX isinstance(abc, ABCMeta) leads to infinite recursion
1045 if not hasattr(abc, '_abc_registry'):
Christian Heimes6b29dd02007-11-30 22:36:10 +00001046 continue
Guido van Rossum64c06e32007-11-22 00:55:51 +00001047 for obj in abc.__subclasses__() + [abc]:
1048 abcs[obj] = obj._abc_registry.copy()
1049
Tim Peters06c5c002006-06-05 00:55:26 +00001050 if indirect_test:
1051 def run_the_test():
1052 indirect_test()
1053 else:
1054 def run_the_test():
Ezio Melottidde5b942010-02-03 05:37:26 +00001055 imp.reload(the_module)
Tim Peters06c5c002006-06-05 00:55:26 +00001056
1057 deltas = []
1058 nwarmup, ntracked, fname = huntrleaks
Ezio Melotti8f6a2872010-02-10 21:40:33 +00001059 fname = os.path.join(test_support.SAVEDCWD, fname)
Tim Peters06c5c002006-06-05 00:55:26 +00001060 repcount = nwarmup + ntracked
1061 print >> sys.stderr, "beginning", repcount, "repetitions"
1062 print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount]
Benjamin Peterson55035792014-02-16 14:51:17 -05001063 dash_R_cleanup(fs, ps, pic, zdc, abcs)
Tim Peters06c5c002006-06-05 00:55:26 +00001064 for i in range(repcount):
Collin Winterac1d9312010-03-17 02:02:30 +00001065 rc_before = sys.gettotalrefcount()
Tim Peters06c5c002006-06-05 00:55:26 +00001066 run_the_test()
1067 sys.stderr.write('.')
Benjamin Peterson55035792014-02-16 14:51:17 -05001068 dash_R_cleanup(fs, ps, pic, zdc, abcs)
Collin Winterac1d9312010-03-17 02:02:30 +00001069 rc_after = sys.gettotalrefcount()
Tim Peters06c5c002006-06-05 00:55:26 +00001070 if i >= nwarmup:
Collin Winterac1d9312010-03-17 02:02:30 +00001071 deltas.append(rc_after - rc_before)
Tim Peters06c5c002006-06-05 00:55:26 +00001072 print >> sys.stderr
1073 if any(deltas):
Neal Norwitzd9841032007-05-11 05:55:15 +00001074 msg = '%s leaked %s references, sum=%s' % (test, deltas, sum(deltas))
1075 print >> sys.stderr, msg
Antoine Pitrou4698d992009-05-31 14:20:14 +00001076 with open(fname, "a") as refrep:
1077 print >> refrep, msg
1078 refrep.flush()
Collin Winter0f489742009-05-14 23:26:30 +00001079 return True
1080 return False
Tim Peters06c5c002006-06-05 00:55:26 +00001081
Benjamin Peterson55035792014-02-16 14:51:17 -05001082def dash_R_cleanup(fs, ps, pic, zdc, abcs):
Georg Brandldffbf5f2008-05-20 07:49:57 +00001083 import gc, copy_reg
Brett Cannon0aa6e1b2008-05-10 21:12:57 +00001084 import _strptime, linecache
1085 dircache = test_support.import_module('dircache', deprecated=True)
Tim Peters06c5c002006-06-05 00:55:26 +00001086 import urlparse, urllib, urllib2, mimetypes, doctest
Amaury Forgeot d'Arc4d0c1172008-04-02 00:25:14 +00001087 import struct, filecmp
Tim Peters06c5c002006-06-05 00:55:26 +00001088 from distutils.dir_util import _path_created
1089
Amaury Forgeot d'Arc607bff12008-04-18 23:31:33 +00001090 # Clear the warnings registry, so they can be displayed again
1091 for mod in sys.modules.values():
1092 if hasattr(mod, '__warningregistry__'):
1093 del mod.__warningregistry__
1094
Tim Peters06c5c002006-06-05 00:55:26 +00001095 # Restore some original values.
1096 warnings.filters[:] = fs
Georg Brandldffbf5f2008-05-20 07:49:57 +00001097 copy_reg.dispatch_table.clear()
1098 copy_reg.dispatch_table.update(ps)
Tim Peters06c5c002006-06-05 00:55:26 +00001099 sys.path_importer_cache.clear()
1100 sys.path_importer_cache.update(pic)
Nick Coghlan4976e322009-11-16 03:55:51 +00001101 try:
1102 import zipimport
1103 except ImportError:
1104 pass # Run unmodified on platforms without zipimport support
1105 else:
1106 zipimport._zip_directory_cache.clear()
1107 zipimport._zip_directory_cache.update(zdc)
Tim Peters06c5c002006-06-05 00:55:26 +00001108
Christian Heimes908caac2008-01-27 23:34:59 +00001109 # clear type cache
Christian Heimes422051a2008-02-04 18:00:12 +00001110 sys._clear_type_cache()
Christian Heimes908caac2008-01-27 23:34:59 +00001111
Guido van Rossum64c06e32007-11-22 00:55:51 +00001112 # Clear ABC registries, restoring previously saved ABC registries.
Amaury Forgeot d'Arc4d0c1172008-04-02 00:25:14 +00001113 for abc, registry in abcs.items():
1114 abc._abc_registry = registry.copy()
1115 abc._abc_cache.clear()
1116 abc._abc_negative_cache.clear()
Guido van Rossum64c06e32007-11-22 00:55:51 +00001117
Tim Peters06c5c002006-06-05 00:55:26 +00001118 # Clear assorted module caches.
1119 _path_created.clear()
1120 re.purge()
1121 _strptime._regex_cache.clear()
1122 urlparse.clear_cache()
1123 urllib.urlcleanup()
1124 urllib2.install_opener(None)
1125 dircache.reset()
1126 linecache.clearcache()
1127 mimetypes._default_mime_types()
Tim Petersc27d6552006-06-07 01:04:59 +00001128 filecmp._cache.clear()
Christian Heimes76d19f62008-01-04 02:54:42 +00001129 struct._clearcache()
Tim Peters06c5c002006-06-05 00:55:26 +00001130 doctest.master = None
Meador Inge0e303172011-11-25 22:25:06 -06001131 try:
1132 import ctypes
1133 except ImportError:
1134 # Don't worry about resetting the cache if ctypes is not supported
1135 pass
1136 else:
1137 ctypes._reset_cache()
Tim Peters06c5c002006-06-05 00:55:26 +00001138
1139 # Collect cyclic trash.
1140 gc.collect()
1141
Florent Xicluna0932dc52010-03-06 08:07:44 +00001142def findtestdir(path=None):
1143 return path or os.path.dirname(__file__) or os.curdir
Guido van Rossum152494a1996-12-20 03:12:20 +00001144
Florent Xicluna0932dc52010-03-06 08:07:44 +00001145def removepy(names):
1146 if not names:
1147 return
1148 for idx, name in enumerate(names):
1149 basename, ext = os.path.splitext(name)
1150 if ext == '.py':
1151 names[idx] = basename
Tim Petersc5000df2002-06-02 21:42:01 +00001152
Guido van Rossum152494a1996-12-20 03:12:20 +00001153def count(n, word):
1154 if n == 1:
Guido van Rossum41360a41998-03-26 19:42:58 +00001155 return "%d %s" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +00001156 else:
Guido van Rossum41360a41998-03-26 19:42:58 +00001157 return "%d %ss" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +00001158
Tim Petersa45da922001-08-12 03:45:50 +00001159def printlist(x, width=70, indent=4):
Tim Peters7c7efe92002-08-23 17:55:54 +00001160 """Print the elements of iterable x to stdout.
Tim Petersa45da922001-08-12 03:45:50 +00001161
1162 Optional arg width (default 70) is the maximum line length.
1163 Optional arg indent (default 4) is the number of blanks with which to
1164 begin each line.
1165 """
1166
Tim Petersba78bc42002-07-04 19:45:06 +00001167 from textwrap import fill
1168 blanks = ' ' * indent
Florent Xicluna355adc52010-03-26 19:32:44 +00001169 # Print the sorted list: 'x' may be a '--random' list or a set()
1170 print fill(' '.join(str(elt) for elt in sorted(x)), width,
Tim Petersba78bc42002-07-04 19:45:06 +00001171 initial_indent=blanks, subsequent_indent=blanks)
Tim Petersa45da922001-08-12 03:45:50 +00001172
Tim Petersde14a302002-04-01 05:04:46 +00001173# Map sys.platform to a string containing the basenames of tests
1174# expected to be skipped on that platform.
Tim Peters2a182db2002-10-09 01:07:11 +00001175#
1176# Special cases:
1177# test_pep277
1178# The _ExpectedSkips constructor adds this to the set of expected
1179# skips if not os.path.supports_unicode_filenames.
Neal Norwitz55b61d22003-02-28 19:57:03 +00001180# test_timeout
1181# Controlled by test_timeout.skip_expected. Requires the network
1182# resource and a socket module.
Georg Brandlb2e208f2007-07-12 09:24:04 +00001183#
1184# Tests that are expected to be skipped everywhere except on one platform
1185# are also handled separately.
Tim Petersde14a302002-04-01 05:04:46 +00001186
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001187_expectations = {
1188 'win32':
1189 """
Tim Petersc7c516a2003-09-20 22:06:13 +00001190 test__locale
Skip Montanaro823ba282003-05-06 20:36:24 +00001191 test_bsddb185
Tim Peters78e35f92002-11-22 20:00:34 +00001192 test_bsddb3
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001193 test_commands
1194 test_crypt
Tim Petersd7030572001-10-22 22:06:08 +00001195 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001196 test_dbm
1197 test_dl
1198 test_fcntl
1199 test_fork1
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001200 test_epoll
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001201 test_gdbm
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001202 test_grp
Tim Petersfd8e6e52003-03-04 00:26:38 +00001203 test_ioctl
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001204 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001205 test_kqueue
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001206 test_mhlib
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001207 test_openpty
Tim Petersefc4b122002-12-10 18:47:56 +00001208 test_ossaudiodev
Georg Brandl27d9ee32007-08-30 10:38:56 +00001209 test_pipes
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001210 test_poll
Tim Peters003eb302003-02-17 21:48:48 +00001211 test_posix
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001212 test_pty
1213 test_pwd
Tim Peters1e33ffa2002-04-23 23:09:02 +00001214 test_resource
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001215 test_signal
Serhiy Storchaka7ece1502015-01-31 10:20:31 +02001216 test_spwd
Tim Peterscea2cc42004-08-04 02:32:03 +00001217 test_threadsignals
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001218 test_timing
Tim Petersb84de022006-03-22 02:58:17 +00001219 test_wait3
1220 test_wait4
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001221 """,
1222 'linux2':
1223 """
Skip Montanaro823ba282003-05-06 20:36:24 +00001224 test_bsddb185
Guido van Rossumf66dacd2001-10-23 15:10:55 +00001225 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001226 test_dl
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001227 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001228 test_kqueue
Guido van Rossum4507ec72003-02-14 19:29:22 +00001229 test_ossaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001230 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001231 'unixware7':
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001232 """
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001233 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001234 test_bsddb185
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001235 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001236 test_epoll
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001237 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001238 test_kqueue
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001239 test_minidom
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001240 test_openpty
1241 test_pyexpat
1242 test_sax
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001243 test_sundry
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001244 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001245 'openunix8':
1246 """
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001247 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001248 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001249 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001250 test_epoll
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001251 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001252 test_kqueue
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001253 test_minidom
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001254 test_openpty
1255 test_pyexpat
1256 test_sax
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001257 test_sundry
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001258 """,
1259 'sco_sv3':
1260 """
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001261 test_asynchat
1262 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001263 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001264 test_dl
1265 test_fork1
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001266 test_epoll
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001267 test_gettext
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001268 test_largefile
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001269 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001270 test_kqueue
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001271 test_minidom
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001272 test_openpty
1273 test_pyexpat
1274 test_queue
1275 test_sax
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001276 test_sundry
1277 test_thread
1278 test_threaded_import
1279 test_threadedtempfile
1280 test_threading
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001281 """,
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001282 'riscos':
1283 """
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001284 test_asynchat
Martin v. Löwisa94568a2003-05-10 07:36:56 +00001285 test_atexit
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001286 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001287 test_bsddb185
Martin v. Löwisa94568a2003-05-10 07:36:56 +00001288 test_bsddb3
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001289 test_commands
1290 test_crypt
1291 test_dbm
1292 test_dl
1293 test_fcntl
1294 test_fork1
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001295 test_epoll
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001296 test_gdbm
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001297 test_grp
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001298 test_largefile
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001299 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001300 test_kqueue
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001301 test_mmap
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001302 test_openpty
1303 test_poll
1304 test_popen2
1305 test_pty
1306 test_pwd
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001307 test_strop
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001308 test_sundry
1309 test_thread
1310 test_threaded_import
1311 test_threadedtempfile
1312 test_threading
1313 test_timing
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001314 """,
Jack Jansen8a97f4a2001-12-05 23:27:32 +00001315 'darwin':
Jack Jansen398c2362001-12-02 21:41:36 +00001316 """
Brett Cannon2bfb94c2003-10-13 04:27:47 +00001317 test__locale
Jack Jansenacda3392002-12-30 23:03:13 +00001318 test_bsddb
Guido van Rossum9d427002002-12-03 10:24:56 +00001319 test_bsddb3
Jack Jansen398c2362001-12-02 21:41:36 +00001320 test_curses
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001321 test_epoll
Skip Montanaro29cdfe62010-10-25 01:35:23 +00001322 test_gdb
Jack Jansen398c2362001-12-02 21:41:36 +00001323 test_gdbm
Jack Jansen398c2362001-12-02 21:41:36 +00001324 test_largefile
Jack Jansenacda3392002-12-30 23:03:13 +00001325 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001326 test_kqueue
Jack Jansen398c2362001-12-02 21:41:36 +00001327 test_minidom
Jack Jansenacda3392002-12-30 23:03:13 +00001328 test_ossaudiodev
Jack Jansen398c2362001-12-02 21:41:36 +00001329 test_poll
Jack Jansen398c2362001-12-02 21:41:36 +00001330 """,
Guido van Rossum11c3f092002-07-17 15:08:24 +00001331 'sunos5':
1332 """
Guido van Rossum11c3f092002-07-17 15:08:24 +00001333 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001334 test_bsddb185
Guido van Rossum11c3f092002-07-17 15:08:24 +00001335 test_curses
1336 test_dbm
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001337 test_epoll
1338 test_kqueue
Guido van Rossum11c3f092002-07-17 15:08:24 +00001339 test_gdbm
Guido van Rossum11c3f092002-07-17 15:08:24 +00001340 test_gzip
Guido van Rossum11c3f092002-07-17 15:08:24 +00001341 test_openpty
Guido van Rossum11c3f092002-07-17 15:08:24 +00001342 test_zipfile
1343 test_zlib
Jeremy Hyltoned375e12002-07-17 15:56:55 +00001344 """,
Skip Montanarob3230212002-03-15 02:54:03 +00001345 'hp-ux11':
1346 """
Skip Montanarob3230212002-03-15 02:54:03 +00001347 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001348 test_bsddb185
Skip Montanarob3230212002-03-15 02:54:03 +00001349 test_curses
1350 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001351 test_epoll
Skip Montanarob3230212002-03-15 02:54:03 +00001352 test_gdbm
Skip Montanarob3230212002-03-15 02:54:03 +00001353 test_gzip
Skip Montanarob3230212002-03-15 02:54:03 +00001354 test_largefile
Skip Montanarob3230212002-03-15 02:54:03 +00001355 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001356 test_kqueue
Skip Montanarob3230212002-03-15 02:54:03 +00001357 test_minidom
Skip Montanarob3230212002-03-15 02:54:03 +00001358 test_openpty
1359 test_pyexpat
1360 test_sax
Skip Montanarob3230212002-03-15 02:54:03 +00001361 test_zipfile
1362 test_zlib
1363 """,
Martin v. Löwisf90ae202002-06-11 06:22:31 +00001364 'atheos':
Tim Petersc411dba2002-07-16 21:35:23 +00001365 """
Skip Montanaro823ba282003-05-06 20:36:24 +00001366 test_bsddb185
Tim Petersc411dba2002-07-16 21:35:23 +00001367 test_curses
1368 test_dl
Tim Petersc411dba2002-07-16 21:35:23 +00001369 test_gdbm
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001370 test_epoll
Tim Petersc411dba2002-07-16 21:35:23 +00001371 test_largefile
Tim Petersc411dba2002-07-16 21:35:23 +00001372 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001373 test_kqueue
Tim Petersc411dba2002-07-16 21:35:23 +00001374 test_mhlib
1375 test_mmap
Tim Petersc411dba2002-07-16 21:35:23 +00001376 test_poll
1377 test_popen2
1378 test_resource
Tim Petersc411dba2002-07-16 21:35:23 +00001379 """,
Jason Tishler25115942002-12-05 15:18:15 +00001380 'cygwin':
1381 """
Skip Montanaro823ba282003-05-06 20:36:24 +00001382 test_bsddb185
Tim Petersb0f89e02002-12-05 17:20:25 +00001383 test_bsddb3
Jason Tishler25115942002-12-05 15:18:15 +00001384 test_curses
1385 test_dbm
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001386 test_epoll
Jason Tishlerc23f39c2003-07-22 18:35:58 +00001387 test_ioctl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001388 test_kqueue
Jason Tishler25115942002-12-05 15:18:15 +00001389 test_largefile
Jason Tishler25115942002-12-05 15:18:15 +00001390 test_locale
Jason Tishler5c4ded22003-02-05 16:46:01 +00001391 test_ossaudiodev
Jason Tishler25115942002-12-05 15:18:15 +00001392 test_socketserver
Jason Tishler25115942002-12-05 15:18:15 +00001393 """,
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001394 'os2emx':
1395 """
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001396 test_audioop
Skip Montanaro823ba282003-05-06 20:36:24 +00001397 test_bsddb185
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001398 test_bsddb3
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001399 test_commands
1400 test_curses
1401 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001402 test_epoll
1403 test_kqueue
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001404 test_largefile
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001405 test_mhlib
1406 test_mmap
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001407 test_openpty
1408 test_ossaudiodev
1409 test_pty
1410 test_resource
1411 test_signal
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001412 """,
Guido van Rossum944a6c32003-11-20 22:11:29 +00001413 'freebsd4':
1414 """
Guido van Rossum944a6c32003-11-20 22:11:29 +00001415 test_bsddb
1416 test_bsddb3
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001417 test_epoll
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001418 test_gdbm
Guido van Rossum944a6c32003-11-20 22:11:29 +00001419 test_locale
Guido van Rossum944a6c32003-11-20 22:11:29 +00001420 test_ossaudiodev
1421 test_pep277
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001422 test_pty
Guido van Rossum944a6c32003-11-20 22:11:29 +00001423 test_socketserver
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001424 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001425 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001426 test_ttk_guionly
1427 test_ttk_textonly
Guido van Rossum944a6c32003-11-20 22:11:29 +00001428 test_timeout
Guido van Rossum944a6c32003-11-20 22:11:29 +00001429 test_urllibnet
Jesse Noller37040cd2008-09-30 00:15:45 +00001430 test_multiprocessing
Martin v. Löwis56f88112003-06-07 20:01:37 +00001431 """,
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001432 'aix5':
1433 """
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001434 test_bsddb
1435 test_bsddb185
1436 test_bsddb3
1437 test_bz2
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001438 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001439 test_epoll
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001440 test_gdbm
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001441 test_gzip
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001442 test_kqueue
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001443 test_ossaudiodev
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001444 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001445 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001446 test_ttk_guionly
1447 test_ttk_textonly
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001448 test_zipimport
1449 test_zlib
1450 """,
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001451 'openbsd3':
1452 """
Florent Xicluna27e02402010-03-24 17:32:58 +00001453 test_ascii_formatd
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001454 test_bsddb
1455 test_bsddb3
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001456 test_ctypes
1457 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001458 test_epoll
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001459 test_gdbm
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001460 test_locale
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001461 test_normalization
1462 test_ossaudiodev
1463 test_pep277
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001464 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001465 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001466 test_ttk_guionly
1467 test_ttk_textonly
Jesse Noller37040cd2008-09-30 00:15:45 +00001468 test_multiprocessing
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001469 """,
Georg Brandl08706872006-06-21 17:53:17 +00001470 'netbsd3':
1471 """
Florent Xicluna27e02402010-03-24 17:32:58 +00001472 test_ascii_formatd
Georg Brandl08706872006-06-21 17:53:17 +00001473 test_bsddb
1474 test_bsddb185
1475 test_bsddb3
Georg Brandl08706872006-06-21 17:53:17 +00001476 test_ctypes
1477 test_curses
1478 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001479 test_epoll
Georg Brandl08706872006-06-21 17:53:17 +00001480 test_gdbm
Georg Brandl08706872006-06-21 17:53:17 +00001481 test_locale
Georg Brandl08706872006-06-21 17:53:17 +00001482 test_ossaudiodev
1483 test_pep277
Georg Brandl08706872006-06-21 17:53:17 +00001484 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001485 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001486 test_ttk_guionly
1487 test_ttk_textonly
Jesse Noller37040cd2008-09-30 00:15:45 +00001488 test_multiprocessing
Georg Brandl08706872006-06-21 17:53:17 +00001489 """,
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001490}
Martin v. Löwis32d0c1b2004-07-26 12:09:13 +00001491_expectations['freebsd5'] = _expectations['freebsd4']
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001492_expectations['freebsd6'] = _expectations['freebsd4']
Hye-Shik Chang4e422812005-07-17 02:36:59 +00001493_expectations['freebsd7'] = _expectations['freebsd4']
Hye-Shik Changea684742007-10-28 12:38:09 +00001494_expectations['freebsd8'] = _expectations['freebsd4']
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001495
Tim Petersb5b7b782001-08-12 01:20:39 +00001496class _ExpectedSkips:
1497 def __init__(self):
Tim Peters2a182db2002-10-09 01:07:11 +00001498 import os.path
Neal Norwitz55b61d22003-02-28 19:57:03 +00001499 from test import test_timeout
Tim Peters1b445d32002-11-24 18:53:11 +00001500
Tim Peters7c7efe92002-08-23 17:55:54 +00001501 self.valid = False
Tim Petersde14a302002-04-01 05:04:46 +00001502 if sys.platform in _expectations:
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001503 s = _expectations[sys.platform]
Raymond Hettingera690a992003-11-16 16:17:49 +00001504 self.expected = set(s.split())
Tim Peters1b445d32002-11-24 18:53:11 +00001505
Georg Brandlb2e208f2007-07-12 09:24:04 +00001506 # expected to be skipped on every platform, even Linux
1507 self.expected.add('test_linuxaudiodev')
1508
Tim Peters2a182db2002-10-09 01:07:11 +00001509 if not os.path.supports_unicode_filenames:
1510 self.expected.add('test_pep277')
Tim Peters1b445d32002-11-24 18:53:11 +00001511
Neal Norwitz55b61d22003-02-28 19:57:03 +00001512 if test_timeout.skip_expected:
1513 self.expected.add('test_timeout')
1514
Martin v. Löwisfba73692004-11-13 11:13:35 +00001515 if sys.maxint == 9223372036854775807L:
Martin v. Löwisfba73692004-11-13 11:13:35 +00001516 self.expected.add('test_imageop')
1517
Ronald Oussoren9545a232010-05-05 19:09:31 +00001518 if sys.platform != "darwin":
Amaury Forgeot d'Arc8f40f062008-06-19 19:57:39 +00001519 MAC_ONLY = ["test_macos", "test_macostools", "test_aepack",
Georg Brandlb2e208f2007-07-12 09:24:04 +00001520 "test_plistlib", "test_scriptpackages",
1521 "test_applesingle"]
Neal Norwitz7035c982003-03-29 22:01:17 +00001522 for skip in MAC_ONLY:
1523 self.expected.add(skip)
Benjamin Peterson6f5a2b52008-06-19 21:39:06 +00001524 elif len(u'\0'.encode('unicode-internal')) == 4:
1525 self.expected.add("test_macostools")
1526
Tim Petersecd79eb2003-01-29 00:35:32 +00001527
1528 if sys.platform != "win32":
Georg Brandlb2e208f2007-07-12 09:24:04 +00001529 # test_sqlite is only reliable on Windows where the library
1530 # is distributed with Python
Neal Norwitz7035c982003-03-29 22:01:17 +00001531 WIN_ONLY = ["test_unicode_file", "test_winreg",
Georg Brandlb2e208f2007-07-12 09:24:04 +00001532 "test_winsound", "test_startfile",
Ross Lagerwallfe2f1ad2011-04-09 20:39:50 +02001533 "test_sqlite", "test_msilib"]
Neal Norwitz7035c982003-03-29 22:01:17 +00001534 for skip in WIN_ONLY:
1535 self.expected.add(skip)
Tim Petersf2715e02003-02-19 02:35:07 +00001536
Brett Cannondc48b742007-05-20 07:09:50 +00001537 if sys.platform != 'irix':
Georg Brandlb2e208f2007-07-12 09:24:04 +00001538 IRIX_ONLY = ["test_imageop", "test_al", "test_cd", "test_cl",
1539 "test_gl", "test_imgfile"]
Brett Cannondc48b742007-05-20 07:09:50 +00001540 for skip in IRIX_ONLY:
1541 self.expected.add(skip)
1542
Georg Brandlb2e208f2007-07-12 09:24:04 +00001543 if sys.platform != 'sunos5':
1544 self.expected.add('test_sunaudiodev')
1545 self.expected.add('test_nis')
1546
Steven Bethardd290b042008-03-21 21:01:18 +00001547 if not sys.py3kwarning:
1548 self.expected.add('test_py3kwarn')
Steven Betharde8e22cf2008-03-18 21:30:13 +00001549
Tim Peters7c7efe92002-08-23 17:55:54 +00001550 self.valid = True
Tim Petersb5b7b782001-08-12 01:20:39 +00001551
1552 def isvalid(self):
1553 "Return true iff _ExpectedSkips knows about the current platform."
1554 return self.valid
1555
1556 def getexpected(self):
1557 """Return set of test names we expect to skip on current platform.
1558
1559 self.isvalid() must be true.
1560 """
1561
1562 assert self.isvalid()
1563 return self.expected
1564
Guido van Rossum152494a1996-12-20 03:12:20 +00001565if __name__ == '__main__':
Florent Xicluna92847452010-09-13 17:36:36 +00001566 # findtestdir() gets the dirname out of __file__, so we have to make it
1567 # absolute before changing the working directory.
1568 # For example __file__ may be relative when running trace or profile.
1569 # See issue #9323.
1570 __file__ = os.path.abspath(__file__)
1571
1572 # sanity check
Florent Xicluna0932dc52010-03-06 08:07:44 +00001573 assert __file__ == os.path.abspath(sys.argv[0])
Ezio Melotti8f6a2872010-02-10 21:40:33 +00001574
Florent Xicluna0932dc52010-03-06 08:07:44 +00001575 # When tests are run from the Python build directory, it is best practice
1576 # to keep the test files in a subfolder. It eases the cleanup of leftover
1577 # files using command "make distclean".
1578 if sysconfig.is_python_build():
Florent Xicluna12735662010-04-23 18:10:12 +00001579 TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build')
1580 TEMPDIR = os.path.abspath(TEMPDIR)
1581 if not os.path.exists(TEMPDIR):
1582 os.mkdir(TEMPDIR)
Ezio Melotti8f6a2872010-02-10 21:40:33 +00001583
1584 # Define a writable temp dir that will be used as cwd while running
1585 # the tests. The name of the dir includes the pid to allow parallel
1586 # testing (see the -j option).
1587 TESTCWD = 'test_python_{}'.format(os.getpid())
1588
Florent Xicluna12735662010-04-23 18:10:12 +00001589 TESTCWD = os.path.join(TEMPDIR, TESTCWD)
Ezio Melotti8f6a2872010-02-10 21:40:33 +00001590
1591 # Run the tests in a context manager that temporary changes the CWD to a
1592 # temporary and writable directory. If it's not possible to create or
1593 # change the CWD, the original CWD will be used. The original CWD is
1594 # available from test_support.SAVEDCWD.
1595 with test_support.temp_cwd(TESTCWD, quiet=True):
1596 main()