blob: eb7155a8bc40bf2617ef0e0d5ecbd6d597caab92 [file] [log] [blame]
Guido van Rossum152494a1996-12-20 03:12:20 +00001#! /usr/bin/env python
2
Martin v. Löwis5f2e0e52008-05-05 17:54:01 +00003"""Regression test.
Guido van Rossum152494a1996-12-20 03:12:20 +00004
5This will find all modules whose name is "test_*" in the test
6directory, and run them. Various command line options provide
7additional facilities.
8
9Command line options:
10
Michael W. Hudson61147f62004-08-03 11:33:28 +000011-v: verbose -- run tests in verbose mode with output to stdout
Martin v. Löwis04824ce2006-03-10 21:26:16 +000012-w: verbose2 -- re-run failed tests in verbose mode
Michael W. Hudson61147f62004-08-03 11:33:28 +000013-q: quiet -- don't print anything except if a test fails
Michael W. Hudson61147f62004-08-03 11:33:28 +000014-x: exclude -- arguments are tests to *exclude*
15-s: single -- run only a single test (see below)
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +000016-S: slow -- print the slowest 10 tests
Michael W. Hudson61147f62004-08-03 11:33:28 +000017-r: random -- randomize test execution order
18-f: fromfile -- read names of tests to run from a file (see below)
19-l: findleaks -- if GC is available detect tests that leak memory
20-u: use -- specify which special resource intensive tests to run
21-h: help -- print this text and exit
22-t: threshold -- call gc.set_threshold(N)
23-T: coverage -- turn on code coverage using the trace module
Walter Dörwaldaee4da62004-11-12 18:51:27 +000024-D: coverdir -- Directory where coverage files are put
25-N: nocoverdir -- Put coverage files alongside modules
Michael W. Hudson61147f62004-08-03 11:33:28 +000026-L: runleaks -- run the leaks(1) command just before exit
27-R: huntrleaks -- search for reference leaks (needs debug build, v. slow)
Thomas Woutersabd08882006-04-26 15:53:30 +000028-M: memlimit -- run very large memory-consuming tests
Antoine Pitrou4698d992009-05-31 14:20:14 +000029-j: multiprocess -- run several processes at once
Guido van Rossum152494a1996-12-20 03:12:20 +000030
31If non-option arguments are present, they are names for tests to run,
32unless -x is given, in which case they are names for tests not to run.
33If no test names are given, all tests are run.
Guido van Rossumf58ed251997-03-07 21:04:33 +000034
Collin Wintera199f012009-03-29 03:44:19 +000035-r randomizes test execution order. You can use --randseed=int to provide a
36int seed value for the randomizer; this is useful for reproducing troublesome
37test orders.
38
Barry Warsaw3b6d0252004-02-07 22:43:03 +000039-T turns on code coverage tracing with the trace module.
40
Walter Dörwaldaee4da62004-11-12 18:51:27 +000041-D specifies the directory where coverage files are put.
42
43-N Put coverage files alongside modules.
44
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000045-s means to run only a single test and exit. This is useful when
46doing memory analysis on the Python interpreter (which tend to consume
47too many resources to run the full regression test non-stop). The
48file /tmp/pynexttest is read to find the next test to run. If this
49file is missing, the first test_*.py file in testdir or on the command
50line is used. (actually tempfile.gettempdir() is used instead of
51/tmp).
Barry Warsawe11e3de1999-01-28 19:51:51 +000052
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000053-f reads the names of tests from the file given as f's argument, one
54or more test names per line. Whitespace is ignored. Blank lines and
55lines beginning with '#' are ignored. This is especially useful for
56whittling down failures involving interactions among tests.
Tim Petersc5000df2002-06-02 21:42:01 +000057
Skip Montanaro0179a182004-06-06 15:53:18 +000058-L causes the leaks(1) command to be run just before exit if it exists.
59leaks(1) is available on Mac OS X and presumably on some other
60FreeBSD-derived systems.
61
Michael W. Hudson61147f62004-08-03 11:33:28 +000062-R runs each test several times and examines sys.gettotalrefcount() to
63see if the test appears to be leaking references. The argument should
64be of the form stab:run:fname where 'stab' is the number of times the
65test is run to let gettotalrefcount settle down, 'run' is the number
66of times further it is run and 'fname' is the name of the file the
67reports are written to. These parameters all have defaults (5, 4 and
68"reflog.txt" respectively), so the minimal invocation is '-R ::'.
69
Thomas Woutersabd08882006-04-26 15:53:30 +000070-M runs tests that require an exorbitant amount of memory. These tests
71typically try to ascertain containers keep working when containing more than
Armin Rigo97ff0472006-08-09 15:37:26 +0000722 billion objects, which only works on 64-bit systems. There are also some
73tests that try to exhaust the address space of the process, which only makes
74sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,
Thomas Woutersabd08882006-04-26 15:53:30 +000075which is a string in the form of '2.5Gb', determines howmuch memory the
76tests will limit themselves to (but they may go slightly over.) The number
77shouldn't be more memory than the machine has (including swap memory). You
78should also keep in mind that swap memory is generally much, much slower
79than RAM, and setting memlimit to all available RAM or higher will heavily
80tax the machine. On the other hand, it is no use running these tests with a
81limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect
82to use more than memlimit memory will be skipped. The big-memory tests
83generally run very, very long.
84
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000085-u is used to specify which special resource intensive tests to run,
86such as those requiring large file support or network connectivity.
87The argument is a comma-separated list of words indicating the
88resources to test. Currently only the following are defined:
Barry Warsaw08fca522001-08-20 22:33:46 +000089
Fred Drake3a15dac2002-04-11 16:39:16 +000090 all - Enable all special resources.
91
Guido van Rossum315aa362003-03-11 14:46:48 +000092 audio - Tests that use the audio device. (There are known
93 cases of broken audio drivers that can crash Python or
94 even the Linux kernel.)
95
Andrew M. Kuchling2158df02001-10-22 15:26:09 +000096 curses - Tests that use curses and will modify the terminal's
97 state and output modes.
Tim Peters1633a2e2001-10-30 05:56:40 +000098
Benjamin Peterson87988b62008-07-05 16:29:38 +000099 lib2to3 - Run the tests for 2to3 (They take a while.)
100
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000101 largefile - It is okay to run some test that may create huge
102 files. These tests can take a long time and may
103 consume >2GB of disk space temporarily.
Barry Warsaw08fca522001-08-20 22:33:46 +0000104
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000105 network - It is okay to run tests that use external network
106 resource, e.g. testing SSL support for sockets.
Martin v. Löwis1c6b1a22002-11-19 17:47:07 +0000107
108 bsddb - It is okay to run the bsddb testsuite, which takes
109 a long time to complete.
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000110
Raymond Hettinger7c85fa42004-07-01 11:01:35 +0000111 decimal - Test the decimal module against a large suite that
112 verifies compliance with standards.
113
Jeremy Hylton4336eda2004-08-07 19:25:33 +0000114 compiler - Test the compiler package by compiling all the source
115 in the standard library and test suite. This takes
Raymond Hettinger68c04532005-06-10 11:05:19 +0000116 a long time. Enabling this resource also allows
117 test_tokenize to verify round-trip lexing on every
118 file in the test library.
Jeremy Hylton4336eda2004-08-07 19:25:33 +0000119
Tim Peterseba28be2005-03-28 01:08:02 +0000120 subprocess Run all tests for the subprocess module.
Peter Astrandf7f1bb72005-03-03 20:47:37 +0000121
Hye-Shik Changaaa2f1d2005-12-10 17:44:27 +0000122 urlfetch - It is okay to download files required on testing.
123
Guilherme Polob1a98de2009-01-28 20:03:26 +0000124 gui - Run tests that require a running GUI.
125
Collin Winterf8089c72009-04-09 16:46:46 +0000126 xpickle - Test pickle and cPickle against Python 2.4, 2.5 and 2.6 to
127 test backwards compatibility. These tests take a long time
128 to run.
129
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000130To enable all resources except one, use '-uall,-<resource>'. For
131example, to run all the tests except for the bsddb tests, give the
132option '-uall,-bsddb'.
Guido van Rossum152494a1996-12-20 03:12:20 +0000133"""
134
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000135import cStringIO
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000136import getopt
Antoine Pitrou4698d992009-05-31 14:20:14 +0000137import json
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000138import os
139import random
140import re
141import sys
142import time
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000143import traceback
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000144import warnings
Benjamin Peterson888a39b2009-03-26 20:48:25 +0000145import unittest
Guido van Rossumdc15c272002-08-12 21:55:51 +0000146
147# I see no other way to suppress these warnings;
148# putting them in test_grammar.py has no effect:
Guido van Rossum88b1def2002-08-14 17:54:48 +0000149warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
Guido van Rossumdc15c272002-08-12 21:55:51 +0000150 ".*test.test_grammar$")
Guido van Rossumc34c4fc2002-09-19 00:42:16 +0000151if sys.maxint > 0x7fffffff:
152 # Also suppress them in <string>, because for 64-bit platforms,
153 # that's where test_grammar.py hides them.
154 warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
155 "<string>")
Guido van Rossum152494a1996-12-20 03:12:20 +0000156
Thomas Wouters9df4e6f2006-04-27 23:13:20 +0000157# Ignore ImportWarnings that only occur in the source tree,
158# (because of modules with the same name as source-directories in Modules/)
Thomas Wouters076ba212006-04-27 23:41:27 +0000159for mod in ("ctypes", "gzip", "zipfile", "tarfile", "encodings.zlib_codec",
160 "test.test_zipimport", "test.test_zlib", "test.test_zipfile",
161 "test.test_codecs", "test.string_tests"):
Thomas Wouters9df4e6f2006-04-27 23:13:20 +0000162 warnings.filterwarnings(module=".*%s$" % (mod,),
163 action="ignore", category=ImportWarning)
164
Guido van Rossumbb484652002-12-02 09:56:21 +0000165# MacOSX (a.k.a. Darwin) has a default stack size that is too small
166# for deeply recursive regular expressions. We see this as crashes in
167# the Python test suite when running test_re.py and test_sre.py. The
168# fix is to set the stack limit to 2048.
169# This approach may also be useful for other Unixy platforms that
170# suffer from small default stack limits.
171if sys.platform == 'darwin':
172 try:
173 import resource
174 except ImportError:
175 pass
176 else:
177 soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
178 newsoft = min(hard, max(soft, 1024*2048))
179 resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
180
Barry Warsaw04f357c2002-07-23 19:04:11 +0000181from test import test_support
Fred Drake3a15dac2002-04-11 16:39:16 +0000182
Benjamin Petersone2886fd2008-07-18 14:26:35 +0000183RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
Collin Winterf8089c72009-04-09 16:46:46 +0000184 'decimal', 'compiler', 'subprocess', 'urlfetch', 'gui',
185 'xpickle')
Fred Drake3a15dac2002-04-11 16:39:16 +0000186
187
Barry Warsaw08fca522001-08-20 22:33:46 +0000188def usage(code, msg=''):
189 print __doc__
190 if msg: print msg
191 sys.exit(code)
192
193
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000194def main(tests=None, testdir=None, verbose=0, quiet=False,
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000195 exclude=False, single=False, randomize=False, fromfile=None,
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000196 findleaks=False, use_resources=None, trace=False, coverdir='coverage',
Collin Wintera199f012009-03-29 03:44:19 +0000197 runleaks=False, huntrleaks=False, verbose2=False, print_slow=False,
Antoine Pitrou4698d992009-05-31 14:20:14 +0000198 random_seed=None, use_mp=None):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000199 """Execute a test suite.
200
Thomas Wouters7e474022000-07-16 12:04:32 +0000201 This also parses command-line options and modifies its behavior
Fred Drake004d5e62000-10-23 17:22:08 +0000202 accordingly.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000203
204 tests -- a list of strings containing test names (optional)
205 testdir -- the directory in which to look for tests (optional)
206
207 Users other than the Python test suite will certainly want to
208 specify testdir; if it's omitted, the directory containing the
Fred Drake004d5e62000-10-23 17:22:08 +0000209 Python test suite is searched for.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000210
211 If the tests argument is omitted, the tests listed on the
212 command-line will be used. If that's empty, too, then all *.py
213 files beginning with test_ will be used.
Skip Montanaroab1c7912000-06-30 16:39:27 +0000214
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000215 The other default arguments (verbose, quiet, exclude,
Collin Wintera199f012009-03-29 03:44:19 +0000216 single, randomize, findleaks, use_resources, trace, coverdir, print_slow and
217 random_seed) allow programmers calling main() directly to set the
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000218 values that would normally be set by flags on the command line.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000219 """
Fred Drake004d5e62000-10-23 17:22:08 +0000220
Tim Peters8dee8092001-09-25 20:05:11 +0000221 test_support.record_original_stdout(sys.stdout)
Guido van Rossum152494a1996-12-20 03:12:20 +0000222 try:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000223 opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsSrf:lu:t:TD:NLR:wM:j:',
Brett Cannona5573b32008-03-18 04:16:06 +0000224 ['help', 'verbose', 'quiet', 'exclude',
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000225 'single', 'slow', 'random', 'fromfile',
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000226 'findleaks', 'use=', 'threshold=', 'trace',
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000227 'coverdir=', 'nocoverdir', 'runleaks',
Thomas Woutersabd08882006-04-26 15:53:30 +0000228 'huntrleaks=', 'verbose2', 'memlimit=',
Antoine Pitrou4698d992009-05-31 14:20:14 +0000229 'randseed=', 'multiprocess=', 'slaveargs=',
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000230 ])
Guido van Rossum152494a1996-12-20 03:12:20 +0000231 except getopt.error, msg:
Barry Warsaw08fca522001-08-20 22:33:46 +0000232 usage(2, msg)
233
234 # Defaults
Collin Wintera199f012009-03-29 03:44:19 +0000235 if random_seed is None:
236 random_seed = random.randrange(10000000)
Barry Warsaw08fca522001-08-20 22:33:46 +0000237 if use_resources is None:
238 use_resources = []
Guido van Rossum152494a1996-12-20 03:12:20 +0000239 for o, a in opts:
Barry Warsaw08fca522001-08-20 22:33:46 +0000240 if o in ('-h', '--help'):
241 usage(0)
242 elif o in ('-v', '--verbose'):
243 verbose += 1
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000244 elif o in ('-w', '--verbose2'):
245 verbose2 = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000246 elif o in ('-q', '--quiet'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000247 quiet = True;
Barry Warsaw08fca522001-08-20 22:33:46 +0000248 verbose = 0
Barry Warsaw08fca522001-08-20 22:33:46 +0000249 elif o in ('-x', '--exclude'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000250 exclude = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000251 elif o in ('-s', '--single'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000252 single = True
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000253 elif o in ('-S', '--slow'):
254 print_slow = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000255 elif o in ('-r', '--randomize'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000256 randomize = True
Collin Wintera199f012009-03-29 03:44:19 +0000257 elif o == '--randseed':
258 random_seed = int(a)
Tim Petersc5000df2002-06-02 21:42:01 +0000259 elif o in ('-f', '--fromfile'):
260 fromfile = a
Barry Warsaw08fca522001-08-20 22:33:46 +0000261 elif o in ('-l', '--findleaks'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000262 findleaks = True
Skip Montanaro0179a182004-06-06 15:53:18 +0000263 elif o in ('-L', '--runleaks'):
264 runleaks = True
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000265 elif o in ('-t', '--threshold'):
266 import gc
267 gc.set_threshold(int(a))
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000268 elif o in ('-T', '--coverage'):
269 trace = True
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000270 elif o in ('-D', '--coverdir'):
271 coverdir = os.path.join(os.getcwd(), a)
272 elif o in ('-N', '--nocoverdir'):
273 coverdir = None
Michael W. Hudson61147f62004-08-03 11:33:28 +0000274 elif o in ('-R', '--huntrleaks'):
275 huntrleaks = a.split(':')
276 if len(huntrleaks) != 3:
277 print a, huntrleaks
278 usage(2, '-R takes three colon-separated arguments')
279 if len(huntrleaks[0]) == 0:
280 huntrleaks[0] = 5
281 else:
282 huntrleaks[0] = int(huntrleaks[0])
283 if len(huntrleaks[1]) == 0:
284 huntrleaks[1] = 4
285 else:
286 huntrleaks[1] = int(huntrleaks[1])
287 if len(huntrleaks[2]) == 0:
288 huntrleaks[2] = "reflog.txt"
Thomas Woutersabd08882006-04-26 15:53:30 +0000289 elif o in ('-M', '--memlimit'):
290 test_support.set_memlimit(a)
Barry Warsaw08fca522001-08-20 22:33:46 +0000291 elif o in ('-u', '--use'):
Guido van Rossumfe3f6962001-09-06 16:09:41 +0000292 u = [x.lower() for x in a.split(',')]
293 for r in u:
Fred Drake3a15dac2002-04-11 16:39:16 +0000294 if r == 'all':
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000295 use_resources[:] = RESOURCE_NAMES
296 continue
297 remove = False
298 if r[0] == '-':
299 remove = True
300 r = r[1:]
Fred Drake3a15dac2002-04-11 16:39:16 +0000301 if r not in RESOURCE_NAMES:
302 usage(1, 'Invalid -u/--use option: ' + a)
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000303 if remove:
304 if r in use_resources:
305 use_resources.remove(r)
306 elif r not in use_resources:
Andrew MacIntyree41abab2002-04-30 12:11:04 +0000307 use_resources.append(r)
Antoine Pitrou4698d992009-05-31 14:20:14 +0000308 elif o in ('-j', '--multiprocess'):
309 use_mp = int(a)
310 elif o == '--slaveargs':
311 args, kwargs = json.loads(a)
312 try:
313 result = runtest(*args, **kwargs)
314 except BaseException, e:
R. David Murray14dcd432009-10-14 13:58:07 +0000315 result = -4, e.__class__.__name__
Antoine Pitrou4698d992009-05-31 14:20:14 +0000316 print # Force a newline (just in case)
317 print json.dumps(result)
318 sys.exit(0)
Tim Petersc5000df2002-06-02 21:42:01 +0000319 if single and fromfile:
320 usage(2, "-s and -f don't go together!")
Antoine Pitrou4698d992009-05-31 14:20:14 +0000321 if use_mp and trace:
322 usage(2, "-T and -j don't go together!")
323 if use_mp and findleaks:
324 usage(2, "-l and -j don't go together!")
Barry Warsaw08fca522001-08-20 22:33:46 +0000325
Guido van Rossum152494a1996-12-20 03:12:20 +0000326 good = []
327 bad = []
328 skipped = []
Fred Drake9a0db072003-02-03 15:19:30 +0000329 resource_denieds = []
R. David Murray14dcd432009-10-14 13:58:07 +0000330 environment_changed = []
Barry Warsawe11e3de1999-01-28 19:51:51 +0000331
Neil Schemenauerd569f232000-09-22 15:29:28 +0000332 if findleaks:
Barry Warsawa873b032000-08-03 15:50:37 +0000333 try:
334 import gc
335 except ImportError:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000336 print 'No GC available, disabling findleaks.'
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000337 findleaks = False
Barry Warsawa873b032000-08-03 15:50:37 +0000338 else:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000339 # Uncomment the line below to report garbage that is not
340 # freeable by reference counting alone. By default only
341 # garbage that is not collectable by the GC is reported.
342 #gc.set_debug(gc.DEBUG_SAVEALL)
Neil Schemenauerd569f232000-09-22 15:29:28 +0000343 found_garbage = []
Barry Warsawa873b032000-08-03 15:50:37 +0000344
Barry Warsawe11e3de1999-01-28 19:51:51 +0000345 if single:
346 from tempfile import gettempdir
347 filename = os.path.join(gettempdir(), 'pynexttest')
348 try:
349 fp = open(filename, 'r')
Eric S. Raymondfc170b12001-02-09 11:51:27 +0000350 next = fp.read().strip()
Barry Warsawe11e3de1999-01-28 19:51:51 +0000351 tests = [next]
352 fp.close()
353 except IOError:
354 pass
Tim Petersc5000df2002-06-02 21:42:01 +0000355
356 if fromfile:
357 tests = []
358 fp = open(fromfile)
359 for line in fp:
360 guts = line.split() # assuming no test has whitespace in its name
361 if guts and not guts[0].startswith('#'):
362 tests.extend(guts)
363 fp.close()
364
365 # Strip .py extensions.
366 if args:
367 args = map(removepy, args)
368 if tests:
369 tests = map(removepy, tests)
370
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000371 stdtests = STDTESTS[:]
372 nottests = NOTTESTS[:]
Guido van Rossum152494a1996-12-20 03:12:20 +0000373 if exclude:
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000374 for arg in args:
375 if arg in stdtests:
376 stdtests.remove(arg)
377 nottests[:0] = args
Guido van Rossum41360a41998-03-26 19:42:58 +0000378 args = []
Guido van Rossum747e1ca1998-08-24 13:48:36 +0000379 tests = tests or args or findtests(testdir, stdtests, nottests)
Barry Warsawe11e3de1999-01-28 19:51:51 +0000380 if single:
381 tests = tests[:1]
Skip Montanaroab1c7912000-06-30 16:39:27 +0000382 if randomize:
Collin Wintera199f012009-03-29 03:44:19 +0000383 random.seed(random_seed)
384 print "Using random seed", random_seed
Skip Montanaroab1c7912000-06-30 16:39:27 +0000385 random.shuffle(tests)
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000386 if trace:
387 import trace
388 tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],
389 trace=False, count=True)
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000390 test_times = []
Barry Warsaw08fca522001-08-20 22:33:46 +0000391 test_support.use_resources = use_resources
Guido van Rossum5796d262000-04-21 21:35:06 +0000392 save_modules = sys.modules.keys()
Antoine Pitrou4698d992009-05-31 14:20:14 +0000393
394 def accumulate_result(test, result):
395 ok, test_time = result
396 test_times.append((test_time, test))
397 if ok > 0:
398 good.append(test)
R. David Murray14dcd432009-10-14 13:58:07 +0000399 elif -2 < ok <= 0:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000400 bad.append(test)
R. David Murray14dcd432009-10-14 13:58:07 +0000401 if ok == -1:
402 environment_changed.append(test)
Guido van Rossum41360a41998-03-26 19:42:58 +0000403 else:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000404 skipped.append(test)
R. David Murray14dcd432009-10-14 13:58:07 +0000405 if ok == -3:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000406 resource_denieds.append(test)
407
408 if use_mp:
409 from threading import Thread
410 from Queue import Queue, Empty
411 from subprocess import Popen, PIPE, STDOUT
412 from collections import deque
413 debug_output_pat = re.compile(r"\[\d+ refs\]$")
414 pending = deque()
415 output = Queue()
416 for test in tests:
417 args_tuple = (
418 (test, verbose, quiet, testdir),
419 dict(huntrleaks=huntrleaks, use_resources=use_resources)
420 )
421 pending.append((test, args_tuple))
422 def work():
423 # A worker thread.
Neal Norwitz14ca3272006-02-28 18:05:43 +0000424 try:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000425 while True:
426 try:
427 test, args_tuple = pending.popleft()
428 except IndexError:
R. David Murray282396f2009-10-18 21:12:37 +0000429 output.put((None, None, None, None))
Antoine Pitrou4698d992009-05-31 14:20:14 +0000430 return
Antoine Pitrou79c3bd82009-06-29 14:14:56 +0000431 # -E is needed by some tests, e.g. test_import
432 popen = Popen([sys.executable, '-E', '-m', 'test.regrtest',
Antoine Pitrou4698d992009-05-31 14:20:14 +0000433 '--slaveargs', json.dumps(args_tuple)],
R. David Murray282396f2009-10-18 21:12:37 +0000434 stdout=PIPE, stderr=PIPE,
Antoine Pitrou4698d992009-05-31 14:20:14 +0000435 universal_newlines=True, close_fds=True)
R. David Murray282396f2009-10-18 21:12:37 +0000436 stdout, stderr = popen.communicate()
437 # Strip last refcount output line if it exists, since it
438 # comes from the shutdown of the interpreter in the subcommand.
439 stderr = debug_output_pat.sub("", stderr)
440 stdout, _, result = stdout.strip().rpartition("\n")
Antoine Pitrou4698d992009-05-31 14:20:14 +0000441 result = json.loads(result)
R. David Murray282396f2009-10-18 21:12:37 +0000442 if not quiet:
443 stdout = test+'\n'+stdout
444 output.put((test, stdout.rstrip(), stderr.rstrip(), result))
Antoine Pitrou4698d992009-05-31 14:20:14 +0000445 except BaseException:
R. David Murray282396f2009-10-18 21:12:37 +0000446 output.put((None, None, None, None))
Neal Norwitz14ca3272006-02-28 18:05:43 +0000447 raise
Antoine Pitrou4698d992009-05-31 14:20:14 +0000448 workers = [Thread(target=work) for i in range(use_mp)]
449 for worker in workers:
450 worker.start()
451 finished = 0
452 while finished < use_mp:
R. David Murray282396f2009-10-18 21:12:37 +0000453 test, stdout, stderr, result = output.get()
Antoine Pitrou4698d992009-05-31 14:20:14 +0000454 if test is None:
455 finished += 1
456 continue
R. David Murray282396f2009-10-18 21:12:37 +0000457 if stdout:
458 print stdout
459 if stderr:
460 print >>sys.stderr, stderr
R. David Murray14dcd432009-10-14 13:58:07 +0000461 if result[0] == -4:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000462 assert result[1] == 'KeyboardInterrupt'
463 pending.clear()
464 raise KeyboardInterrupt # What else?
465 accumulate_result(test, result)
466 for worker in workers:
467 worker.join()
468 else:
469 for test in tests:
470 if not quiet:
471 print test
472 sys.stdout.flush()
473 if trace:
474 # If we're tracing code coverage, then we don't exit with status
475 # if on a false return value from main.
476 tracer.runctx('runtest(test, verbose, quiet, testdir)',
477 globals=globals(), locals=vars())
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000478 else:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000479 try:
480 result = runtest(test, verbose, quiet,
481 testdir, huntrleaks)
482 accumulate_result(test, result)
483 except KeyboardInterrupt:
484 # print a newline separate from the ^C
485 print
486 break
487 except:
488 raise
489 if findleaks:
490 gc.collect()
491 if gc.garbage:
492 print "Warning: test created", len(gc.garbage),
493 print "uncollectable object(s)."
494 # move the uncollectable objects somewhere so we don't see
495 # them again
496 found_garbage.extend(gc.garbage)
497 del gc.garbage[:]
498 # Unload the newly imported modules (best effort finalization)
499 for module in sys.modules.keys():
500 if module not in save_modules and module.startswith("test."):
501 test_support.unload(module)
Jeremy Hylton7a1ea0e2001-10-17 13:45:28 +0000502
503 # The lists won't be sorted if running with -r
504 good.sort()
505 bad.sort()
506 skipped.sort()
R. David Murray14dcd432009-10-14 13:58:07 +0000507 environment_changed.sort()
Tim Peterse0c446b2001-10-18 21:57:37 +0000508
Guido van Rossum152494a1996-12-20 03:12:20 +0000509 if good and not quiet:
Guido van Rossum41360a41998-03-26 19:42:58 +0000510 if not bad and not skipped and len(good) > 1:
511 print "All",
512 print count(len(good), "test"), "OK."
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000513 if print_slow:
514 test_times.sort(reverse=True)
515 print "10 slowest tests:"
516 for time, test in test_times[:10]:
517 print "%s: %.1fs" % (test, time)
Guido van Rossum152494a1996-12-20 03:12:20 +0000518 if bad:
R. David Murray14dcd432009-10-14 13:58:07 +0000519 bad = set(bad) - set(environment_changed)
520 if bad:
521 print count(len(bad), "test"), "failed:"
522 printlist(bad)
523 if environment_changed:
524 print "{} altered the execution environment:".format(
525 count(len(environment_changed), "test"))
526 printlist(environment_changed)
Guido van Rossum152494a1996-12-20 03:12:20 +0000527 if skipped and not quiet:
Tim Petersa45da922001-08-12 03:45:50 +0000528 print count(len(skipped), "test"), "skipped:"
529 printlist(skipped)
Barry Warsawe11e3de1999-01-28 19:51:51 +0000530
Tim Petersb5b7b782001-08-12 01:20:39 +0000531 e = _ExpectedSkips()
Tim Petersa2be2d62001-08-12 02:01:09 +0000532 plat = sys.platform
Tim Petersb5b7b782001-08-12 01:20:39 +0000533 if e.isvalid():
Raymond Hettingera690a992003-11-16 16:17:49 +0000534 surprise = set(skipped) - e.getexpected() - set(resource_denieds)
Tim Petersb5b7b782001-08-12 01:20:39 +0000535 if surprise:
536 print count(len(surprise), "skip"), \
Tim Petersa45da922001-08-12 03:45:50 +0000537 "unexpected on", plat + ":"
538 printlist(surprise)
Tim Petersb5b7b782001-08-12 01:20:39 +0000539 else:
540 print "Those skips are all expected on", plat + "."
541 else:
542 print "Ask someone to teach regrtest.py about which tests are"
543 print "expected to get skipped on", plat + "."
544
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000545 if verbose2 and bad:
546 print "Re-running failed tests in verbose mode"
547 for test in bad:
Tim Peters922dd7d2006-03-10 23:37:10 +0000548 print "Re-running test %r in verbose mode" % test
549 sys.stdout.flush()
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000550 try:
Neal Norwitz5fe5cf62008-03-18 17:58:02 +0000551 test_support.verbose = True
Antoine Pitrou4698d992009-05-31 14:20:14 +0000552 ok = runtest(test, True, quiet, testdir,
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000553 huntrleaks)
554 except KeyboardInterrupt:
555 # print a newline separate from the ^C
556 print
557 break
558 except:
559 raise
560
Barry Warsawe11e3de1999-01-28 19:51:51 +0000561 if single:
562 alltests = findtests(testdir, stdtests, nottests)
563 for i in range(len(alltests)):
564 if tests[0] == alltests[i]:
565 if i == len(alltests) - 1:
566 os.unlink(filename)
567 else:
568 fp = open(filename, 'w')
569 fp.write(alltests[i+1] + '\n')
570 fp.close()
571 break
572 else:
573 os.unlink(filename)
574
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000575 if trace:
576 r = tracer.results()
577 r.write_results(show_missing=True, summary=True, coverdir=coverdir)
578
Skip Montanaro0179a182004-06-06 15:53:18 +0000579 if runleaks:
580 os.system("leaks %d" % os.getpid())
581
Tim Peters5943b4a2003-07-23 00:30:39 +0000582 sys.exit(len(bad) > 0)
Barry Warsaw08fca522001-08-20 22:33:46 +0000583
Guido van Rossum152494a1996-12-20 03:12:20 +0000584
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000585STDTESTS = [
Guido van Rossum152494a1996-12-20 03:12:20 +0000586 'test_grammar',
587 'test_opcodes',
Collin Winterf567ca32007-03-12 15:57:19 +0000588 'test_dict',
Guido van Rossum152494a1996-12-20 03:12:20 +0000589 'test_builtin',
590 'test_exceptions',
591 'test_types',
Collin Winter55453142007-03-08 19:58:14 +0000592 'test_unittest',
593 'test_doctest',
594 'test_doctest2',
Guido van Rossum152494a1996-12-20 03:12:20 +0000595 ]
596
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000597NOTTESTS = [
Guido van Rossum152494a1996-12-20 03:12:20 +0000598 'test_support',
Jeremy Hylton62e2c7e2001-02-28 17:48:06 +0000599 'test_future1',
600 'test_future2',
Guido van Rossum152494a1996-12-20 03:12:20 +0000601 ]
602
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000603def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):
Guido van Rossum152494a1996-12-20 03:12:20 +0000604 """Return a list of all applicable test modules."""
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000605 if not testdir: testdir = findtestdir()
Guido van Rossum152494a1996-12-20 03:12:20 +0000606 names = os.listdir(testdir)
607 tests = []
608 for name in names:
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000609 if name[:5] == "test_" and name[-3:] == os.extsep+"py":
Guido van Rossum41360a41998-03-26 19:42:58 +0000610 modname = name[:-3]
611 if modname not in stdtests and modname not in nottests:
612 tests.append(modname)
Guido van Rossum152494a1996-12-20 03:12:20 +0000613 tests.sort()
614 return stdtests + tests
615
Antoine Pitrou4698d992009-05-31 14:20:14 +0000616def runtest(test, verbose, quiet,
617 testdir=None, huntrleaks=False, use_resources=None):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000618 """Run a single test.
Tim Peterse5584862006-06-09 19:24:44 +0000619
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000620 test -- the name of the test
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000621 verbose -- if true, print more messages
Trent Mickf29f47b2000-08-11 19:02:59 +0000622 quiet -- if true, don't print 'skipped' messages (probably redundant)
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000623 test_times -- a list of (time, test_name) pairs
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000624 testdir -- test directory
Tim Peterse5584862006-06-09 19:24:44 +0000625 huntrleaks -- run multiple times to test for leaks; requires a debug
626 build; a triple corresponding to -R's three arguments
627 Return:
R. David Murray14dcd432009-10-14 13:58:07 +0000628 -4 KeyboardInterrupt when run under -j
629 -3 test skipped because resource denied
630 -2 test skipped for some other reason
631 -1 test failed because it changed the execution environment
Tim Peterse5584862006-06-09 19:24:44 +0000632 0 test failed
633 1 test passed
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000634 """
Tim Peters06c5c002006-06-05 00:55:26 +0000635
Antoine Pitrou4698d992009-05-31 14:20:14 +0000636 test_support.verbose = verbose # Tell tests to be moderately quiet
637 if use_resources is not None:
638 test_support.use_resources = use_resources
Tim Peterse5584862006-06-09 19:24:44 +0000639 try:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000640 return runtest_inner(test, verbose, quiet,
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000641 testdir, huntrleaks)
Tim Peterse5584862006-06-09 19:24:44 +0000642 finally:
643 cleanup_test_droppings(test, verbose)
644
R. David Murray14dcd432009-10-14 13:58:07 +0000645
646# Unit tests are supposed to leave the execution environment unchanged
647# once they complete. But sometimes tests have bugs, especially when
648# tests fail, and the changes to environment go on to mess up other
649# tests. This can cause issues with buildbot stability, since tests
650# are run in random order and so problems may appear to come and go.
651# There are a few things we can save and restore to mitigate this, and
652# the following context manager handles this task.
653
654class saved_test_environment:
655 """Save bits of the test environment and restore them at block exit.
656
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000657 with saved_test_environment(testname, verbose, quiet):
R. David Murray14dcd432009-10-14 13:58:07 +0000658 #stuff
659
660 Unless quiet is True, a warning is printed to stderr if any of
661 the saved items was changed by the test. The attribute 'changed'
662 is initially False, but is set to True if a change is detected.
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000663
664 If verbose is more than 1, the before and after state of changed
665 items is also printed.
R. David Murray14dcd432009-10-14 13:58:07 +0000666 """
667
668 changed = False
669
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000670 def __init__(self, testname, verbose=0, quiet=False):
R. David Murray14dcd432009-10-14 13:58:07 +0000671 self.testname = testname
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000672 self.verbose = verbose
R. David Murray14dcd432009-10-14 13:58:07 +0000673 self.quiet = quiet
674
675 # To add things to save and restore, add a name XXX to the resources list
676 # and add corresponding get_XXX/restore_XXX functions. get_XXX should
677 # return the value to be saved and compared against a second call to the
678 # get function when test execution completes. restore_XXX should accept
679 # the saved value and restore the resource using it. It will be called if
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000680 # and only if a change in the value is detected.
681 #
682 # Note: XXX will have any '.' replaced with '_' characters when determining
683 # the corresponding method names.
R. David Murray14dcd432009-10-14 13:58:07 +0000684
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000685 resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr',
686 'os.environ', 'sys.path')
R. David Murray14dcd432009-10-14 13:58:07 +0000687
688 def get_sys_argv(self):
Nick Coghlan8157e192009-10-17 15:48:16 +0000689 return id(sys.argv), sys.argv, sys.argv[:]
R. David Murray14dcd432009-10-14 13:58:07 +0000690 def restore_sys_argv(self, saved_argv):
Nick Coghlan8157e192009-10-17 15:48:16 +0000691 sys.argv = saved_argv[1]
692 sys.argv[:] = saved_argv[2]
R. David Murray14dcd432009-10-14 13:58:07 +0000693
694 def get_cwd(self):
695 return os.getcwd()
696 def restore_cwd(self, saved_cwd):
697 os.chdir(saved_cwd)
698
699 def get_sys_stdout(self):
700 return sys.stdout
701 def restore_sys_stdout(self, saved_stdout):
702 sys.stdout = saved_stdout
703
704 def get_sys_stderr(self):
705 return sys.stderr
706 def restore_sys_stderr(self, saved_stderr):
707 sys.stderr = saved_stderr
708
709 def get_sys_stdin(self):
710 return sys.stdin
711 def restore_sys_stdin(self, saved_stdin):
712 sys.stdin = saved_stdin
713
714 def get_os_environ(self):
Nick Coghlan8157e192009-10-17 15:48:16 +0000715 return id(os.environ), os.environ, dict(os.environ)
R. David Murray14dcd432009-10-14 13:58:07 +0000716 def restore_os_environ(self, saved_environ):
Nick Coghlan8157e192009-10-17 15:48:16 +0000717 os.environ = saved_environ[1]
R. David Murray14dcd432009-10-14 13:58:07 +0000718 os.environ.clear()
Nick Coghlan8157e192009-10-17 15:48:16 +0000719 os.environ.update(saved_environ[2])
R. David Murray14dcd432009-10-14 13:58:07 +0000720
721 def get_sys_path(self):
Nick Coghlan8157e192009-10-17 15:48:16 +0000722 return id(sys.path), sys.path, sys.path[:]
R. David Murray14dcd432009-10-14 13:58:07 +0000723 def restore_sys_path(self, saved_path):
Nick Coghlan8157e192009-10-17 15:48:16 +0000724 sys.path = saved_path[1]
725 sys.path[:] = saved_path[2]
R. David Murray14dcd432009-10-14 13:58:07 +0000726
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000727 def resource_info(self):
728 for name in self.resources:
729 method_suffix = name.replace('.', '_')
730 get_name = 'get_' + method_suffix
731 restore_name = 'restore_' + method_suffix
732 yield name, getattr(self, get_name), getattr(self, restore_name)
733
R. David Murray14dcd432009-10-14 13:58:07 +0000734 def __enter__(self):
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000735 self.saved_values = dict((name, get()) for name, get, restore
736 in self.resource_info())
R. David Murray14dcd432009-10-14 13:58:07 +0000737 return self
738
739 def __exit__(self, exc_type, exc_val, exc_tb):
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000740 for name, get, restore in self.resource_info():
741 current = get()
742 original = self.saved_values[name]
743 # Check for changes to the resource's value
744 if current != original:
R. David Murray14dcd432009-10-14 13:58:07 +0000745 self.changed = True
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000746 restore(original)
R. David Murray14dcd432009-10-14 13:58:07 +0000747 if not self.quiet:
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000748 print >>sys.stderr, (
749 "Warning -- {} was modified by {}".format(
750 name, self.testname))
751 if self.verbose > 1:
752 print >>sys.stderr, (
Nick Coghlana3e97ad2009-10-17 07:34:27 +0000753 " Before: {}\n After: {} ".format(
754 original, current))
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000755 # XXX (ncoghlan): for most resources (e.g. sys.path) identity
756 # matters at least as much as value. For others (e.g. cwd),
757 # identity is irrelevant. Should we add a mechanism to check
758 # for substitution in the cases where it matters?
R. David Murray14dcd432009-10-14 13:58:07 +0000759 return False
760
761
Antoine Pitrou4698d992009-05-31 14:20:14 +0000762def runtest_inner(test, verbose, quiet,
Jeffrey Yasskin36dbcb92008-03-18 05:45:40 +0000763 testdir=None, huntrleaks=False):
Guido van Rossum152494a1996-12-20 03:12:20 +0000764 test_support.unload(test)
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000765 if not testdir:
766 testdir = findtestdir()
Tim Peters9390cc12001-09-28 20:14:46 +0000767 if verbose:
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000768 capture_stdout = None
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000769 else:
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000770 capture_stdout = cStringIO.StringIO()
Tim Peters06c5c002006-06-05 00:55:26 +0000771
Antoine Pitrou4698d992009-05-31 14:20:14 +0000772 test_time = 0.0
Collin Winter0f489742009-05-14 23:26:30 +0000773 refleak = False # True if the test leaked references.
Guido van Rossum152494a1996-12-20 03:12:20 +0000774 try:
Tim Peters342ca752001-09-25 19:13:20 +0000775 save_stdout = sys.stdout
Guido van Rossum41360a41998-03-26 19:42:58 +0000776 try:
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000777 if capture_stdout:
778 sys.stdout = capture_stdout
Barry Warsaw408b6d32002-07-30 23:27:12 +0000779 if test.startswith('test.'):
780 abstest = test
781 else:
782 # Always import it from the test package
783 abstest = 'test.' + test
Nick Coghlan2ee358e2009-10-17 07:30:40 +0000784 with saved_test_environment(test, verbose, quiet) as environment:
R. David Murray14dcd432009-10-14 13:58:07 +0000785 start_time = time.time()
786 the_package = __import__(abstest, globals(), locals(), [])
787 the_module = getattr(the_package, test)
788 # Old tests run to completion simply as a side-effect of
789 # being imported. For tests based on unittest or doctest,
790 # explicitly invoke their test_main() function (if it exists).
791 indirect_test = getattr(the_module, "test_main", None)
792 if indirect_test is not None:
793 indirect_test()
794 if huntrleaks:
795 refleak = dash_R(the_module, test, indirect_test,
796 huntrleaks)
797 test_time = time.time() - start_time
Guido van Rossum41360a41998-03-26 19:42:58 +0000798 finally:
Tim Peters342ca752001-09-25 19:13:20 +0000799 sys.stdout = save_stdout
Fred Drake9a0db072003-02-03 15:19:30 +0000800 except test_support.ResourceDenied, msg:
801 if not quiet:
802 print test, "skipped --", msg
803 sys.stdout.flush()
R. David Murray14dcd432009-10-14 13:58:07 +0000804 return -3, test_time
R. David Murray612f1d52009-03-31 19:49:15 +0000805 except unittest.SkipTest, msg:
Trent Mickf29f47b2000-08-11 19:02:59 +0000806 if not quiet:
Fred Drakede4742b2002-10-17 20:36:08 +0000807 print test, "skipped --", msg
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000808 sys.stdout.flush()
R. David Murray14dcd432009-10-14 13:58:07 +0000809 return -2, test_time
Fred Drakefe5c22a2000-08-18 16:04:05 +0000810 except KeyboardInterrupt:
811 raise
Guido van Rossum152494a1996-12-20 03:12:20 +0000812 except test_support.TestFailed, msg:
Guido van Rossum41360a41998-03-26 19:42:58 +0000813 print "test", test, "failed --", msg
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000814 sys.stdout.flush()
Antoine Pitrou4698d992009-05-31 14:20:14 +0000815 return 0, test_time
Guido van Rossum9e48b271997-07-16 01:56:13 +0000816 except:
Guido van Rossum41360a41998-03-26 19:42:58 +0000817 type, value = sys.exc_info()[:2]
Fred Drake27c4b392000-08-23 20:34:40 +0000818 print "test", test, "crashed --", str(type) + ":", value
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000819 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000820 if verbose:
821 traceback.print_exc(file=sys.stdout)
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000822 sys.stdout.flush()
Antoine Pitrou4698d992009-05-31 14:20:14 +0000823 return 0, test_time
Guido van Rossum152494a1996-12-20 03:12:20 +0000824 else:
Collin Winter0f489742009-05-14 23:26:30 +0000825 if refleak:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000826 return 0, test_time
R. David Murray14dcd432009-10-14 13:58:07 +0000827 if environment.changed:
828 return -1, test_time
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000829 # Except in verbose mode, tests should not print anything
830 if verbose or huntrleaks:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000831 return 1, test_time
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000832 output = capture_stdout.getvalue()
833 if not output:
Antoine Pitrou4698d992009-05-31 14:20:14 +0000834 return 1, test_time
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000835 print "test", test, "produced unexpected output:"
Amaury Forgeot d'Arcea13dc62008-03-29 13:14:52 +0000836 print "*" * 70
837 print output
838 print "*" * 70
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000839 sys.stdout.flush()
Antoine Pitrou4698d992009-05-31 14:20:14 +0000840 return 0, test_time
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000841
Tim Peterse5584862006-06-09 19:24:44 +0000842def cleanup_test_droppings(testname, verbose):
843 import shutil
Benjamin Petersona9b4d472009-04-29 22:44:15 +0000844 import stat
Tim Peterse5584862006-06-09 19:24:44 +0000845
846 # Try to clean up junk commonly left behind. While tests shouldn't leave
847 # any files or directories behind, when a test fails that can be tedious
848 # for it to arrange. The consequences can be especially nasty on Windows,
849 # since if a test leaves a file open, it cannot be deleted by name (while
850 # there's nothing we can do about that here either, we can display the
851 # name of the offending test, which is a real help).
852 for name in (test_support.TESTFN,
853 "db_home",
854 ):
855 if not os.path.exists(name):
856 continue
857
858 if os.path.isdir(name):
859 kind, nuker = "directory", shutil.rmtree
860 elif os.path.isfile(name):
861 kind, nuker = "file", os.unlink
862 else:
863 raise SystemError("os.path says %r exists but is neither "
864 "directory nor file" % name)
865
866 if verbose:
867 print "%r left behind %s %r" % (testname, kind, name)
868 try:
Benjamin Petersona9b4d472009-04-29 22:44:15 +0000869 # if we have chmod, fix possible permissions problems
870 # that might prevent cleanup
871 if (hasattr(os, 'chmod')):
872 os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO)
Tim Peterse5584862006-06-09 19:24:44 +0000873 nuker(name)
874 except Exception, msg:
875 print >> sys.stderr, ("%r left behind %s %r and it couldn't be "
876 "removed: %s" % (testname, kind, name, msg))
877
Tim Peters06c5c002006-06-05 00:55:26 +0000878def dash_R(the_module, test, indirect_test, huntrleaks):
Collin Winter0f489742009-05-14 23:26:30 +0000879 """Run a test multiple times, looking for reference leaks.
880
881 Returns:
882 False if the test didn't leak references; True if we detected refleaks.
883 """
Tim Peters06c5c002006-06-05 00:55:26 +0000884 # This code is hackish and inelegant, but it seems to do the job.
Amaury Forgeot d'Arce55df1f2009-07-28 20:47:55 +0000885 import copy_reg, _abcoll, _pyio
Tim Peters06c5c002006-06-05 00:55:26 +0000886
887 if not hasattr(sys, 'gettotalrefcount'):
888 raise Exception("Tracking reference leaks requires a debug build "
889 "of Python")
890
891 # Save current values for dash_R_cleanup() to restore.
892 fs = warnings.filters[:]
Georg Brandldffbf5f2008-05-20 07:49:57 +0000893 ps = copy_reg.dispatch_table.copy()
Tim Peters06c5c002006-06-05 00:55:26 +0000894 pic = sys.path_importer_cache.copy()
Guido van Rossum64c06e32007-11-22 00:55:51 +0000895 abcs = {}
Amaury Forgeot d'Arce55df1f2009-07-28 20:47:55 +0000896 modules = _abcoll, _pyio
Amaury Forgeot d'Arc4d0c1172008-04-02 00:25:14 +0000897 for abc in [getattr(mod, a) for mod in modules for a in mod.__all__]:
898 # XXX isinstance(abc, ABCMeta) leads to infinite recursion
899 if not hasattr(abc, '_abc_registry'):
Christian Heimes6b29dd02007-11-30 22:36:10 +0000900 continue
Guido van Rossum64c06e32007-11-22 00:55:51 +0000901 for obj in abc.__subclasses__() + [abc]:
902 abcs[obj] = obj._abc_registry.copy()
903
Tim Peters06c5c002006-06-05 00:55:26 +0000904 if indirect_test:
905 def run_the_test():
906 indirect_test()
907 else:
908 def run_the_test():
909 reload(the_module)
910
911 deltas = []
912 nwarmup, ntracked, fname = huntrleaks
913 repcount = nwarmup + ntracked
914 print >> sys.stderr, "beginning", repcount, "repetitions"
915 print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount]
Guido van Rossum64c06e32007-11-22 00:55:51 +0000916 dash_R_cleanup(fs, ps, pic, abcs)
Tim Peters06c5c002006-06-05 00:55:26 +0000917 for i in range(repcount):
918 rc = sys.gettotalrefcount()
919 run_the_test()
920 sys.stderr.write('.')
Guido van Rossum64c06e32007-11-22 00:55:51 +0000921 dash_R_cleanup(fs, ps, pic, abcs)
Tim Peters06c5c002006-06-05 00:55:26 +0000922 if i >= nwarmup:
923 deltas.append(sys.gettotalrefcount() - rc - 2)
924 print >> sys.stderr
925 if any(deltas):
Neal Norwitzd9841032007-05-11 05:55:15 +0000926 msg = '%s leaked %s references, sum=%s' % (test, deltas, sum(deltas))
927 print >> sys.stderr, msg
Antoine Pitrou4698d992009-05-31 14:20:14 +0000928 with open(fname, "a") as refrep:
929 print >> refrep, msg
930 refrep.flush()
Collin Winter0f489742009-05-14 23:26:30 +0000931 return True
932 return False
Tim Peters06c5c002006-06-05 00:55:26 +0000933
Guido van Rossum64c06e32007-11-22 00:55:51 +0000934def dash_R_cleanup(fs, ps, pic, abcs):
Georg Brandldffbf5f2008-05-20 07:49:57 +0000935 import gc, copy_reg
Brett Cannon0aa6e1b2008-05-10 21:12:57 +0000936 import _strptime, linecache
937 dircache = test_support.import_module('dircache', deprecated=True)
Tim Peters06c5c002006-06-05 00:55:26 +0000938 import urlparse, urllib, urllib2, mimetypes, doctest
Amaury Forgeot d'Arc4d0c1172008-04-02 00:25:14 +0000939 import struct, filecmp
Tim Peters06c5c002006-06-05 00:55:26 +0000940 from distutils.dir_util import _path_created
941
Amaury Forgeot d'Arc607bff12008-04-18 23:31:33 +0000942 # Clear the warnings registry, so they can be displayed again
943 for mod in sys.modules.values():
944 if hasattr(mod, '__warningregistry__'):
945 del mod.__warningregistry__
946
Tim Peters06c5c002006-06-05 00:55:26 +0000947 # Restore some original values.
948 warnings.filters[:] = fs
Georg Brandldffbf5f2008-05-20 07:49:57 +0000949 copy_reg.dispatch_table.clear()
950 copy_reg.dispatch_table.update(ps)
Tim Peters06c5c002006-06-05 00:55:26 +0000951 sys.path_importer_cache.clear()
952 sys.path_importer_cache.update(pic)
953
Christian Heimes908caac2008-01-27 23:34:59 +0000954 # clear type cache
Christian Heimes422051a2008-02-04 18:00:12 +0000955 sys._clear_type_cache()
Christian Heimes908caac2008-01-27 23:34:59 +0000956
Guido van Rossum64c06e32007-11-22 00:55:51 +0000957 # Clear ABC registries, restoring previously saved ABC registries.
Amaury Forgeot d'Arc4d0c1172008-04-02 00:25:14 +0000958 for abc, registry in abcs.items():
959 abc._abc_registry = registry.copy()
960 abc._abc_cache.clear()
961 abc._abc_negative_cache.clear()
Guido van Rossum64c06e32007-11-22 00:55:51 +0000962
Tim Peters06c5c002006-06-05 00:55:26 +0000963 # Clear assorted module caches.
964 _path_created.clear()
965 re.purge()
966 _strptime._regex_cache.clear()
967 urlparse.clear_cache()
968 urllib.urlcleanup()
969 urllib2.install_opener(None)
970 dircache.reset()
971 linecache.clearcache()
972 mimetypes._default_mime_types()
Tim Petersc27d6552006-06-07 01:04:59 +0000973 filecmp._cache.clear()
Christian Heimes76d19f62008-01-04 02:54:42 +0000974 struct._clearcache()
Tim Peters06c5c002006-06-05 00:55:26 +0000975 doctest.master = None
976
977 # Collect cyclic trash.
978 gc.collect()
979
Guido van Rossum152494a1996-12-20 03:12:20 +0000980def findtestdir():
981 if __name__ == '__main__':
Guido van Rossum41360a41998-03-26 19:42:58 +0000982 file = sys.argv[0]
Guido van Rossum152494a1996-12-20 03:12:20 +0000983 else:
Guido van Rossum41360a41998-03-26 19:42:58 +0000984 file = __file__
Guido van Rossum152494a1996-12-20 03:12:20 +0000985 testdir = os.path.dirname(file) or os.curdir
986 return testdir
987
Tim Petersc5000df2002-06-02 21:42:01 +0000988def removepy(name):
989 if name.endswith(os.extsep + "py"):
990 name = name[:-3]
991 return name
992
Guido van Rossum152494a1996-12-20 03:12:20 +0000993def count(n, word):
994 if n == 1:
Guido van Rossum41360a41998-03-26 19:42:58 +0000995 return "%d %s" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +0000996 else:
Guido van Rossum41360a41998-03-26 19:42:58 +0000997 return "%d %ss" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +0000998
Tim Petersa45da922001-08-12 03:45:50 +0000999def printlist(x, width=70, indent=4):
Tim Peters7c7efe92002-08-23 17:55:54 +00001000 """Print the elements of iterable x to stdout.
Tim Petersa45da922001-08-12 03:45:50 +00001001
1002 Optional arg width (default 70) is the maximum line length.
1003 Optional arg indent (default 4) is the number of blanks with which to
1004 begin each line.
1005 """
1006
Tim Petersba78bc42002-07-04 19:45:06 +00001007 from textwrap import fill
1008 blanks = ' ' * indent
1009 print fill(' '.join(map(str, x)), width,
1010 initial_indent=blanks, subsequent_indent=blanks)
Tim Petersa45da922001-08-12 03:45:50 +00001011
Tim Petersde14a302002-04-01 05:04:46 +00001012# Map sys.platform to a string containing the basenames of tests
1013# expected to be skipped on that platform.
Tim Peters2a182db2002-10-09 01:07:11 +00001014#
1015# Special cases:
1016# test_pep277
1017# The _ExpectedSkips constructor adds this to the set of expected
1018# skips if not os.path.supports_unicode_filenames.
Neal Norwitz55b61d22003-02-28 19:57:03 +00001019# test_timeout
1020# Controlled by test_timeout.skip_expected. Requires the network
1021# resource and a socket module.
Georg Brandlb2e208f2007-07-12 09:24:04 +00001022#
1023# Tests that are expected to be skipped everywhere except on one platform
1024# are also handled separately.
Tim Petersde14a302002-04-01 05:04:46 +00001025
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001026_expectations = {
1027 'win32':
1028 """
Tim Petersc7c516a2003-09-20 22:06:13 +00001029 test__locale
Skip Montanaro823ba282003-05-06 20:36:24 +00001030 test_bsddb185
Tim Peters78e35f92002-11-22 20:00:34 +00001031 test_bsddb3
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001032 test_commands
1033 test_crypt
Tim Petersd7030572001-10-22 22:06:08 +00001034 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001035 test_dbm
1036 test_dl
1037 test_fcntl
1038 test_fork1
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001039 test_epoll
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001040 test_gdbm
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001041 test_grp
Tim Petersfd8e6e52003-03-04 00:26:38 +00001042 test_ioctl
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001043 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001044 test_kqueue
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001045 test_mhlib
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001046 test_openpty
Tim Petersefc4b122002-12-10 18:47:56 +00001047 test_ossaudiodev
Georg Brandl27d9ee32007-08-30 10:38:56 +00001048 test_pipes
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001049 test_poll
Tim Peters003eb302003-02-17 21:48:48 +00001050 test_posix
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001051 test_pty
1052 test_pwd
Tim Peters1e33ffa2002-04-23 23:09:02 +00001053 test_resource
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001054 test_signal
Tim Peterscea2cc42004-08-04 02:32:03 +00001055 test_threadsignals
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001056 test_timing
Tim Petersb84de022006-03-22 02:58:17 +00001057 test_wait3
1058 test_wait4
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001059 """,
1060 'linux2':
1061 """
Skip Montanaro823ba282003-05-06 20:36:24 +00001062 test_bsddb185
Guido van Rossumf66dacd2001-10-23 15:10:55 +00001063 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001064 test_dl
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001065 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001066 test_kqueue
Guido van Rossum4507ec72003-02-14 19:29:22 +00001067 test_ossaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001068 """,
Jack Jansen49a806e2001-08-28 14:49:00 +00001069 'mac':
Guido van Rossumaa782362001-09-02 03:58:41 +00001070 """
Jack Jansen67975142003-01-08 16:31:11 +00001071 test_atexit
Guido van Rossumaa782362001-09-02 03:58:41 +00001072 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001073 test_bsddb185
Jack Jansen67975142003-01-08 16:31:11 +00001074 test_bsddb3
1075 test_bz2
Guido van Rossumaa782362001-09-02 03:58:41 +00001076 test_commands
1077 test_crypt
Jack Jansenb3be2162001-11-30 14:16:36 +00001078 test_curses
Guido van Rossumaa782362001-09-02 03:58:41 +00001079 test_dbm
1080 test_dl
1081 test_fcntl
1082 test_fork1
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001083 test_epoll
Guido van Rossumaa782362001-09-02 03:58:41 +00001084 test_grp
Jack Jansenc4d6bdd2003-03-07 15:38:11 +00001085 test_ioctl
Guido van Rossumaa782362001-09-02 03:58:41 +00001086 test_largefile
Guido van Rossumaa782362001-09-02 03:58:41 +00001087 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001088 test_kqueue
Guido van Rossumaa782362001-09-02 03:58:41 +00001089 test_mmap
Guido van Rossumaa782362001-09-02 03:58:41 +00001090 test_openpty
Jack Jansen67975142003-01-08 16:31:11 +00001091 test_ossaudiodev
Guido van Rossumaa782362001-09-02 03:58:41 +00001092 test_poll
Jack Jansen67975142003-01-08 16:31:11 +00001093 test_popen
Guido van Rossumaa782362001-09-02 03:58:41 +00001094 test_popen2
Jack Jansen5bb97e62003-02-21 22:33:55 +00001095 test_posix
Guido van Rossumaa782362001-09-02 03:58:41 +00001096 test_pty
1097 test_pwd
Jack Jansen67975142003-01-08 16:31:11 +00001098 test_resource
Guido van Rossumaa782362001-09-02 03:58:41 +00001099 test_signal
Guido van Rossumaa782362001-09-02 03:58:41 +00001100 test_sundry
Jack Jansenc4d6bdd2003-03-07 15:38:11 +00001101 test_tarfile
Guido van Rossumaa782362001-09-02 03:58:41 +00001102 test_timing
Guido van Rossumaa782362001-09-02 03:58:41 +00001103 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001104 'unixware7':
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001105 """
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001106 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001107 test_bsddb185
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001108 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001109 test_epoll
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001110 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001111 test_kqueue
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001112 test_minidom
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001113 test_openpty
1114 test_pyexpat
1115 test_sax
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001116 test_sundry
Martin v. Löwis0ace3262001-09-05 14:38:48 +00001117 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001118 'openunix8':
1119 """
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001120 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001121 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001122 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001123 test_epoll
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001124 test_largefile
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001125 test_kqueue
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001126 test_minidom
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001127 test_openpty
1128 test_pyexpat
1129 test_sax
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001130 test_sundry
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001131 """,
1132 'sco_sv3':
1133 """
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001134 test_asynchat
1135 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001136 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001137 test_dl
1138 test_fork1
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001139 test_epoll
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001140 test_gettext
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001141 test_largefile
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001142 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001143 test_kqueue
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001144 test_minidom
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001145 test_openpty
1146 test_pyexpat
1147 test_queue
1148 test_sax
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001149 test_sundry
1150 test_thread
1151 test_threaded_import
1152 test_threadedtempfile
1153 test_threading
Martin v. Löwis21ee4092002-09-30 16:19:48 +00001154 """,
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001155 'riscos':
1156 """
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001157 test_asynchat
Martin v. Löwisa94568a2003-05-10 07:36:56 +00001158 test_atexit
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001159 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001160 test_bsddb185
Martin v. Löwisa94568a2003-05-10 07:36:56 +00001161 test_bsddb3
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001162 test_commands
1163 test_crypt
1164 test_dbm
1165 test_dl
1166 test_fcntl
1167 test_fork1
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001168 test_epoll
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001169 test_gdbm
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001170 test_grp
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001171 test_largefile
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001172 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001173 test_kqueue
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001174 test_mmap
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001175 test_openpty
1176 test_poll
1177 test_popen2
1178 test_pty
1179 test_pwd
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001180 test_strop
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001181 test_sundry
1182 test_thread
1183 test_threaded_import
1184 test_threadedtempfile
1185 test_threading
1186 test_timing
Guido van Rossume2ae77b2001-10-24 20:42:55 +00001187 """,
Jack Jansen8a97f4a2001-12-05 23:27:32 +00001188 'darwin':
Jack Jansen398c2362001-12-02 21:41:36 +00001189 """
Brett Cannon2bfb94c2003-10-13 04:27:47 +00001190 test__locale
Jack Jansenacda3392002-12-30 23:03:13 +00001191 test_bsddb
Guido van Rossum9d427002002-12-03 10:24:56 +00001192 test_bsddb3
Jack Jansen398c2362001-12-02 21:41:36 +00001193 test_curses
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001194 test_epoll
Jack Jansen398c2362001-12-02 21:41:36 +00001195 test_gdbm
Jack Jansen398c2362001-12-02 21:41:36 +00001196 test_largefile
Jack Jansenacda3392002-12-30 23:03:13 +00001197 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001198 test_kqueue
Jack Jansen398c2362001-12-02 21:41:36 +00001199 test_minidom
Jack Jansenacda3392002-12-30 23:03:13 +00001200 test_ossaudiodev
Jack Jansen398c2362001-12-02 21:41:36 +00001201 test_poll
Jack Jansen398c2362001-12-02 21:41:36 +00001202 """,
Guido van Rossum11c3f092002-07-17 15:08:24 +00001203 'sunos5':
1204 """
Guido van Rossum11c3f092002-07-17 15:08:24 +00001205 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001206 test_bsddb185
Guido van Rossum11c3f092002-07-17 15:08:24 +00001207 test_curses
1208 test_dbm
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001209 test_epoll
1210 test_kqueue
Guido van Rossum11c3f092002-07-17 15:08:24 +00001211 test_gdbm
Guido van Rossum11c3f092002-07-17 15:08:24 +00001212 test_gzip
Guido van Rossum11c3f092002-07-17 15:08:24 +00001213 test_openpty
Guido van Rossum11c3f092002-07-17 15:08:24 +00001214 test_zipfile
1215 test_zlib
Jeremy Hyltoned375e12002-07-17 15:56:55 +00001216 """,
Skip Montanarob3230212002-03-15 02:54:03 +00001217 'hp-ux11':
1218 """
Skip Montanarob3230212002-03-15 02:54:03 +00001219 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +00001220 test_bsddb185
Skip Montanarob3230212002-03-15 02:54:03 +00001221 test_curses
1222 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001223 test_epoll
Skip Montanarob3230212002-03-15 02:54:03 +00001224 test_gdbm
Skip Montanarob3230212002-03-15 02:54:03 +00001225 test_gzip
Skip Montanarob3230212002-03-15 02:54:03 +00001226 test_largefile
Skip Montanarob3230212002-03-15 02:54:03 +00001227 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001228 test_kqueue
Skip Montanarob3230212002-03-15 02:54:03 +00001229 test_minidom
Skip Montanarob3230212002-03-15 02:54:03 +00001230 test_openpty
1231 test_pyexpat
1232 test_sax
Skip Montanarob3230212002-03-15 02:54:03 +00001233 test_zipfile
1234 test_zlib
1235 """,
Martin v. Löwisf90ae202002-06-11 06:22:31 +00001236 'atheos':
Tim Petersc411dba2002-07-16 21:35:23 +00001237 """
Skip Montanaro823ba282003-05-06 20:36:24 +00001238 test_bsddb185
Tim Petersc411dba2002-07-16 21:35:23 +00001239 test_curses
1240 test_dl
Tim Petersc411dba2002-07-16 21:35:23 +00001241 test_gdbm
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001242 test_epoll
Tim Petersc411dba2002-07-16 21:35:23 +00001243 test_largefile
Tim Petersc411dba2002-07-16 21:35:23 +00001244 test_locale
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001245 test_kqueue
Tim Petersc411dba2002-07-16 21:35:23 +00001246 test_mhlib
1247 test_mmap
Tim Petersc411dba2002-07-16 21:35:23 +00001248 test_poll
1249 test_popen2
1250 test_resource
Tim Petersc411dba2002-07-16 21:35:23 +00001251 """,
Jason Tishler25115942002-12-05 15:18:15 +00001252 'cygwin':
1253 """
Skip Montanaro823ba282003-05-06 20:36:24 +00001254 test_bsddb185
Tim Petersb0f89e02002-12-05 17:20:25 +00001255 test_bsddb3
Jason Tishler25115942002-12-05 15:18:15 +00001256 test_curses
1257 test_dbm
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001258 test_epoll
Jason Tishlerc23f39c2003-07-22 18:35:58 +00001259 test_ioctl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001260 test_kqueue
Jason Tishler25115942002-12-05 15:18:15 +00001261 test_largefile
Jason Tishler25115942002-12-05 15:18:15 +00001262 test_locale
Jason Tishler5c4ded22003-02-05 16:46:01 +00001263 test_ossaudiodev
Jason Tishler25115942002-12-05 15:18:15 +00001264 test_socketserver
Jason Tishler25115942002-12-05 15:18:15 +00001265 """,
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001266 'os2emx':
1267 """
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001268 test_audioop
Skip Montanaro823ba282003-05-06 20:36:24 +00001269 test_bsddb185
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001270 test_bsddb3
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001271 test_commands
1272 test_curses
1273 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001274 test_epoll
1275 test_kqueue
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001276 test_largefile
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001277 test_mhlib
1278 test_mmap
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001279 test_openpty
1280 test_ossaudiodev
1281 test_pty
1282 test_resource
1283 test_signal
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001284 """,
Guido van Rossum944a6c32003-11-20 22:11:29 +00001285 'freebsd4':
1286 """
Guido van Rossum944a6c32003-11-20 22:11:29 +00001287 test_bsddb
1288 test_bsddb3
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001289 test_epoll
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001290 test_gdbm
Guido van Rossum944a6c32003-11-20 22:11:29 +00001291 test_locale
Guido van Rossum944a6c32003-11-20 22:11:29 +00001292 test_ossaudiodev
1293 test_pep277
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001294 test_pty
Guido van Rossum944a6c32003-11-20 22:11:29 +00001295 test_socketserver
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001296 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001297 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001298 test_ttk_guionly
1299 test_ttk_textonly
Guido van Rossum944a6c32003-11-20 22:11:29 +00001300 test_timeout
Guido van Rossum944a6c32003-11-20 22:11:29 +00001301 test_urllibnet
Jesse Noller37040cd2008-09-30 00:15:45 +00001302 test_multiprocessing
Martin v. Löwis56f88112003-06-07 20:01:37 +00001303 """,
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001304 'aix5':
1305 """
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001306 test_bsddb
1307 test_bsddb185
1308 test_bsddb3
1309 test_bz2
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001310 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001311 test_epoll
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001312 test_gdbm
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001313 test_gzip
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001314 test_kqueue
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001315 test_ossaudiodev
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001316 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001317 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001318 test_ttk_guionly
1319 test_ttk_textonly
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001320 test_zipimport
1321 test_zlib
1322 """,
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001323 'openbsd3':
1324 """
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001325 test_bsddb
1326 test_bsddb3
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001327 test_ctypes
1328 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001329 test_epoll
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001330 test_gdbm
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001331 test_locale
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001332 test_normalization
1333 test_ossaudiodev
1334 test_pep277
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001335 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001336 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001337 test_ttk_guionly
1338 test_ttk_textonly
Jesse Noller37040cd2008-09-30 00:15:45 +00001339 test_multiprocessing
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001340 """,
Georg Brandl08706872006-06-21 17:53:17 +00001341 'netbsd3':
1342 """
Georg Brandl08706872006-06-21 17:53:17 +00001343 test_bsddb
1344 test_bsddb185
1345 test_bsddb3
Georg Brandl08706872006-06-21 17:53:17 +00001346 test_ctypes
1347 test_curses
1348 test_dl
Christian Heimes0e9ab5f2008-03-21 23:49:44 +00001349 test_epoll
Georg Brandl08706872006-06-21 17:53:17 +00001350 test_gdbm
Georg Brandl08706872006-06-21 17:53:17 +00001351 test_locale
Georg Brandl08706872006-06-21 17:53:17 +00001352 test_ossaudiodev
1353 test_pep277
Georg Brandl08706872006-06-21 17:53:17 +00001354 test_tcl
Guilherme Polobbb7efd2009-02-09 19:21:21 +00001355 test_tk
Guilherme Polo7a77ee82009-01-28 19:28:04 +00001356 test_ttk_guionly
1357 test_ttk_textonly
Jesse Noller37040cd2008-09-30 00:15:45 +00001358 test_multiprocessing
Georg Brandl08706872006-06-21 17:53:17 +00001359 """,
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001360}
Martin v. Löwis32d0c1b2004-07-26 12:09:13 +00001361_expectations['freebsd5'] = _expectations['freebsd4']
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001362_expectations['freebsd6'] = _expectations['freebsd4']
Hye-Shik Chang4e422812005-07-17 02:36:59 +00001363_expectations['freebsd7'] = _expectations['freebsd4']
Hye-Shik Changea684742007-10-28 12:38:09 +00001364_expectations['freebsd8'] = _expectations['freebsd4']
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001365
Tim Petersb5b7b782001-08-12 01:20:39 +00001366class _ExpectedSkips:
1367 def __init__(self):
Tim Peters2a182db2002-10-09 01:07:11 +00001368 import os.path
Neal Norwitz55b61d22003-02-28 19:57:03 +00001369 from test import test_timeout
Tim Peters1b445d32002-11-24 18:53:11 +00001370
Tim Peters7c7efe92002-08-23 17:55:54 +00001371 self.valid = False
Tim Petersde14a302002-04-01 05:04:46 +00001372 if sys.platform in _expectations:
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001373 s = _expectations[sys.platform]
Raymond Hettingera690a992003-11-16 16:17:49 +00001374 self.expected = set(s.split())
Tim Peters1b445d32002-11-24 18:53:11 +00001375
Georg Brandlb2e208f2007-07-12 09:24:04 +00001376 # expected to be skipped on every platform, even Linux
1377 self.expected.add('test_linuxaudiodev')
1378
Tim Peters2a182db2002-10-09 01:07:11 +00001379 if not os.path.supports_unicode_filenames:
1380 self.expected.add('test_pep277')
Tim Peters1b445d32002-11-24 18:53:11 +00001381
Neal Norwitz55b61d22003-02-28 19:57:03 +00001382 if test_timeout.skip_expected:
1383 self.expected.add('test_timeout')
1384
Martin v. Löwisfba73692004-11-13 11:13:35 +00001385 if sys.maxint == 9223372036854775807L:
Martin v. Löwisfba73692004-11-13 11:13:35 +00001386 self.expected.add('test_imageop')
1387
Jack Jansen6afc5e02003-01-29 16:24:16 +00001388 if not sys.platform in ("mac", "darwin"):
Amaury Forgeot d'Arc8f40f062008-06-19 19:57:39 +00001389 MAC_ONLY = ["test_macos", "test_macostools", "test_aepack",
Georg Brandlb2e208f2007-07-12 09:24:04 +00001390 "test_plistlib", "test_scriptpackages",
1391 "test_applesingle"]
Neal Norwitz7035c982003-03-29 22:01:17 +00001392 for skip in MAC_ONLY:
1393 self.expected.add(skip)
Benjamin Peterson6f5a2b52008-06-19 21:39:06 +00001394 elif len(u'\0'.encode('unicode-internal')) == 4:
1395 self.expected.add("test_macostools")
1396
Tim Petersecd79eb2003-01-29 00:35:32 +00001397
1398 if sys.platform != "win32":
Georg Brandlb2e208f2007-07-12 09:24:04 +00001399 # test_sqlite is only reliable on Windows where the library
1400 # is distributed with Python
Neal Norwitz7035c982003-03-29 22:01:17 +00001401 WIN_ONLY = ["test_unicode_file", "test_winreg",
Georg Brandlb2e208f2007-07-12 09:24:04 +00001402 "test_winsound", "test_startfile",
1403 "test_sqlite"]
Neal Norwitz7035c982003-03-29 22:01:17 +00001404 for skip in WIN_ONLY:
1405 self.expected.add(skip)
Tim Petersf2715e02003-02-19 02:35:07 +00001406
Brett Cannondc48b742007-05-20 07:09:50 +00001407 if sys.platform != 'irix':
Georg Brandlb2e208f2007-07-12 09:24:04 +00001408 IRIX_ONLY = ["test_imageop", "test_al", "test_cd", "test_cl",
1409 "test_gl", "test_imgfile"]
Brett Cannondc48b742007-05-20 07:09:50 +00001410 for skip in IRIX_ONLY:
1411 self.expected.add(skip)
1412
Georg Brandlb2e208f2007-07-12 09:24:04 +00001413 if sys.platform != 'sunos5':
1414 self.expected.add('test_sunaudiodev')
1415 self.expected.add('test_nis')
1416
Steven Bethardd290b042008-03-21 21:01:18 +00001417 if not sys.py3kwarning:
1418 self.expected.add('test_py3kwarn')
Steven Betharde8e22cf2008-03-18 21:30:13 +00001419
Tim Peters7c7efe92002-08-23 17:55:54 +00001420 self.valid = True
Tim Petersb5b7b782001-08-12 01:20:39 +00001421
1422 def isvalid(self):
1423 "Return true iff _ExpectedSkips knows about the current platform."
1424 return self.valid
1425
1426 def getexpected(self):
1427 """Return set of test names we expect to skip on current platform.
1428
1429 self.isvalid() must be true.
1430 """
1431
1432 assert self.isvalid()
1433 return self.expected
1434
Guido van Rossum152494a1996-12-20 03:12:20 +00001435if __name__ == '__main__':
Barry Warsaw408b6d32002-07-30 23:27:12 +00001436 # Remove regrtest.py's own directory from the module search path. This
1437 # prevents relative imports from working, and relative imports will screw
1438 # up the testing framework. E.g. if both test.test_support and
1439 # test_support are imported, they will not contain the same globals, and
1440 # much of the testing framework relies on the globals in the
1441 # test.test_support module.
1442 mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0])))
1443 i = pathlen = len(sys.path)
1444 while i >= 0:
1445 i -= 1
1446 if os.path.abspath(os.path.normpath(sys.path[i])) == mydir:
1447 del sys.path[i]
Barry Warsaw08fca522001-08-20 22:33:46 +00001448 main()