blob: be60023c1a941ed244b22939215d5f7c2b67fdb1 [file] [log] [blame]
Guido van Rossum152494a1996-12-20 03:12:20 +00001#! /usr/bin/env python
2
3"""Regression test.
4
5This will find all modules whose name is "test_*" in the test
6directory, and run them. Various command line options provide
7additional facilities.
8
9Command line options:
10
Michael W. Hudson61147f62004-08-03 11:33:28 +000011-v: verbose -- run tests in verbose mode with output to stdout
Martin v. Löwis04824ce2006-03-10 21:26:16 +000012-w: verbose2 -- re-run failed tests in verbose mode
Michael W. Hudson61147f62004-08-03 11:33:28 +000013-q: quiet -- don't print anything except if a test fails
14-g: generate -- write the output file for a test instead of comparing it
15-x: exclude -- arguments are tests to *exclude*
16-s: single -- run only a single test (see below)
17-r: random -- randomize test execution order
18-f: fromfile -- read names of tests to run from a file (see below)
19-l: findleaks -- if GC is available detect tests that leak memory
20-u: use -- specify which special resource intensive tests to run
21-h: help -- print this text and exit
22-t: threshold -- call gc.set_threshold(N)
23-T: coverage -- turn on code coverage using the trace module
Walter Dörwaldaee4da62004-11-12 18:51:27 +000024-D: coverdir -- Directory where coverage files are put
25-N: nocoverdir -- Put coverage files alongside modules
Michael W. Hudson61147f62004-08-03 11:33:28 +000026-L: runleaks -- run the leaks(1) command just before exit
27-R: huntrleaks -- search for reference leaks (needs debug build, v. slow)
Guido van Rossum152494a1996-12-20 03:12:20 +000028
29If non-option arguments are present, they are names for tests to run,
30unless -x is given, in which case they are names for tests not to run.
31If no test names are given, all tests are run.
Guido van Rossumf58ed251997-03-07 21:04:33 +000032
Guido van Rossuma4122201997-08-18 20:08:24 +000033-v is incompatible with -g and does not compare test output files.
Barry Warsawe11e3de1999-01-28 19:51:51 +000034
Barry Warsaw3b6d0252004-02-07 22:43:03 +000035-T turns on code coverage tracing with the trace module.
36
Walter Dörwaldaee4da62004-11-12 18:51:27 +000037-D specifies the directory where coverage files are put.
38
39-N Put coverage files alongside modules.
40
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000041-s means to run only a single test and exit. This is useful when
42doing memory analysis on the Python interpreter (which tend to consume
43too many resources to run the full regression test non-stop). The
44file /tmp/pynexttest is read to find the next test to run. If this
45file is missing, the first test_*.py file in testdir or on the command
46line is used. (actually tempfile.gettempdir() is used instead of
47/tmp).
Barry Warsawe11e3de1999-01-28 19:51:51 +000048
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000049-f reads the names of tests from the file given as f's argument, one
50or more test names per line. Whitespace is ignored. Blank lines and
51lines beginning with '#' are ignored. This is especially useful for
52whittling down failures involving interactions among tests.
Tim Petersc5000df2002-06-02 21:42:01 +000053
Skip Montanaro0179a182004-06-06 15:53:18 +000054-L causes the leaks(1) command to be run just before exit if it exists.
55leaks(1) is available on Mac OS X and presumably on some other
56FreeBSD-derived systems.
57
Michael W. Hudson61147f62004-08-03 11:33:28 +000058-R runs each test several times and examines sys.gettotalrefcount() to
59see if the test appears to be leaking references. The argument should
60be of the form stab:run:fname where 'stab' is the number of times the
61test is run to let gettotalrefcount settle down, 'run' is the number
62of times further it is run and 'fname' is the name of the file the
63reports are written to. These parameters all have defaults (5, 4 and
64"reflog.txt" respectively), so the minimal invocation is '-R ::'.
65
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000066-u is used to specify which special resource intensive tests to run,
67such as those requiring large file support or network connectivity.
68The argument is a comma-separated list of words indicating the
69resources to test. Currently only the following are defined:
Barry Warsaw08fca522001-08-20 22:33:46 +000070
Fred Drake3a15dac2002-04-11 16:39:16 +000071 all - Enable all special resources.
72
Guido van Rossum315aa362003-03-11 14:46:48 +000073 audio - Tests that use the audio device. (There are known
74 cases of broken audio drivers that can crash Python or
75 even the Linux kernel.)
76
Andrew M. Kuchling2158df02001-10-22 15:26:09 +000077 curses - Tests that use curses and will modify the terminal's
78 state and output modes.
Tim Peters1633a2e2001-10-30 05:56:40 +000079
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000080 largefile - It is okay to run some test that may create huge
81 files. These tests can take a long time and may
82 consume >2GB of disk space temporarily.
Barry Warsaw08fca522001-08-20 22:33:46 +000083
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000084 network - It is okay to run tests that use external network
85 resource, e.g. testing SSL support for sockets.
Martin v. Löwis1c6b1a22002-11-19 17:47:07 +000086
87 bsddb - It is okay to run the bsddb testsuite, which takes
88 a long time to complete.
Fred Drake4dd0f7e2002-11-26 21:44:56 +000089
Raymond Hettinger7c85fa42004-07-01 11:01:35 +000090 decimal - Test the decimal module against a large suite that
91 verifies compliance with standards.
92
Jeremy Hylton4336eda2004-08-07 19:25:33 +000093 compiler - Test the compiler package by compiling all the source
94 in the standard library and test suite. This takes
Raymond Hettinger68c04532005-06-10 11:05:19 +000095 a long time. Enabling this resource also allows
96 test_tokenize to verify round-trip lexing on every
97 file in the test library.
Jeremy Hylton4336eda2004-08-07 19:25:33 +000098
Tim Peterseba28be2005-03-28 01:08:02 +000099 subprocess Run all tests for the subprocess module.
Peter Astrandf7f1bb72005-03-03 20:47:37 +0000100
Hye-Shik Changaaa2f1d2005-12-10 17:44:27 +0000101 urlfetch - It is okay to download files required on testing.
102
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000103To enable all resources except one, use '-uall,-<resource>'. For
104example, to run all the tests except for the bsddb tests, give the
105option '-uall,-bsddb'.
Guido van Rossum152494a1996-12-20 03:12:20 +0000106"""
107
Guido van Rossum152494a1996-12-20 03:12:20 +0000108import os
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000109import sys
Guido van Rossum152494a1996-12-20 03:12:20 +0000110import getopt
Skip Montanaroab1c7912000-06-30 16:39:27 +0000111import random
Guido van Rossumdc15c272002-08-12 21:55:51 +0000112import warnings
Neal Norwitz94a9c092006-03-16 06:30:02 +0000113import re
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000114import cStringIO
115import traceback
Guido van Rossumdc15c272002-08-12 21:55:51 +0000116
117# I see no other way to suppress these warnings;
118# putting them in test_grammar.py has no effect:
Guido van Rossum88b1def2002-08-14 17:54:48 +0000119warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
Guido van Rossumdc15c272002-08-12 21:55:51 +0000120 ".*test.test_grammar$")
Guido van Rossumc34c4fc2002-09-19 00:42:16 +0000121if sys.maxint > 0x7fffffff:
122 # Also suppress them in <string>, because for 64-bit platforms,
123 # that's where test_grammar.py hides them.
124 warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
125 "<string>")
Guido van Rossum152494a1996-12-20 03:12:20 +0000126
Guido van Rossumbb484652002-12-02 09:56:21 +0000127# MacOSX (a.k.a. Darwin) has a default stack size that is too small
128# for deeply recursive regular expressions. We see this as crashes in
129# the Python test suite when running test_re.py and test_sre.py. The
130# fix is to set the stack limit to 2048.
131# This approach may also be useful for other Unixy platforms that
132# suffer from small default stack limits.
133if sys.platform == 'darwin':
134 try:
135 import resource
136 except ImportError:
137 pass
138 else:
139 soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
140 newsoft = min(hard, max(soft, 1024*2048))
141 resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
142
Barry Warsaw04f357c2002-07-23 19:04:11 +0000143from test import test_support
Fred Drake3a15dac2002-04-11 16:39:16 +0000144
Raymond Hettinger7c85fa42004-07-01 11:01:35 +0000145RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
Hye-Shik Changaaa2f1d2005-12-10 17:44:27 +0000146 'decimal', 'compiler', 'subprocess', 'urlfetch')
Fred Drake3a15dac2002-04-11 16:39:16 +0000147
148
Barry Warsaw08fca522001-08-20 22:33:46 +0000149def usage(code, msg=''):
150 print __doc__
151 if msg: print msg
152 sys.exit(code)
153
154
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000155def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False,
156 exclude=False, single=False, randomize=False, fromfile=None,
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000157 findleaks=False, use_resources=None, trace=False, coverdir='coverage',
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000158 runleaks=False, huntrleaks=False, verbose2=False):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000159 """Execute a test suite.
160
Thomas Wouters7e474022000-07-16 12:04:32 +0000161 This also parses command-line options and modifies its behavior
Fred Drake004d5e62000-10-23 17:22:08 +0000162 accordingly.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000163
164 tests -- a list of strings containing test names (optional)
165 testdir -- the directory in which to look for tests (optional)
166
167 Users other than the Python test suite will certainly want to
168 specify testdir; if it's omitted, the directory containing the
Fred Drake004d5e62000-10-23 17:22:08 +0000169 Python test suite is searched for.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000170
171 If the tests argument is omitted, the tests listed on the
172 command-line will be used. If that's empty, too, then all *.py
173 files beginning with test_ will be used.
Skip Montanaroab1c7912000-06-30 16:39:27 +0000174
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000175 The other default arguments (verbose, quiet, generate, exclude, single,
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000176 randomize, findleaks, use_resources, trace and coverdir) allow programmers
177 calling main() directly to set the values that would normally be set by
178 flags on the command line.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000179 """
Fred Drake004d5e62000-10-23 17:22:08 +0000180
Tim Peters8dee8092001-09-25 20:05:11 +0000181 test_support.record_original_stdout(sys.stdout)
Guido van Rossum152494a1996-12-20 03:12:20 +0000182 try:
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000183 opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TD:NLR:w',
Barry Warsaw08fca522001-08-20 22:33:46 +0000184 ['help', 'verbose', 'quiet', 'generate',
Tim Petersc5000df2002-06-02 21:42:01 +0000185 'exclude', 'single', 'random', 'fromfile',
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000186 'findleaks', 'use=', 'threshold=', 'trace',
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000187 'coverdir=', 'nocoverdir', 'runleaks',
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000188 'huntrleaks=', 'verbose2',
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000189 ])
Guido van Rossum152494a1996-12-20 03:12:20 +0000190 except getopt.error, msg:
Barry Warsaw08fca522001-08-20 22:33:46 +0000191 usage(2, msg)
192
193 # Defaults
194 if use_resources is None:
195 use_resources = []
Guido van Rossum152494a1996-12-20 03:12:20 +0000196 for o, a in opts:
Barry Warsaw08fca522001-08-20 22:33:46 +0000197 if o in ('-h', '--help'):
198 usage(0)
199 elif o in ('-v', '--verbose'):
200 verbose += 1
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000201 elif o in ('-w', '--verbose2'):
202 verbose2 = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000203 elif o in ('-q', '--quiet'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000204 quiet = True;
Barry Warsaw08fca522001-08-20 22:33:46 +0000205 verbose = 0
206 elif o in ('-g', '--generate'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000207 generate = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000208 elif o in ('-x', '--exclude'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000209 exclude = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000210 elif o in ('-s', '--single'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000211 single = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000212 elif o in ('-r', '--randomize'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000213 randomize = True
Tim Petersc5000df2002-06-02 21:42:01 +0000214 elif o in ('-f', '--fromfile'):
215 fromfile = a
Barry Warsaw08fca522001-08-20 22:33:46 +0000216 elif o in ('-l', '--findleaks'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000217 findleaks = True
Skip Montanaro0179a182004-06-06 15:53:18 +0000218 elif o in ('-L', '--runleaks'):
219 runleaks = True
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000220 elif o in ('-t', '--threshold'):
221 import gc
222 gc.set_threshold(int(a))
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000223 elif o in ('-T', '--coverage'):
224 trace = True
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000225 elif o in ('-D', '--coverdir'):
226 coverdir = os.path.join(os.getcwd(), a)
227 elif o in ('-N', '--nocoverdir'):
228 coverdir = None
Michael W. Hudson61147f62004-08-03 11:33:28 +0000229 elif o in ('-R', '--huntrleaks'):
230 huntrleaks = a.split(':')
231 if len(huntrleaks) != 3:
232 print a, huntrleaks
233 usage(2, '-R takes three colon-separated arguments')
234 if len(huntrleaks[0]) == 0:
235 huntrleaks[0] = 5
236 else:
237 huntrleaks[0] = int(huntrleaks[0])
238 if len(huntrleaks[1]) == 0:
239 huntrleaks[1] = 4
240 else:
241 huntrleaks[1] = int(huntrleaks[1])
242 if len(huntrleaks[2]) == 0:
243 huntrleaks[2] = "reflog.txt"
Barry Warsaw08fca522001-08-20 22:33:46 +0000244 elif o in ('-u', '--use'):
Guido van Rossumfe3f6962001-09-06 16:09:41 +0000245 u = [x.lower() for x in a.split(',')]
246 for r in u:
Fred Drake3a15dac2002-04-11 16:39:16 +0000247 if r == 'all':
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000248 use_resources[:] = RESOURCE_NAMES
249 continue
250 remove = False
251 if r[0] == '-':
252 remove = True
253 r = r[1:]
Fred Drake3a15dac2002-04-11 16:39:16 +0000254 if r not in RESOURCE_NAMES:
255 usage(1, 'Invalid -u/--use option: ' + a)
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000256 if remove:
257 if r in use_resources:
258 use_resources.remove(r)
259 elif r not in use_resources:
Andrew MacIntyree41abab2002-04-30 12:11:04 +0000260 use_resources.append(r)
Guido van Rossuma4122201997-08-18 20:08:24 +0000261 if generate and verbose:
Barry Warsaw08fca522001-08-20 22:33:46 +0000262 usage(2, "-g and -v don't go together!")
Tim Petersc5000df2002-06-02 21:42:01 +0000263 if single and fromfile:
264 usage(2, "-s and -f don't go together!")
Barry Warsaw08fca522001-08-20 22:33:46 +0000265
Guido van Rossum152494a1996-12-20 03:12:20 +0000266 good = []
267 bad = []
268 skipped = []
Fred Drake9a0db072003-02-03 15:19:30 +0000269 resource_denieds = []
Barry Warsawe11e3de1999-01-28 19:51:51 +0000270
Neil Schemenauerd569f232000-09-22 15:29:28 +0000271 if findleaks:
Barry Warsawa873b032000-08-03 15:50:37 +0000272 try:
273 import gc
274 except ImportError:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000275 print 'No GC available, disabling findleaks.'
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000276 findleaks = False
Barry Warsawa873b032000-08-03 15:50:37 +0000277 else:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000278 # Uncomment the line below to report garbage that is not
279 # freeable by reference counting alone. By default only
280 # garbage that is not collectable by the GC is reported.
281 #gc.set_debug(gc.DEBUG_SAVEALL)
Neil Schemenauerd569f232000-09-22 15:29:28 +0000282 found_garbage = []
Barry Warsawa873b032000-08-03 15:50:37 +0000283
Barry Warsawe11e3de1999-01-28 19:51:51 +0000284 if single:
285 from tempfile import gettempdir
286 filename = os.path.join(gettempdir(), 'pynexttest')
287 try:
288 fp = open(filename, 'r')
Eric S. Raymondfc170b12001-02-09 11:51:27 +0000289 next = fp.read().strip()
Barry Warsawe11e3de1999-01-28 19:51:51 +0000290 tests = [next]
291 fp.close()
292 except IOError:
293 pass
Tim Petersc5000df2002-06-02 21:42:01 +0000294
295 if fromfile:
296 tests = []
297 fp = open(fromfile)
298 for line in fp:
299 guts = line.split() # assuming no test has whitespace in its name
300 if guts and not guts[0].startswith('#'):
301 tests.extend(guts)
302 fp.close()
303
304 # Strip .py extensions.
305 if args:
306 args = map(removepy, args)
307 if tests:
308 tests = map(removepy, tests)
309
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000310 stdtests = STDTESTS[:]
311 nottests = NOTTESTS[:]
Guido van Rossum152494a1996-12-20 03:12:20 +0000312 if exclude:
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000313 for arg in args:
314 if arg in stdtests:
315 stdtests.remove(arg)
316 nottests[:0] = args
Guido van Rossum41360a41998-03-26 19:42:58 +0000317 args = []
Guido van Rossum747e1ca1998-08-24 13:48:36 +0000318 tests = tests or args or findtests(testdir, stdtests, nottests)
Barry Warsawe11e3de1999-01-28 19:51:51 +0000319 if single:
320 tests = tests[:1]
Skip Montanaroab1c7912000-06-30 16:39:27 +0000321 if randomize:
322 random.shuffle(tests)
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000323 if trace:
324 import trace
325 tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],
326 trace=False, count=True)
Guido van Rossum41360a41998-03-26 19:42:58 +0000327 test_support.verbose = verbose # Tell tests to be moderately quiet
Barry Warsaw08fca522001-08-20 22:33:46 +0000328 test_support.use_resources = use_resources
Guido van Rossum5796d262000-04-21 21:35:06 +0000329 save_modules = sys.modules.keys()
Guido van Rossum152494a1996-12-20 03:12:20 +0000330 for test in tests:
Guido van Rossum41360a41998-03-26 19:42:58 +0000331 if not quiet:
332 print test
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000333 sys.stdout.flush()
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000334 if trace:
335 # If we're tracing code coverage, then we don't exit with status
336 # if on a false return value from main.
337 tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',
338 globals=globals(), locals=vars())
Guido van Rossum41360a41998-03-26 19:42:58 +0000339 else:
Neal Norwitz14ca3272006-02-28 18:05:43 +0000340 try:
341 ok = runtest(test, generate, verbose, quiet, testdir,
342 huntrleaks)
343 except KeyboardInterrupt:
344 # print a newline separate from the ^C
345 print
346 break
347 except:
348 raise
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000349 if ok > 0:
350 good.append(test)
351 elif ok == 0:
352 bad.append(test)
353 else:
354 skipped.append(test)
355 if ok == -2:
356 resource_denieds.append(test)
Neil Schemenauerd569f232000-09-22 15:29:28 +0000357 if findleaks:
358 gc.collect()
359 if gc.garbage:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000360 print "Warning: test created", len(gc.garbage),
361 print "uncollectable object(s)."
362 # move the uncollectable objects somewhere so we don't see
363 # them again
Neil Schemenauerd569f232000-09-22 15:29:28 +0000364 found_garbage.extend(gc.garbage)
365 del gc.garbage[:]
Guido van Rossum5796d262000-04-21 21:35:06 +0000366 # Unload the newly imported modules (best effort finalization)
367 for module in sys.modules.keys():
Guido van Rossum51931142000-05-05 14:27:39 +0000368 if module not in save_modules and module.startswith("test."):
Guido van Rossum5796d262000-04-21 21:35:06 +0000369 test_support.unload(module)
Jeremy Hylton7a1ea0e2001-10-17 13:45:28 +0000370
371 # The lists won't be sorted if running with -r
372 good.sort()
373 bad.sort()
374 skipped.sort()
Tim Peterse0c446b2001-10-18 21:57:37 +0000375
Guido van Rossum152494a1996-12-20 03:12:20 +0000376 if good and not quiet:
Guido van Rossum41360a41998-03-26 19:42:58 +0000377 if not bad and not skipped and len(good) > 1:
378 print "All",
379 print count(len(good), "test"), "OK."
Tim Peters1a4d77b2000-12-30 22:21:22 +0000380 if verbose:
Barry Warsaw408b6d32002-07-30 23:27:12 +0000381 print "CAUTION: stdout isn't compared in verbose mode:"
382 print "a test that passes in verbose mode may fail without it."
Guido van Rossum152494a1996-12-20 03:12:20 +0000383 if bad:
Tim Petersa45da922001-08-12 03:45:50 +0000384 print count(len(bad), "test"), "failed:"
385 printlist(bad)
Guido van Rossum152494a1996-12-20 03:12:20 +0000386 if skipped and not quiet:
Tim Petersa45da922001-08-12 03:45:50 +0000387 print count(len(skipped), "test"), "skipped:"
388 printlist(skipped)
Barry Warsawe11e3de1999-01-28 19:51:51 +0000389
Tim Petersb5b7b782001-08-12 01:20:39 +0000390 e = _ExpectedSkips()
Tim Petersa2be2d62001-08-12 02:01:09 +0000391 plat = sys.platform
Tim Petersb5b7b782001-08-12 01:20:39 +0000392 if e.isvalid():
Raymond Hettingera690a992003-11-16 16:17:49 +0000393 surprise = set(skipped) - e.getexpected() - set(resource_denieds)
Tim Petersb5b7b782001-08-12 01:20:39 +0000394 if surprise:
395 print count(len(surprise), "skip"), \
Tim Petersa45da922001-08-12 03:45:50 +0000396 "unexpected on", plat + ":"
397 printlist(surprise)
Tim Petersb5b7b782001-08-12 01:20:39 +0000398 else:
399 print "Those skips are all expected on", plat + "."
400 else:
401 print "Ask someone to teach regrtest.py about which tests are"
402 print "expected to get skipped on", plat + "."
403
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000404 if verbose2 and bad:
405 print "Re-running failed tests in verbose mode"
406 for test in bad:
Tim Peters922dd7d2006-03-10 23:37:10 +0000407 print "Re-running test %r in verbose mode" % test
408 sys.stdout.flush()
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000409 try:
410 test_support.verbose = 1
411 ok = runtest(test, generate, 1, quiet, testdir,
412 huntrleaks)
413 except KeyboardInterrupt:
414 # print a newline separate from the ^C
415 print
416 break
417 except:
418 raise
419
Barry Warsawe11e3de1999-01-28 19:51:51 +0000420 if single:
421 alltests = findtests(testdir, stdtests, nottests)
422 for i in range(len(alltests)):
423 if tests[0] == alltests[i]:
424 if i == len(alltests) - 1:
425 os.unlink(filename)
426 else:
427 fp = open(filename, 'w')
428 fp.write(alltests[i+1] + '\n')
429 fp.close()
430 break
431 else:
432 os.unlink(filename)
433
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000434 if trace:
435 r = tracer.results()
436 r.write_results(show_missing=True, summary=True, coverdir=coverdir)
437
Skip Montanaro0179a182004-06-06 15:53:18 +0000438 if runleaks:
439 os.system("leaks %d" % os.getpid())
440
Tim Peters5943b4a2003-07-23 00:30:39 +0000441 sys.exit(len(bad) > 0)
Barry Warsaw08fca522001-08-20 22:33:46 +0000442
Guido van Rossum152494a1996-12-20 03:12:20 +0000443
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000444STDTESTS = [
Guido van Rossum152494a1996-12-20 03:12:20 +0000445 'test_grammar',
446 'test_opcodes',
447 'test_operations',
448 'test_builtin',
449 'test_exceptions',
450 'test_types',
451 ]
452
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000453NOTTESTS = [
Guido van Rossum152494a1996-12-20 03:12:20 +0000454 'test_support',
Jeremy Hylton62e2c7e2001-02-28 17:48:06 +0000455 'test_future1',
456 'test_future2',
Jeremy Hylton8471a352001-08-20 20:33:42 +0000457 'test_future3',
Guido van Rossum152494a1996-12-20 03:12:20 +0000458 ]
459
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000460def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):
Guido van Rossum152494a1996-12-20 03:12:20 +0000461 """Return a list of all applicable test modules."""
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000462 if not testdir: testdir = findtestdir()
Guido van Rossum152494a1996-12-20 03:12:20 +0000463 names = os.listdir(testdir)
464 tests = []
465 for name in names:
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000466 if name[:5] == "test_" and name[-3:] == os.extsep+"py":
Guido van Rossum41360a41998-03-26 19:42:58 +0000467 modname = name[:-3]
468 if modname not in stdtests and modname not in nottests:
469 tests.append(modname)
Guido van Rossum152494a1996-12-20 03:12:20 +0000470 tests.sort()
471 return stdtests + tests
472
Michael W. Hudson61147f62004-08-03 11:33:28 +0000473def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000474 """Run a single test.
475 test -- the name of the test
476 generate -- if true, generate output, instead of running the test
477 and comparing it to a previously created output file
478 verbose -- if true, print more messages
Trent Mickf29f47b2000-08-11 19:02:59 +0000479 quiet -- if true, don't print 'skipped' messages (probably redundant)
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000480 testdir -- test directory
481 """
Guido van Rossum152494a1996-12-20 03:12:20 +0000482 test_support.unload(test)
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000483 if not testdir:
484 testdir = findtestdir()
Guido van Rossum152494a1996-12-20 03:12:20 +0000485 outputdir = os.path.join(testdir, "output")
486 outputfile = os.path.join(outputdir, test)
Tim Peters9390cc12001-09-28 20:14:46 +0000487 if verbose:
Guido van Rossum41360a41998-03-26 19:42:58 +0000488 cfp = None
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000489 else:
Raymond Hettinger74e67662003-05-17 20:44:12 +0000490 cfp = cStringIO.StringIO()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000491 if huntrleaks:
492 refrep = open(huntrleaks[2], "a")
Guido van Rossum152494a1996-12-20 03:12:20 +0000493 try:
Tim Peters342ca752001-09-25 19:13:20 +0000494 save_stdout = sys.stdout
Guido van Rossum41360a41998-03-26 19:42:58 +0000495 try:
496 if cfp:
497 sys.stdout = cfp
498 print test # Output file starts with test name
Barry Warsaw408b6d32002-07-30 23:27:12 +0000499 if test.startswith('test.'):
500 abstest = test
501 else:
502 # Always import it from the test package
503 abstest = 'test.' + test
504 the_package = __import__(abstest, globals(), locals(), [])
505 the_module = getattr(the_package, test)
Tim Petersd9742212001-05-22 18:28:25 +0000506 # Most tests run to completion simply as a side-effect of
507 # being imported. For the benefit of tests that can't run
508 # that way (like test_threaded_import), explicitly invoke
509 # their test_main() function (if it exists).
510 indirect_test = getattr(the_module, "test_main", None)
511 if indirect_test is not None:
512 indirect_test()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000513 if huntrleaks:
514 # This code *is* hackish and inelegant, yes.
515 # But it seems to do the job.
516 import copy_reg
517 fs = warnings.filters[:]
518 ps = copy_reg.dispatch_table.copy()
519 pic = sys.path_importer_cache.copy()
520 import gc
521 def cleanup():
Neal Norwitz545d4962005-11-25 02:55:49 +0000522 import _strptime, linecache, warnings, dircache
523 import urlparse, urllib, urllib2
Michael W. Hudson61147f62004-08-03 11:33:28 +0000524 from distutils.dir_util import _path_created
525 _path_created.clear()
526 warnings.filters[:] = fs
527 gc.collect()
Neal Norwitz94a9c092006-03-16 06:30:02 +0000528 re.purge()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000529 _strptime._regex_cache.clear()
530 urlparse.clear_cache()
Neal Norwitz545d4962005-11-25 02:55:49 +0000531 urllib.urlcleanup()
532 urllib2.install_opener(None)
Michael W. Hudson61147f62004-08-03 11:33:28 +0000533 copy_reg.dispatch_table.clear()
534 copy_reg.dispatch_table.update(ps)
535 sys.path_importer_cache.clear()
536 sys.path_importer_cache.update(pic)
537 dircache.reset()
Michael W. Hudson1a728522005-06-14 09:31:28 +0000538 linecache.clearcache()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000539 if indirect_test:
540 def run_the_test():
541 indirect_test()
542 else:
543 def run_the_test():
544 reload(the_module)
545 deltas = []
546 repcount = huntrleaks[0] + huntrleaks[1]
547 print >> sys.stderr, "beginning", repcount, "repetitions"
548 print >> sys.stderr, \
549 ("1234567890"*(repcount//10 + 1))[:repcount]
550 for i in range(repcount):
551 rc = sys.gettotalrefcount()
552 run_the_test()
553 sys.stderr.write('.')
554 cleanup()
555 deltas.append(sys.gettotalrefcount() - rc - 2)
556 print >>sys.stderr
557 if max(map(abs, deltas[-huntrleaks[1]:])) > 0:
Michael W. Hudsone667e662004-08-12 18:27:48 +0000558 print >>sys.stderr, test, 'leaked', \
559 deltas[-huntrleaks[1]:], 'references'
Michael W. Hudson61147f62004-08-03 11:33:28 +0000560 print >>refrep, test, 'leaked', \
561 deltas[-huntrleaks[1]:], 'references'
562 # The end of the huntrleaks hackishness.
Guido van Rossum41360a41998-03-26 19:42:58 +0000563 finally:
Tim Peters342ca752001-09-25 19:13:20 +0000564 sys.stdout = save_stdout
Fred Drake9a0db072003-02-03 15:19:30 +0000565 except test_support.ResourceDenied, msg:
566 if not quiet:
567 print test, "skipped --", msg
568 sys.stdout.flush()
569 return -2
Thomas Wouters3af826e2000-08-04 13:17:51 +0000570 except (ImportError, test_support.TestSkipped), msg:
Trent Mickf29f47b2000-08-11 19:02:59 +0000571 if not quiet:
Fred Drakede4742b2002-10-17 20:36:08 +0000572 print test, "skipped --", msg
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000573 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000574 return -1
Fred Drakefe5c22a2000-08-18 16:04:05 +0000575 except KeyboardInterrupt:
576 raise
Guido van Rossum152494a1996-12-20 03:12:20 +0000577 except test_support.TestFailed, msg:
Guido van Rossum41360a41998-03-26 19:42:58 +0000578 print "test", test, "failed --", msg
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000579 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000580 return 0
Guido van Rossum9e48b271997-07-16 01:56:13 +0000581 except:
Guido van Rossum41360a41998-03-26 19:42:58 +0000582 type, value = sys.exc_info()[:2]
Fred Drake27c4b392000-08-23 20:34:40 +0000583 print "test", test, "crashed --", str(type) + ":", value
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000584 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000585 if verbose:
586 traceback.print_exc(file=sys.stdout)
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000587 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000588 return 0
Guido van Rossum152494a1996-12-20 03:12:20 +0000589 else:
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000590 if not cfp:
591 return 1
592 output = cfp.getvalue()
Fred Drakee51fe8d2001-05-29 17:10:51 +0000593 if generate:
Fred Drakee51fe8d2001-05-29 17:10:51 +0000594 if output == test + "\n":
595 if os.path.exists(outputfile):
596 # Write it since it already exists (and the contents
597 # may have changed), but let the user know it isn't
598 # needed:
Fred Drakee51fe8d2001-05-29 17:10:51 +0000599 print "output file", outputfile, \
600 "is no longer needed; consider removing it"
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000601 else:
602 # We don't need it, so don't create it.
603 return 1
604 fp = open(outputfile, "w")
605 fp.write(output)
606 fp.close()
607 return 1
608 if os.path.exists(outputfile):
609 fp = open(outputfile, "r")
610 expected = fp.read()
611 fp.close()
612 else:
613 expected = test + "\n"
Michael W. Hudson61147f62004-08-03 11:33:28 +0000614 if output == expected or huntrleaks:
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000615 return 1
616 print "test", test, "produced unexpected output:"
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000617 sys.stdout.flush()
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000618 reportdiff(expected, output)
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000619 sys.stdout.flush()
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000620 return 0
621
622def reportdiff(expected, output):
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000623 import difflib
Tim Petersc377b162001-09-22 05:31:03 +0000624 print "*" * 70
625 a = expected.splitlines(1)
626 b = output.splitlines(1)
Guido van Rossumcf691932001-09-21 21:06:22 +0000627 sm = difflib.SequenceMatcher(a=a, b=b)
628 tuples = sm.get_opcodes()
Tim Petersc377b162001-09-22 05:31:03 +0000629
Guido van Rossumcf691932001-09-21 21:06:22 +0000630 def pair(x0, x1):
Tim Petersc377b162001-09-22 05:31:03 +0000631 # x0:x1 are 0-based slice indices; convert to 1-based line indices.
Guido van Rossumcf691932001-09-21 21:06:22 +0000632 x0 += 1
633 if x0 >= x1:
Tim Petersc377b162001-09-22 05:31:03 +0000634 return "line " + str(x0)
Guido van Rossumcf691932001-09-21 21:06:22 +0000635 else:
Tim Petersc377b162001-09-22 05:31:03 +0000636 return "lines %d-%d" % (x0, x1)
637
Guido van Rossumcf691932001-09-21 21:06:22 +0000638 for op, a0, a1, b0, b1 in tuples:
639 if op == 'equal':
640 pass
Tim Petersc377b162001-09-22 05:31:03 +0000641
Guido van Rossumcf691932001-09-21 21:06:22 +0000642 elif op == 'delete':
Tim Petersc377b162001-09-22 05:31:03 +0000643 print "***", pair(a0, a1), "of expected output missing:"
Guido van Rossumcf691932001-09-21 21:06:22 +0000644 for line in a[a0:a1]:
Tim Petersc377b162001-09-22 05:31:03 +0000645 print "-", line,
646
Guido van Rossumcf691932001-09-21 21:06:22 +0000647 elif op == 'replace':
Tim Petersc377b162001-09-22 05:31:03 +0000648 print "*** mismatch between", pair(a0, a1), "of expected", \
649 "output and", pair(b0, b1), "of actual output:"
650 for line in difflib.ndiff(a[a0:a1], b[b0:b1]):
651 print line,
652
Guido van Rossumcf691932001-09-21 21:06:22 +0000653 elif op == 'insert':
Tim Petersc377b162001-09-22 05:31:03 +0000654 print "***", pair(b0, b1), "of actual output doesn't appear", \
655 "in expected output after line", str(a1)+":"
Guido van Rossumcf691932001-09-21 21:06:22 +0000656 for line in b[b0:b1]:
Tim Petersc377b162001-09-22 05:31:03 +0000657 print "+", line,
658
Guido van Rossumcf691932001-09-21 21:06:22 +0000659 else:
660 print "get_opcodes() returned bad tuple?!?!", (op, a0, a1, b0, b1)
Tim Petersc377b162001-09-22 05:31:03 +0000661
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000662 print "*" * 70
Guido van Rossum152494a1996-12-20 03:12:20 +0000663
664def findtestdir():
665 if __name__ == '__main__':
Guido van Rossum41360a41998-03-26 19:42:58 +0000666 file = sys.argv[0]
Guido van Rossum152494a1996-12-20 03:12:20 +0000667 else:
Guido van Rossum41360a41998-03-26 19:42:58 +0000668 file = __file__
Guido van Rossum152494a1996-12-20 03:12:20 +0000669 testdir = os.path.dirname(file) or os.curdir
670 return testdir
671
Tim Petersc5000df2002-06-02 21:42:01 +0000672def removepy(name):
673 if name.endswith(os.extsep + "py"):
674 name = name[:-3]
675 return name
676
Guido van Rossum152494a1996-12-20 03:12:20 +0000677def count(n, word):
678 if n == 1:
Guido van Rossum41360a41998-03-26 19:42:58 +0000679 return "%d %s" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +0000680 else:
Guido van Rossum41360a41998-03-26 19:42:58 +0000681 return "%d %ss" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +0000682
Tim Petersa45da922001-08-12 03:45:50 +0000683def printlist(x, width=70, indent=4):
Tim Peters7c7efe92002-08-23 17:55:54 +0000684 """Print the elements of iterable x to stdout.
Tim Petersa45da922001-08-12 03:45:50 +0000685
686 Optional arg width (default 70) is the maximum line length.
687 Optional arg indent (default 4) is the number of blanks with which to
688 begin each line.
689 """
690
Tim Petersba78bc42002-07-04 19:45:06 +0000691 from textwrap import fill
692 blanks = ' ' * indent
693 print fill(' '.join(map(str, x)), width,
694 initial_indent=blanks, subsequent_indent=blanks)
Tim Petersa45da922001-08-12 03:45:50 +0000695
Tim Petersde14a302002-04-01 05:04:46 +0000696# Map sys.platform to a string containing the basenames of tests
697# expected to be skipped on that platform.
Tim Peters2a182db2002-10-09 01:07:11 +0000698#
699# Special cases:
700# test_pep277
701# The _ExpectedSkips constructor adds this to the set of expected
702# skips if not os.path.supports_unicode_filenames.
Tim Petersb4ee4eb2002-12-04 03:26:57 +0000703# test_socket_ssl
704# Controlled by test_socket_ssl.skip_expected. Requires the network
705# resource, and a socket module with ssl support.
Neal Norwitz55b61d22003-02-28 19:57:03 +0000706# test_timeout
707# Controlled by test_timeout.skip_expected. Requires the network
708# resource and a socket module.
Tim Petersde14a302002-04-01 05:04:46 +0000709
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000710_expectations = {
711 'win32':
712 """
Tim Petersc7c516a2003-09-20 22:06:13 +0000713 test__locale
Raymond Hettinger901dc982003-11-20 19:02:02 +0000714 test_applesingle
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000715 test_al
Skip Montanaro823ba282003-05-06 20:36:24 +0000716 test_bsddb185
Tim Peters78e35f92002-11-22 20:00:34 +0000717 test_bsddb3
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000718 test_cd
719 test_cl
720 test_commands
721 test_crypt
Tim Petersd7030572001-10-22 22:06:08 +0000722 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000723 test_dbm
724 test_dl
725 test_fcntl
726 test_fork1
727 test_gdbm
728 test_gl
729 test_grp
730 test_imgfile
Tim Petersfd8e6e52003-03-04 00:26:38 +0000731 test_ioctl
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000732 test_largefile
733 test_linuxaudiodev
734 test_mhlib
735 test_nis
736 test_openpty
Tim Petersefc4b122002-12-10 18:47:56 +0000737 test_ossaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000738 test_poll
Tim Peters003eb302003-02-17 21:48:48 +0000739 test_posix
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000740 test_pty
741 test_pwd
Tim Peters1e33ffa2002-04-23 23:09:02 +0000742 test_resource
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000743 test_signal
Anthony Baxterc51ee692006-04-01 00:57:31 +0000744 test_sqlite
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000745 test_sunaudiodev
Tim Peterscea2cc42004-08-04 02:32:03 +0000746 test_threadsignals
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000747 test_timing
Tim Petersb84de022006-03-22 02:58:17 +0000748 test_wait3
749 test_wait4
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000750 """,
751 'linux2':
752 """
753 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000754 test_applesingle
Skip Montanaro823ba282003-05-06 20:36:24 +0000755 test_bsddb185
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000756 test_cd
757 test_cl
Guido van Rossumf66dacd2001-10-23 15:10:55 +0000758 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000759 test_dl
760 test_gl
761 test_imgfile
762 test_largefile
Guido van Rossum4507ec72003-02-14 19:29:22 +0000763 test_linuxaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000764 test_nis
765 test_ntpath
Guido van Rossum4507ec72003-02-14 19:29:22 +0000766 test_ossaudiodev
Anthony Baxterc51ee692006-04-01 00:57:31 +0000767 test_sqlite
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000768 test_sunaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000769 """,
Jack Jansen49a806e2001-08-28 14:49:00 +0000770 'mac':
Guido van Rossumaa782362001-09-02 03:58:41 +0000771 """
772 test_al
Jack Jansen67975142003-01-08 16:31:11 +0000773 test_atexit
Guido van Rossumaa782362001-09-02 03:58:41 +0000774 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000775 test_bsddb185
Jack Jansen67975142003-01-08 16:31:11 +0000776 test_bsddb3
777 test_bz2
Guido van Rossumaa782362001-09-02 03:58:41 +0000778 test_cd
779 test_cl
780 test_commands
781 test_crypt
Jack Jansenb3be2162001-11-30 14:16:36 +0000782 test_curses
Guido van Rossumaa782362001-09-02 03:58:41 +0000783 test_dbm
784 test_dl
785 test_fcntl
786 test_fork1
787 test_gl
788 test_grp
Jack Jansenc4d6bdd2003-03-07 15:38:11 +0000789 test_ioctl
Guido van Rossumaa782362001-09-02 03:58:41 +0000790 test_imgfile
791 test_largefile
792 test_linuxaudiodev
793 test_locale
794 test_mmap
795 test_nis
796 test_ntpath
797 test_openpty
Jack Jansen67975142003-01-08 16:31:11 +0000798 test_ossaudiodev
Guido van Rossumaa782362001-09-02 03:58:41 +0000799 test_poll
Jack Jansen67975142003-01-08 16:31:11 +0000800 test_popen
Guido van Rossumaa782362001-09-02 03:58:41 +0000801 test_popen2
Jack Jansen5bb97e62003-02-21 22:33:55 +0000802 test_posix
Guido van Rossumaa782362001-09-02 03:58:41 +0000803 test_pty
804 test_pwd
Jack Jansen67975142003-01-08 16:31:11 +0000805 test_resource
Guido van Rossumaa782362001-09-02 03:58:41 +0000806 test_signal
Anthony Baxterc51ee692006-04-01 00:57:31 +0000807 test_sqlite
Guido van Rossumaa782362001-09-02 03:58:41 +0000808 test_sunaudiodev
809 test_sundry
Jack Jansenc4d6bdd2003-03-07 15:38:11 +0000810 test_tarfile
Guido van Rossumaa782362001-09-02 03:58:41 +0000811 test_timing
Guido van Rossumaa782362001-09-02 03:58:41 +0000812 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000813 'unixware7':
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000814 """
815 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000816 test_applesingle
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000817 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000818 test_bsddb185
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000819 test_cd
820 test_cl
821 test_dl
822 test_gl
823 test_imgfile
824 test_largefile
825 test_linuxaudiodev
826 test_minidom
827 test_nis
828 test_ntpath
829 test_openpty
830 test_pyexpat
831 test_sax
Anthony Baxterc51ee692006-04-01 00:57:31 +0000832 test_sqlite
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000833 test_sunaudiodev
834 test_sundry
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000835 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000836 'openunix8':
837 """
838 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000839 test_applesingle
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000840 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000841 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000842 test_cd
843 test_cl
844 test_dl
845 test_gl
846 test_imgfile
847 test_largefile
848 test_linuxaudiodev
849 test_minidom
850 test_nis
851 test_ntpath
852 test_openpty
853 test_pyexpat
854 test_sax
Anthony Baxterc51ee692006-04-01 00:57:31 +0000855 test_sqlite
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000856 test_sunaudiodev
857 test_sundry
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000858 """,
859 'sco_sv3':
860 """
861 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000862 test_applesingle
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000863 test_asynchat
864 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000865 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000866 test_cd
867 test_cl
868 test_dl
869 test_fork1
870 test_gettext
871 test_gl
872 test_imgfile
873 test_largefile
874 test_linuxaudiodev
875 test_locale
876 test_minidom
877 test_nis
878 test_ntpath
879 test_openpty
880 test_pyexpat
881 test_queue
882 test_sax
Anthony Baxterc51ee692006-04-01 00:57:31 +0000883 test_sqlite
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000884 test_sunaudiodev
885 test_sundry
886 test_thread
887 test_threaded_import
888 test_threadedtempfile
889 test_threading
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000890 """,
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000891 'riscos':
892 """
893 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000894 test_applesingle
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000895 test_asynchat
Martin v. Löwisa94568a2003-05-10 07:36:56 +0000896 test_atexit
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000897 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000898 test_bsddb185
Martin v. Löwisa94568a2003-05-10 07:36:56 +0000899 test_bsddb3
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000900 test_cd
901 test_cl
902 test_commands
903 test_crypt
904 test_dbm
905 test_dl
906 test_fcntl
907 test_fork1
908 test_gdbm
909 test_gl
910 test_grp
911 test_imgfile
912 test_largefile
913 test_linuxaudiodev
914 test_locale
915 test_mmap
916 test_nis
917 test_ntpath
918 test_openpty
919 test_poll
920 test_popen2
921 test_pty
922 test_pwd
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000923 test_strop
Anthony Baxterc51ee692006-04-01 00:57:31 +0000924 test_sqlite
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000925 test_sunaudiodev
926 test_sundry
927 test_thread
928 test_threaded_import
929 test_threadedtempfile
930 test_threading
931 test_timing
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000932 """,
Jack Jansen8a97f4a2001-12-05 23:27:32 +0000933 'darwin':
Jack Jansen398c2362001-12-02 21:41:36 +0000934 """
Brett Cannon2bfb94c2003-10-13 04:27:47 +0000935 test__locale
Jack Jansen398c2362001-12-02 21:41:36 +0000936 test_al
Jack Jansenacda3392002-12-30 23:03:13 +0000937 test_bsddb
Guido van Rossum9d427002002-12-03 10:24:56 +0000938 test_bsddb3
Jack Jansen398c2362001-12-02 21:41:36 +0000939 test_cd
940 test_cl
941 test_curses
942 test_dl
943 test_gdbm
944 test_gl
945 test_imgfile
946 test_largefile
947 test_linuxaudiodev
Jack Jansenacda3392002-12-30 23:03:13 +0000948 test_locale
Jack Jansen398c2362001-12-02 21:41:36 +0000949 test_minidom
950 test_nis
951 test_ntpath
Jack Jansenacda3392002-12-30 23:03:13 +0000952 test_ossaudiodev
Jack Jansen398c2362001-12-02 21:41:36 +0000953 test_poll
Anthony Baxterc51ee692006-04-01 00:57:31 +0000954 test_sqlite
Jack Jansen398c2362001-12-02 21:41:36 +0000955 test_sunaudiodev
Jack Jansen398c2362001-12-02 21:41:36 +0000956 """,
Guido van Rossum11c3f092002-07-17 15:08:24 +0000957 'sunos5':
958 """
959 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000960 test_applesingle
Guido van Rossum11c3f092002-07-17 15:08:24 +0000961 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000962 test_bsddb185
Guido van Rossum11c3f092002-07-17 15:08:24 +0000963 test_cd
964 test_cl
965 test_curses
966 test_dbm
Guido van Rossum11c3f092002-07-17 15:08:24 +0000967 test_gdbm
968 test_gl
969 test_gzip
970 test_imgfile
971 test_linuxaudiodev
Guido van Rossum11c3f092002-07-17 15:08:24 +0000972 test_openpty
Anthony Baxterc51ee692006-04-01 00:57:31 +0000973 test_sqlite
Guido van Rossum11c3f092002-07-17 15:08:24 +0000974 test_zipfile
975 test_zlib
Jeremy Hyltoned375e12002-07-17 15:56:55 +0000976 """,
Skip Montanarob3230212002-03-15 02:54:03 +0000977 'hp-ux11':
978 """
979 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000980 test_applesingle
Skip Montanarob3230212002-03-15 02:54:03 +0000981 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000982 test_bsddb185
Skip Montanarob3230212002-03-15 02:54:03 +0000983 test_cd
984 test_cl
985 test_curses
986 test_dl
987 test_gdbm
988 test_gl
989 test_gzip
990 test_imgfile
991 test_largefile
992 test_linuxaudiodev
993 test_locale
994 test_minidom
995 test_nis
996 test_ntpath
997 test_openpty
998 test_pyexpat
999 test_sax
Anthony Baxterc51ee692006-04-01 00:57:31 +00001000 test_sqlite
Skip Montanarob3230212002-03-15 02:54:03 +00001001 test_sunaudiodev
Skip Montanarob3230212002-03-15 02:54:03 +00001002 test_zipfile
1003 test_zlib
1004 """,
Martin v. Löwisf90ae202002-06-11 06:22:31 +00001005 'atheos':
Tim Petersc411dba2002-07-16 21:35:23 +00001006 """
1007 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +00001008 test_applesingle
Skip Montanaro823ba282003-05-06 20:36:24 +00001009 test_bsddb185
Tim Petersc411dba2002-07-16 21:35:23 +00001010 test_cd
1011 test_cl
1012 test_curses
1013 test_dl
Tim Petersc411dba2002-07-16 21:35:23 +00001014 test_gdbm
1015 test_gl
1016 test_imgfile
1017 test_largefile
1018 test_linuxaudiodev
1019 test_locale
1020 test_mhlib
1021 test_mmap
Tim Petersc411dba2002-07-16 21:35:23 +00001022 test_nis
1023 test_poll
1024 test_popen2
1025 test_resource
Anthony Baxterc51ee692006-04-01 00:57:31 +00001026 test_sqlite
Tim Petersc411dba2002-07-16 21:35:23 +00001027 test_sunaudiodev
Tim Petersc411dba2002-07-16 21:35:23 +00001028 """,
Jason Tishler25115942002-12-05 15:18:15 +00001029 'cygwin':
1030 """
1031 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +00001032 test_applesingle
Skip Montanaro823ba282003-05-06 20:36:24 +00001033 test_bsddb185
Tim Petersb0f89e02002-12-05 17:20:25 +00001034 test_bsddb3
Jason Tishler25115942002-12-05 15:18:15 +00001035 test_cd
1036 test_cl
1037 test_curses
1038 test_dbm
Jason Tishler25115942002-12-05 15:18:15 +00001039 test_gl
1040 test_imgfile
Jason Tishlerc23f39c2003-07-22 18:35:58 +00001041 test_ioctl
Jason Tishler25115942002-12-05 15:18:15 +00001042 test_largefile
1043 test_linuxaudiodev
1044 test_locale
Jason Tishler25115942002-12-05 15:18:15 +00001045 test_nis
Jason Tishler5c4ded22003-02-05 16:46:01 +00001046 test_ossaudiodev
Jason Tishler25115942002-12-05 15:18:15 +00001047 test_socketserver
Anthony Baxterc51ee692006-04-01 00:57:31 +00001048 test_sqlite
Jason Tishler25115942002-12-05 15:18:15 +00001049 test_sunaudiodev
Jason Tishler25115942002-12-05 15:18:15 +00001050 """,
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001051 'os2emx':
1052 """
1053 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +00001054 test_applesingle
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001055 test_audioop
Skip Montanaro823ba282003-05-06 20:36:24 +00001056 test_bsddb185
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001057 test_bsddb3
1058 test_cd
1059 test_cl
1060 test_commands
1061 test_curses
1062 test_dl
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001063 test_gl
1064 test_imgfile
1065 test_largefile
1066 test_linuxaudiodev
1067 test_mhlib
1068 test_mmap
1069 test_nis
1070 test_openpty
1071 test_ossaudiodev
1072 test_pty
1073 test_resource
1074 test_signal
Anthony Baxterc51ee692006-04-01 00:57:31 +00001075 test_sqlite
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001076 test_sunaudiodev
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001077 """,
Guido van Rossum944a6c32003-11-20 22:11:29 +00001078 'freebsd4':
1079 """
1080 test_aepack
1081 test_al
1082 test_applesingle
1083 test_bsddb
1084 test_bsddb3
1085 test_cd
1086 test_cl
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001087 test_gdbm
Guido van Rossum944a6c32003-11-20 22:11:29 +00001088 test_gl
1089 test_imgfile
1090 test_linuxaudiodev
1091 test_locale
1092 test_macfs
1093 test_macostools
1094 test_nis
Guido van Rossum944a6c32003-11-20 22:11:29 +00001095 test_ossaudiodev
1096 test_pep277
1097 test_plistlib
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001098 test_pty
Guido van Rossum944a6c32003-11-20 22:11:29 +00001099 test_scriptpackages
1100 test_socket_ssl
1101 test_socketserver
Anthony Baxterc51ee692006-04-01 00:57:31 +00001102 test_sqlite
Guido van Rossum944a6c32003-11-20 22:11:29 +00001103 test_sunaudiodev
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001104 test_tcl
Guido van Rossum944a6c32003-11-20 22:11:29 +00001105 test_timeout
1106 test_unicode_file
1107 test_urllibnet
1108 test_winreg
1109 test_winsound
Martin v. Löwis56f88112003-06-07 20:01:37 +00001110 """,
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001111 'aix5':
1112 """
1113 test_aepack
1114 test_al
1115 test_applesingle
1116 test_bsddb
1117 test_bsddb185
1118 test_bsddb3
1119 test_bz2
1120 test_cd
1121 test_cl
1122 test_dl
1123 test_gdbm
1124 test_gl
1125 test_gzip
1126 test_imgfile
1127 test_linuxaudiodev
1128 test_macfs
1129 test_macostools
1130 test_nis
1131 test_ossaudiodev
Anthony Baxterc51ee692006-04-01 00:57:31 +00001132 test_sqlite
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001133 test_sunaudiodev
1134 test_tcl
1135 test_winreg
1136 test_winsound
1137 test_zipimport
1138 test_zlib
1139 """,
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001140 'openbsd3':
1141 """
1142 test_aepack
1143 test_al
1144 test_applesingle
1145 test_bsddb
1146 test_bsddb3
1147 test_cd
1148 test_cl
1149 test_ctypes
1150 test_dl
1151 test_gdbm
1152 test_gl
1153 test_imgfile
1154 test_linuxaudiodev
1155 test_locale
1156 test_macfs
1157 test_macostools
1158 test_nis
1159 test_normalization
1160 test_ossaudiodev
1161 test_pep277
1162 test_plistlib
1163 test_scriptpackages
1164 test_tcl
Anthony Baxterc51ee692006-04-01 00:57:31 +00001165 test_sqlite
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001166 test_sunaudiodev
1167 test_unicode_file
1168 test_winreg
1169 test_winsound
1170 """,
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001171}
Martin v. Löwis32d0c1b2004-07-26 12:09:13 +00001172_expectations['freebsd5'] = _expectations['freebsd4']
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001173_expectations['freebsd6'] = _expectations['freebsd4']
Hye-Shik Chang4e422812005-07-17 02:36:59 +00001174_expectations['freebsd7'] = _expectations['freebsd4']
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001175
Tim Petersb5b7b782001-08-12 01:20:39 +00001176class _ExpectedSkips:
1177 def __init__(self):
Tim Peters2a182db2002-10-09 01:07:11 +00001178 import os.path
Tim Petersb4ee4eb2002-12-04 03:26:57 +00001179 from test import test_socket_ssl
Neal Norwitz55b61d22003-02-28 19:57:03 +00001180 from test import test_timeout
Tim Peters1b445d32002-11-24 18:53:11 +00001181
Tim Peters7c7efe92002-08-23 17:55:54 +00001182 self.valid = False
Tim Petersde14a302002-04-01 05:04:46 +00001183 if sys.platform in _expectations:
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001184 s = _expectations[sys.platform]
Raymond Hettingera690a992003-11-16 16:17:49 +00001185 self.expected = set(s.split())
Tim Peters1b445d32002-11-24 18:53:11 +00001186
Tim Peters2a182db2002-10-09 01:07:11 +00001187 if not os.path.supports_unicode_filenames:
1188 self.expected.add('test_pep277')
Tim Peters1b445d32002-11-24 18:53:11 +00001189
Tim Petersb4ee4eb2002-12-04 03:26:57 +00001190 if test_socket_ssl.skip_expected:
1191 self.expected.add('test_socket_ssl')
1192
Neal Norwitz55b61d22003-02-28 19:57:03 +00001193 if test_timeout.skip_expected:
1194 self.expected.add('test_timeout')
1195
Martin v. Löwisfba73692004-11-13 11:13:35 +00001196 if sys.maxint == 9223372036854775807L:
1197 self.expected.add('test_rgbimg')
1198 self.expected.add('test_imageop')
1199
Jack Jansen6afc5e02003-01-29 16:24:16 +00001200 if not sys.platform in ("mac", "darwin"):
Neal Norwitz7035c982003-03-29 22:01:17 +00001201 MAC_ONLY = ["test_macostools", "test_macfs", "test_aepack",
1202 "test_plistlib", "test_scriptpackages"]
1203 for skip in MAC_ONLY:
1204 self.expected.add(skip)
Tim Petersecd79eb2003-01-29 00:35:32 +00001205
1206 if sys.platform != "win32":
Neal Norwitz7035c982003-03-29 22:01:17 +00001207 WIN_ONLY = ["test_unicode_file", "test_winreg",
1208 "test_winsound"]
1209 for skip in WIN_ONLY:
1210 self.expected.add(skip)
Tim Petersf2715e02003-02-19 02:35:07 +00001211
Tim Peters7c7efe92002-08-23 17:55:54 +00001212 self.valid = True
Tim Petersb5b7b782001-08-12 01:20:39 +00001213
1214 def isvalid(self):
1215 "Return true iff _ExpectedSkips knows about the current platform."
1216 return self.valid
1217
1218 def getexpected(self):
1219 """Return set of test names we expect to skip on current platform.
1220
1221 self.isvalid() must be true.
1222 """
1223
1224 assert self.isvalid()
1225 return self.expected
1226
Guido van Rossum152494a1996-12-20 03:12:20 +00001227if __name__ == '__main__':
Barry Warsaw408b6d32002-07-30 23:27:12 +00001228 # Remove regrtest.py's own directory from the module search path. This
1229 # prevents relative imports from working, and relative imports will screw
1230 # up the testing framework. E.g. if both test.test_support and
1231 # test_support are imported, they will not contain the same globals, and
1232 # much of the testing framework relies on the globals in the
1233 # test.test_support module.
1234 mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0])))
1235 i = pathlen = len(sys.path)
1236 while i >= 0:
1237 i -= 1
1238 if os.path.abspath(os.path.normpath(sys.path[i])) == mydir:
1239 del sys.path[i]
1240 if len(sys.path) == pathlen:
1241 print 'Could not find %r in sys.path to remove it' % mydir
Barry Warsaw08fca522001-08-20 22:33:46 +00001242 main()