blob: 11090867507be642e82186ee68954b9a9ce2cb55 [file] [log] [blame]
Guido van Rossum152494a1996-12-20 03:12:20 +00001#! /usr/bin/env python
2
3"""Regression test.
4
5This will find all modules whose name is "test_*" in the test
6directory, and run them. Various command line options provide
7additional facilities.
8
9Command line options:
10
Michael W. Hudson61147f62004-08-03 11:33:28 +000011-v: verbose -- run tests in verbose mode with output to stdout
Martin v. Löwis04824ce2006-03-10 21:26:16 +000012-w: verbose2 -- re-run failed tests in verbose mode
Michael W. Hudson61147f62004-08-03 11:33:28 +000013-q: quiet -- don't print anything except if a test fails
14-g: generate -- write the output file for a test instead of comparing it
15-x: exclude -- arguments are tests to *exclude*
16-s: single -- run only a single test (see below)
17-r: random -- randomize test execution order
18-f: fromfile -- read names of tests to run from a file (see below)
19-l: findleaks -- if GC is available detect tests that leak memory
20-u: use -- specify which special resource intensive tests to run
21-h: help -- print this text and exit
22-t: threshold -- call gc.set_threshold(N)
23-T: coverage -- turn on code coverage using the trace module
Walter Dörwaldaee4da62004-11-12 18:51:27 +000024-D: coverdir -- Directory where coverage files are put
25-N: nocoverdir -- Put coverage files alongside modules
Michael W. Hudson61147f62004-08-03 11:33:28 +000026-L: runleaks -- run the leaks(1) command just before exit
27-R: huntrleaks -- search for reference leaks (needs debug build, v. slow)
Guido van Rossum152494a1996-12-20 03:12:20 +000028
29If non-option arguments are present, they are names for tests to run,
30unless -x is given, in which case they are names for tests not to run.
31If no test names are given, all tests are run.
Guido van Rossumf58ed251997-03-07 21:04:33 +000032
Guido van Rossuma4122201997-08-18 20:08:24 +000033-v is incompatible with -g and does not compare test output files.
Barry Warsawe11e3de1999-01-28 19:51:51 +000034
Barry Warsaw3b6d0252004-02-07 22:43:03 +000035-T turns on code coverage tracing with the trace module.
36
Walter Dörwaldaee4da62004-11-12 18:51:27 +000037-D specifies the directory where coverage files are put.
38
39-N Put coverage files alongside modules.
40
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000041-s means to run only a single test and exit. This is useful when
42doing memory analysis on the Python interpreter (which tend to consume
43too many resources to run the full regression test non-stop). The
44file /tmp/pynexttest is read to find the next test to run. If this
45file is missing, the first test_*.py file in testdir or on the command
46line is used. (actually tempfile.gettempdir() is used instead of
47/tmp).
Barry Warsawe11e3de1999-01-28 19:51:51 +000048
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000049-f reads the names of tests from the file given as f's argument, one
50or more test names per line. Whitespace is ignored. Blank lines and
51lines beginning with '#' are ignored. This is especially useful for
52whittling down failures involving interactions among tests.
Tim Petersc5000df2002-06-02 21:42:01 +000053
Skip Montanaro0179a182004-06-06 15:53:18 +000054-L causes the leaks(1) command to be run just before exit if it exists.
55leaks(1) is available on Mac OS X and presumably on some other
56FreeBSD-derived systems.
57
Michael W. Hudson61147f62004-08-03 11:33:28 +000058-R runs each test several times and examines sys.gettotalrefcount() to
59see if the test appears to be leaking references. The argument should
60be of the form stab:run:fname where 'stab' is the number of times the
61test is run to let gettotalrefcount settle down, 'run' is the number
62of times further it is run and 'fname' is the name of the file the
63reports are written to. These parameters all have defaults (5, 4 and
64"reflog.txt" respectively), so the minimal invocation is '-R ::'.
65
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000066-u is used to specify which special resource intensive tests to run,
67such as those requiring large file support or network connectivity.
68The argument is a comma-separated list of words indicating the
69resources to test. Currently only the following are defined:
Barry Warsaw08fca522001-08-20 22:33:46 +000070
Fred Drake3a15dac2002-04-11 16:39:16 +000071 all - Enable all special resources.
72
Guido van Rossum315aa362003-03-11 14:46:48 +000073 audio - Tests that use the audio device. (There are known
74 cases of broken audio drivers that can crash Python or
75 even the Linux kernel.)
76
Andrew M. Kuchling2158df02001-10-22 15:26:09 +000077 curses - Tests that use curses and will modify the terminal's
78 state and output modes.
Tim Peters1633a2e2001-10-30 05:56:40 +000079
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000080 largefile - It is okay to run some test that may create huge
81 files. These tests can take a long time and may
82 consume >2GB of disk space temporarily.
Barry Warsaw08fca522001-08-20 22:33:46 +000083
Guido van Rossum9e9d4f82002-06-07 15:17:03 +000084 network - It is okay to run tests that use external network
85 resource, e.g. testing SSL support for sockets.
Martin v. Löwis1c6b1a22002-11-19 17:47:07 +000086
87 bsddb - It is okay to run the bsddb testsuite, which takes
88 a long time to complete.
Fred Drake4dd0f7e2002-11-26 21:44:56 +000089
Raymond Hettinger7c85fa42004-07-01 11:01:35 +000090 decimal - Test the decimal module against a large suite that
91 verifies compliance with standards.
92
Jeremy Hylton4336eda2004-08-07 19:25:33 +000093 compiler - Test the compiler package by compiling all the source
94 in the standard library and test suite. This takes
Raymond Hettinger68c04532005-06-10 11:05:19 +000095 a long time. Enabling this resource also allows
96 test_tokenize to verify round-trip lexing on every
97 file in the test library.
Jeremy Hylton4336eda2004-08-07 19:25:33 +000098
Tim Peterseba28be2005-03-28 01:08:02 +000099 subprocess Run all tests for the subprocess module.
Peter Astrandf7f1bb72005-03-03 20:47:37 +0000100
Hye-Shik Changaaa2f1d2005-12-10 17:44:27 +0000101 urlfetch - It is okay to download files required on testing.
102
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000103To enable all resources except one, use '-uall,-<resource>'. For
104example, to run all the tests except for the bsddb tests, give the
105option '-uall,-bsddb'.
Guido van Rossum152494a1996-12-20 03:12:20 +0000106"""
107
Guido van Rossum152494a1996-12-20 03:12:20 +0000108import os
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000109import sys
Guido van Rossum152494a1996-12-20 03:12:20 +0000110import getopt
Skip Montanaroab1c7912000-06-30 16:39:27 +0000111import random
Guido van Rossumdc15c272002-08-12 21:55:51 +0000112import warnings
Neal Norwitz94a9c092006-03-16 06:30:02 +0000113import re
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000114import cStringIO
115import traceback
Guido van Rossumdc15c272002-08-12 21:55:51 +0000116
117# I see no other way to suppress these warnings;
118# putting them in test_grammar.py has no effect:
Guido van Rossum88b1def2002-08-14 17:54:48 +0000119warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
Guido van Rossumdc15c272002-08-12 21:55:51 +0000120 ".*test.test_grammar$")
Guido van Rossumc34c4fc2002-09-19 00:42:16 +0000121if sys.maxint > 0x7fffffff:
122 # Also suppress them in <string>, because for 64-bit platforms,
123 # that's where test_grammar.py hides them.
124 warnings.filterwarnings("ignore", "hex/oct constants", FutureWarning,
125 "<string>")
Guido van Rossum152494a1996-12-20 03:12:20 +0000126
Guido van Rossumbb484652002-12-02 09:56:21 +0000127# MacOSX (a.k.a. Darwin) has a default stack size that is too small
128# for deeply recursive regular expressions. We see this as crashes in
129# the Python test suite when running test_re.py and test_sre.py. The
130# fix is to set the stack limit to 2048.
131# This approach may also be useful for other Unixy platforms that
132# suffer from small default stack limits.
133if sys.platform == 'darwin':
134 try:
135 import resource
136 except ImportError:
137 pass
138 else:
139 soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
140 newsoft = min(hard, max(soft, 1024*2048))
141 resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
142
Barry Warsaw04f357c2002-07-23 19:04:11 +0000143from test import test_support
Fred Drake3a15dac2002-04-11 16:39:16 +0000144
Raymond Hettinger7c85fa42004-07-01 11:01:35 +0000145RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
Hye-Shik Changaaa2f1d2005-12-10 17:44:27 +0000146 'decimal', 'compiler', 'subprocess', 'urlfetch')
Fred Drake3a15dac2002-04-11 16:39:16 +0000147
148
Barry Warsaw08fca522001-08-20 22:33:46 +0000149def usage(code, msg=''):
150 print __doc__
151 if msg: print msg
152 sys.exit(code)
153
154
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000155def main(tests=None, testdir=None, verbose=0, quiet=False, generate=False,
156 exclude=False, single=False, randomize=False, fromfile=None,
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000157 findleaks=False, use_resources=None, trace=False, coverdir='coverage',
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000158 runleaks=False, huntrleaks=False, verbose2=False):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000159 """Execute a test suite.
160
Thomas Wouters7e474022000-07-16 12:04:32 +0000161 This also parses command-line options and modifies its behavior
Fred Drake004d5e62000-10-23 17:22:08 +0000162 accordingly.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000163
164 tests -- a list of strings containing test names (optional)
165 testdir -- the directory in which to look for tests (optional)
166
167 Users other than the Python test suite will certainly want to
168 specify testdir; if it's omitted, the directory containing the
Fred Drake004d5e62000-10-23 17:22:08 +0000169 Python test suite is searched for.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000170
171 If the tests argument is omitted, the tests listed on the
172 command-line will be used. If that's empty, too, then all *.py
173 files beginning with test_ will be used.
Skip Montanaroab1c7912000-06-30 16:39:27 +0000174
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000175 The other default arguments (verbose, quiet, generate, exclude, single,
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000176 randomize, findleaks, use_resources, trace and coverdir) allow programmers
177 calling main() directly to set the values that would normally be set by
178 flags on the command line.
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000179 """
Fred Drake004d5e62000-10-23 17:22:08 +0000180
Tim Peters8dee8092001-09-25 20:05:11 +0000181 test_support.record_original_stdout(sys.stdout)
Guido van Rossum152494a1996-12-20 03:12:20 +0000182 try:
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000183 opts, args = getopt.getopt(sys.argv[1:], 'hvgqxsrf:lu:t:TD:NLR:w',
Barry Warsaw08fca522001-08-20 22:33:46 +0000184 ['help', 'verbose', 'quiet', 'generate',
Tim Petersc5000df2002-06-02 21:42:01 +0000185 'exclude', 'single', 'random', 'fromfile',
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000186 'findleaks', 'use=', 'threshold=', 'trace',
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000187 'coverdir=', 'nocoverdir', 'runleaks',
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000188 'huntrleaks=', 'verbose2',
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000189 ])
Guido van Rossum152494a1996-12-20 03:12:20 +0000190 except getopt.error, msg:
Barry Warsaw08fca522001-08-20 22:33:46 +0000191 usage(2, msg)
192
193 # Defaults
194 if use_resources is None:
195 use_resources = []
Guido van Rossum152494a1996-12-20 03:12:20 +0000196 for o, a in opts:
Barry Warsaw08fca522001-08-20 22:33:46 +0000197 if o in ('-h', '--help'):
198 usage(0)
199 elif o in ('-v', '--verbose'):
200 verbose += 1
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000201 elif o in ('-w', '--verbose2'):
202 verbose2 = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000203 elif o in ('-q', '--quiet'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000204 quiet = True;
Barry Warsaw08fca522001-08-20 22:33:46 +0000205 verbose = 0
206 elif o in ('-g', '--generate'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000207 generate = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000208 elif o in ('-x', '--exclude'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000209 exclude = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000210 elif o in ('-s', '--single'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000211 single = True
Barry Warsaw08fca522001-08-20 22:33:46 +0000212 elif o in ('-r', '--randomize'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000213 randomize = True
Tim Petersc5000df2002-06-02 21:42:01 +0000214 elif o in ('-f', '--fromfile'):
215 fromfile = a
Barry Warsaw08fca522001-08-20 22:33:46 +0000216 elif o in ('-l', '--findleaks'):
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000217 findleaks = True
Skip Montanaro0179a182004-06-06 15:53:18 +0000218 elif o in ('-L', '--runleaks'):
219 runleaks = True
Guido van Rossum9e9d4f82002-06-07 15:17:03 +0000220 elif o in ('-t', '--threshold'):
221 import gc
222 gc.set_threshold(int(a))
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000223 elif o in ('-T', '--coverage'):
224 trace = True
Walter Dörwaldaee4da62004-11-12 18:51:27 +0000225 elif o in ('-D', '--coverdir'):
226 coverdir = os.path.join(os.getcwd(), a)
227 elif o in ('-N', '--nocoverdir'):
228 coverdir = None
Michael W. Hudson61147f62004-08-03 11:33:28 +0000229 elif o in ('-R', '--huntrleaks'):
230 huntrleaks = a.split(':')
231 if len(huntrleaks) != 3:
232 print a, huntrleaks
233 usage(2, '-R takes three colon-separated arguments')
234 if len(huntrleaks[0]) == 0:
235 huntrleaks[0] = 5
236 else:
237 huntrleaks[0] = int(huntrleaks[0])
238 if len(huntrleaks[1]) == 0:
239 huntrleaks[1] = 4
240 else:
241 huntrleaks[1] = int(huntrleaks[1])
242 if len(huntrleaks[2]) == 0:
243 huntrleaks[2] = "reflog.txt"
Barry Warsaw08fca522001-08-20 22:33:46 +0000244 elif o in ('-u', '--use'):
Guido van Rossumfe3f6962001-09-06 16:09:41 +0000245 u = [x.lower() for x in a.split(',')]
246 for r in u:
Fred Drake3a15dac2002-04-11 16:39:16 +0000247 if r == 'all':
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000248 use_resources[:] = RESOURCE_NAMES
249 continue
250 remove = False
251 if r[0] == '-':
252 remove = True
253 r = r[1:]
Fred Drake3a15dac2002-04-11 16:39:16 +0000254 if r not in RESOURCE_NAMES:
255 usage(1, 'Invalid -u/--use option: ' + a)
Fred Drake4dd0f7e2002-11-26 21:44:56 +0000256 if remove:
257 if r in use_resources:
258 use_resources.remove(r)
259 elif r not in use_resources:
Andrew MacIntyree41abab2002-04-30 12:11:04 +0000260 use_resources.append(r)
Guido van Rossuma4122201997-08-18 20:08:24 +0000261 if generate and verbose:
Barry Warsaw08fca522001-08-20 22:33:46 +0000262 usage(2, "-g and -v don't go together!")
Tim Petersc5000df2002-06-02 21:42:01 +0000263 if single and fromfile:
264 usage(2, "-s and -f don't go together!")
Barry Warsaw08fca522001-08-20 22:33:46 +0000265
Guido van Rossum152494a1996-12-20 03:12:20 +0000266 good = []
267 bad = []
268 skipped = []
Fred Drake9a0db072003-02-03 15:19:30 +0000269 resource_denieds = []
Barry Warsawe11e3de1999-01-28 19:51:51 +0000270
Neil Schemenauerd569f232000-09-22 15:29:28 +0000271 if findleaks:
Barry Warsawa873b032000-08-03 15:50:37 +0000272 try:
273 import gc
274 except ImportError:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000275 print 'No GC available, disabling findleaks.'
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000276 findleaks = False
Barry Warsawa873b032000-08-03 15:50:37 +0000277 else:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000278 # Uncomment the line below to report garbage that is not
279 # freeable by reference counting alone. By default only
280 # garbage that is not collectable by the GC is reported.
281 #gc.set_debug(gc.DEBUG_SAVEALL)
Neil Schemenauerd569f232000-09-22 15:29:28 +0000282 found_garbage = []
Barry Warsawa873b032000-08-03 15:50:37 +0000283
Barry Warsawe11e3de1999-01-28 19:51:51 +0000284 if single:
285 from tempfile import gettempdir
286 filename = os.path.join(gettempdir(), 'pynexttest')
287 try:
288 fp = open(filename, 'r')
Eric S. Raymondfc170b12001-02-09 11:51:27 +0000289 next = fp.read().strip()
Barry Warsawe11e3de1999-01-28 19:51:51 +0000290 tests = [next]
291 fp.close()
292 except IOError:
293 pass
Tim Petersc5000df2002-06-02 21:42:01 +0000294
295 if fromfile:
296 tests = []
297 fp = open(fromfile)
298 for line in fp:
299 guts = line.split() # assuming no test has whitespace in its name
300 if guts and not guts[0].startswith('#'):
301 tests.extend(guts)
302 fp.close()
303
304 # Strip .py extensions.
305 if args:
306 args = map(removepy, args)
307 if tests:
308 tests = map(removepy, tests)
309
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000310 stdtests = STDTESTS[:]
311 nottests = NOTTESTS[:]
Guido van Rossum152494a1996-12-20 03:12:20 +0000312 if exclude:
Guido van Rossum6c74fea1998-08-25 12:29:08 +0000313 for arg in args:
314 if arg in stdtests:
315 stdtests.remove(arg)
316 nottests[:0] = args
Guido van Rossum41360a41998-03-26 19:42:58 +0000317 args = []
Guido van Rossum747e1ca1998-08-24 13:48:36 +0000318 tests = tests or args or findtests(testdir, stdtests, nottests)
Barry Warsawe11e3de1999-01-28 19:51:51 +0000319 if single:
320 tests = tests[:1]
Skip Montanaroab1c7912000-06-30 16:39:27 +0000321 if randomize:
322 random.shuffle(tests)
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000323 if trace:
324 import trace
325 tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],
326 trace=False, count=True)
Guido van Rossum41360a41998-03-26 19:42:58 +0000327 test_support.verbose = verbose # Tell tests to be moderately quiet
Barry Warsaw08fca522001-08-20 22:33:46 +0000328 test_support.use_resources = use_resources
Guido van Rossum5796d262000-04-21 21:35:06 +0000329 save_modules = sys.modules.keys()
Guido van Rossum152494a1996-12-20 03:12:20 +0000330 for test in tests:
Guido van Rossum41360a41998-03-26 19:42:58 +0000331 if not quiet:
332 print test
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000333 sys.stdout.flush()
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000334 if trace:
335 # If we're tracing code coverage, then we don't exit with status
336 # if on a false return value from main.
337 tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',
338 globals=globals(), locals=vars())
Guido van Rossum41360a41998-03-26 19:42:58 +0000339 else:
Neal Norwitz14ca3272006-02-28 18:05:43 +0000340 try:
341 ok = runtest(test, generate, verbose, quiet, testdir,
342 huntrleaks)
343 except KeyboardInterrupt:
344 # print a newline separate from the ^C
345 print
346 break
347 except:
348 raise
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000349 if ok > 0:
350 good.append(test)
351 elif ok == 0:
352 bad.append(test)
353 else:
354 skipped.append(test)
355 if ok == -2:
356 resource_denieds.append(test)
Neil Schemenauerd569f232000-09-22 15:29:28 +0000357 if findleaks:
358 gc.collect()
359 if gc.garbage:
Neil Schemenauer8a00abc2000-10-13 01:32:42 +0000360 print "Warning: test created", len(gc.garbage),
361 print "uncollectable object(s)."
362 # move the uncollectable objects somewhere so we don't see
363 # them again
Neil Schemenauerd569f232000-09-22 15:29:28 +0000364 found_garbage.extend(gc.garbage)
365 del gc.garbage[:]
Guido van Rossum5796d262000-04-21 21:35:06 +0000366 # Unload the newly imported modules (best effort finalization)
367 for module in sys.modules.keys():
Guido van Rossum51931142000-05-05 14:27:39 +0000368 if module not in save_modules and module.startswith("test."):
Guido van Rossum5796d262000-04-21 21:35:06 +0000369 test_support.unload(module)
Jeremy Hylton7a1ea0e2001-10-17 13:45:28 +0000370
371 # The lists won't be sorted if running with -r
372 good.sort()
373 bad.sort()
374 skipped.sort()
Tim Peterse0c446b2001-10-18 21:57:37 +0000375
Guido van Rossum152494a1996-12-20 03:12:20 +0000376 if good and not quiet:
Guido van Rossum41360a41998-03-26 19:42:58 +0000377 if not bad and not skipped and len(good) > 1:
378 print "All",
379 print count(len(good), "test"), "OK."
Tim Peters1a4d77b2000-12-30 22:21:22 +0000380 if verbose:
Barry Warsaw408b6d32002-07-30 23:27:12 +0000381 print "CAUTION: stdout isn't compared in verbose mode:"
382 print "a test that passes in verbose mode may fail without it."
Guido van Rossum152494a1996-12-20 03:12:20 +0000383 if bad:
Tim Petersa45da922001-08-12 03:45:50 +0000384 print count(len(bad), "test"), "failed:"
385 printlist(bad)
Guido van Rossum152494a1996-12-20 03:12:20 +0000386 if skipped and not quiet:
Tim Petersa45da922001-08-12 03:45:50 +0000387 print count(len(skipped), "test"), "skipped:"
388 printlist(skipped)
Barry Warsawe11e3de1999-01-28 19:51:51 +0000389
Tim Petersb5b7b782001-08-12 01:20:39 +0000390 e = _ExpectedSkips()
Tim Petersa2be2d62001-08-12 02:01:09 +0000391 plat = sys.platform
Tim Petersb5b7b782001-08-12 01:20:39 +0000392 if e.isvalid():
Raymond Hettingera690a992003-11-16 16:17:49 +0000393 surprise = set(skipped) - e.getexpected() - set(resource_denieds)
Tim Petersb5b7b782001-08-12 01:20:39 +0000394 if surprise:
395 print count(len(surprise), "skip"), \
Tim Petersa45da922001-08-12 03:45:50 +0000396 "unexpected on", plat + ":"
397 printlist(surprise)
Tim Petersb5b7b782001-08-12 01:20:39 +0000398 else:
399 print "Those skips are all expected on", plat + "."
400 else:
401 print "Ask someone to teach regrtest.py about which tests are"
402 print "expected to get skipped on", plat + "."
403
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000404 if verbose2 and bad:
405 print "Re-running failed tests in verbose mode"
406 for test in bad:
Tim Peters922dd7d2006-03-10 23:37:10 +0000407 print "Re-running test %r in verbose mode" % test
408 sys.stdout.flush()
Martin v. Löwis04824ce2006-03-10 21:26:16 +0000409 try:
410 test_support.verbose = 1
411 ok = runtest(test, generate, 1, quiet, testdir,
412 huntrleaks)
413 except KeyboardInterrupt:
414 # print a newline separate from the ^C
415 print
416 break
417 except:
418 raise
419
Barry Warsawe11e3de1999-01-28 19:51:51 +0000420 if single:
421 alltests = findtests(testdir, stdtests, nottests)
422 for i in range(len(alltests)):
423 if tests[0] == alltests[i]:
424 if i == len(alltests) - 1:
425 os.unlink(filename)
426 else:
427 fp = open(filename, 'w')
428 fp.write(alltests[i+1] + '\n')
429 fp.close()
430 break
431 else:
432 os.unlink(filename)
433
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000434 if trace:
435 r = tracer.results()
436 r.write_results(show_missing=True, summary=True, coverdir=coverdir)
437
Skip Montanaro0179a182004-06-06 15:53:18 +0000438 if runleaks:
439 os.system("leaks %d" % os.getpid())
440
Tim Peters5943b4a2003-07-23 00:30:39 +0000441 sys.exit(len(bad) > 0)
Barry Warsaw08fca522001-08-20 22:33:46 +0000442
Guido van Rossum152494a1996-12-20 03:12:20 +0000443
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000444STDTESTS = [
Guido van Rossum152494a1996-12-20 03:12:20 +0000445 'test_grammar',
446 'test_opcodes',
447 'test_operations',
448 'test_builtin',
449 'test_exceptions',
450 'test_types',
451 ]
452
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000453NOTTESTS = [
Guido van Rossum152494a1996-12-20 03:12:20 +0000454 'test_support',
Jeremy Hylton62e2c7e2001-02-28 17:48:06 +0000455 'test_future1',
456 'test_future2',
Jeremy Hylton8471a352001-08-20 20:33:42 +0000457 'test_future3',
Guido van Rossum152494a1996-12-20 03:12:20 +0000458 ]
459
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000460def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):
Guido van Rossum152494a1996-12-20 03:12:20 +0000461 """Return a list of all applicable test modules."""
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000462 if not testdir: testdir = findtestdir()
Guido van Rossum152494a1996-12-20 03:12:20 +0000463 names = os.listdir(testdir)
464 tests = []
465 for name in names:
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000466 if name[:5] == "test_" and name[-3:] == os.extsep+"py":
Guido van Rossum41360a41998-03-26 19:42:58 +0000467 modname = name[:-3]
468 if modname not in stdtests and modname not in nottests:
469 tests.append(modname)
Guido van Rossum152494a1996-12-20 03:12:20 +0000470 tests.sort()
471 return stdtests + tests
472
Michael W. Hudson61147f62004-08-03 11:33:28 +0000473def runtest(test, generate, verbose, quiet, testdir=None, huntrleaks=False):
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000474 """Run a single test.
475 test -- the name of the test
476 generate -- if true, generate output, instead of running the test
477 and comparing it to a previously created output file
478 verbose -- if true, print more messages
Trent Mickf29f47b2000-08-11 19:02:59 +0000479 quiet -- if true, don't print 'skipped' messages (probably redundant)
Guido van Rossum6fd83b71998-08-01 17:04:08 +0000480 testdir -- test directory
481 """
Guido van Rossum152494a1996-12-20 03:12:20 +0000482 test_support.unload(test)
Barry Warsaw3b6d0252004-02-07 22:43:03 +0000483 if not testdir:
484 testdir = findtestdir()
Guido van Rossum152494a1996-12-20 03:12:20 +0000485 outputdir = os.path.join(testdir, "output")
486 outputfile = os.path.join(outputdir, test)
Tim Peters9390cc12001-09-28 20:14:46 +0000487 if verbose:
Guido van Rossum41360a41998-03-26 19:42:58 +0000488 cfp = None
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000489 else:
Raymond Hettinger74e67662003-05-17 20:44:12 +0000490 cfp = cStringIO.StringIO()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000491 if huntrleaks:
492 refrep = open(huntrleaks[2], "a")
Guido van Rossum152494a1996-12-20 03:12:20 +0000493 try:
Tim Peters342ca752001-09-25 19:13:20 +0000494 save_stdout = sys.stdout
Guido van Rossum41360a41998-03-26 19:42:58 +0000495 try:
496 if cfp:
497 sys.stdout = cfp
498 print test # Output file starts with test name
Barry Warsaw408b6d32002-07-30 23:27:12 +0000499 if test.startswith('test.'):
500 abstest = test
501 else:
502 # Always import it from the test package
503 abstest = 'test.' + test
504 the_package = __import__(abstest, globals(), locals(), [])
505 the_module = getattr(the_package, test)
Tim Petersd9742212001-05-22 18:28:25 +0000506 # Most tests run to completion simply as a side-effect of
507 # being imported. For the benefit of tests that can't run
508 # that way (like test_threaded_import), explicitly invoke
509 # their test_main() function (if it exists).
510 indirect_test = getattr(the_module, "test_main", None)
511 if indirect_test is not None:
512 indirect_test()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000513 if huntrleaks:
514 # This code *is* hackish and inelegant, yes.
515 # But it seems to do the job.
516 import copy_reg
517 fs = warnings.filters[:]
518 ps = copy_reg.dispatch_table.copy()
519 pic = sys.path_importer_cache.copy()
520 import gc
521 def cleanup():
Neal Norwitz545d4962005-11-25 02:55:49 +0000522 import _strptime, linecache, warnings, dircache
Neal Norwitz5102c4e2006-04-06 08:00:20 +0000523 import urlparse, urllib, urllib2, mimetypes
Michael W. Hudson61147f62004-08-03 11:33:28 +0000524 from distutils.dir_util import _path_created
525 _path_created.clear()
526 warnings.filters[:] = fs
527 gc.collect()
Neal Norwitz94a9c092006-03-16 06:30:02 +0000528 re.purge()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000529 _strptime._regex_cache.clear()
530 urlparse.clear_cache()
Neal Norwitz545d4962005-11-25 02:55:49 +0000531 urllib.urlcleanup()
532 urllib2.install_opener(None)
Michael W. Hudson61147f62004-08-03 11:33:28 +0000533 copy_reg.dispatch_table.clear()
534 copy_reg.dispatch_table.update(ps)
535 sys.path_importer_cache.clear()
536 sys.path_importer_cache.update(pic)
537 dircache.reset()
Michael W. Hudson1a728522005-06-14 09:31:28 +0000538 linecache.clearcache()
Neal Norwitz5102c4e2006-04-06 08:00:20 +0000539 mimetypes._default_mime_types()
Michael W. Hudson61147f62004-08-03 11:33:28 +0000540 if indirect_test:
541 def run_the_test():
542 indirect_test()
543 else:
544 def run_the_test():
545 reload(the_module)
546 deltas = []
547 repcount = huntrleaks[0] + huntrleaks[1]
548 print >> sys.stderr, "beginning", repcount, "repetitions"
549 print >> sys.stderr, \
550 ("1234567890"*(repcount//10 + 1))[:repcount]
551 for i in range(repcount):
552 rc = sys.gettotalrefcount()
553 run_the_test()
554 sys.stderr.write('.')
555 cleanup()
556 deltas.append(sys.gettotalrefcount() - rc - 2)
557 print >>sys.stderr
558 if max(map(abs, deltas[-huntrleaks[1]:])) > 0:
Michael W. Hudsone667e662004-08-12 18:27:48 +0000559 print >>sys.stderr, test, 'leaked', \
560 deltas[-huntrleaks[1]:], 'references'
Michael W. Hudson61147f62004-08-03 11:33:28 +0000561 print >>refrep, test, 'leaked', \
562 deltas[-huntrleaks[1]:], 'references'
563 # The end of the huntrleaks hackishness.
Guido van Rossum41360a41998-03-26 19:42:58 +0000564 finally:
Tim Peters342ca752001-09-25 19:13:20 +0000565 sys.stdout = save_stdout
Fred Drake9a0db072003-02-03 15:19:30 +0000566 except test_support.ResourceDenied, msg:
567 if not quiet:
568 print test, "skipped --", msg
569 sys.stdout.flush()
570 return -2
Thomas Wouters3af826e2000-08-04 13:17:51 +0000571 except (ImportError, test_support.TestSkipped), msg:
Trent Mickf29f47b2000-08-11 19:02:59 +0000572 if not quiet:
Fred Drakede4742b2002-10-17 20:36:08 +0000573 print test, "skipped --", msg
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000574 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000575 return -1
Fred Drakefe5c22a2000-08-18 16:04:05 +0000576 except KeyboardInterrupt:
577 raise
Guido van Rossum152494a1996-12-20 03:12:20 +0000578 except test_support.TestFailed, msg:
Guido van Rossum41360a41998-03-26 19:42:58 +0000579 print "test", test, "failed --", msg
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000580 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000581 return 0
Guido van Rossum9e48b271997-07-16 01:56:13 +0000582 except:
Guido van Rossum41360a41998-03-26 19:42:58 +0000583 type, value = sys.exc_info()[:2]
Fred Drake27c4b392000-08-23 20:34:40 +0000584 print "test", test, "crashed --", str(type) + ":", value
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000585 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000586 if verbose:
587 traceback.print_exc(file=sys.stdout)
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000588 sys.stdout.flush()
Guido van Rossum41360a41998-03-26 19:42:58 +0000589 return 0
Guido van Rossum152494a1996-12-20 03:12:20 +0000590 else:
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000591 if not cfp:
592 return 1
593 output = cfp.getvalue()
Fred Drakee51fe8d2001-05-29 17:10:51 +0000594 if generate:
Fred Drakee51fe8d2001-05-29 17:10:51 +0000595 if output == test + "\n":
596 if os.path.exists(outputfile):
597 # Write it since it already exists (and the contents
598 # may have changed), but let the user know it isn't
599 # needed:
Fred Drakee51fe8d2001-05-29 17:10:51 +0000600 print "output file", outputfile, \
601 "is no longer needed; consider removing it"
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000602 else:
603 # We don't need it, so don't create it.
604 return 1
605 fp = open(outputfile, "w")
606 fp.write(output)
607 fp.close()
608 return 1
609 if os.path.exists(outputfile):
610 fp = open(outputfile, "r")
611 expected = fp.read()
612 fp.close()
613 else:
614 expected = test + "\n"
Michael W. Hudson61147f62004-08-03 11:33:28 +0000615 if output == expected or huntrleaks:
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000616 return 1
617 print "test", test, "produced unexpected output:"
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000618 sys.stdout.flush()
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000619 reportdiff(expected, output)
Guido van Rossum3cda93e2002-09-13 21:28:03 +0000620 sys.stdout.flush()
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000621 return 0
622
623def reportdiff(expected, output):
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000624 import difflib
Tim Petersc377b162001-09-22 05:31:03 +0000625 print "*" * 70
626 a = expected.splitlines(1)
627 b = output.splitlines(1)
Guido van Rossumcf691932001-09-21 21:06:22 +0000628 sm = difflib.SequenceMatcher(a=a, b=b)
629 tuples = sm.get_opcodes()
Tim Petersc377b162001-09-22 05:31:03 +0000630
Guido van Rossumcf691932001-09-21 21:06:22 +0000631 def pair(x0, x1):
Tim Petersc377b162001-09-22 05:31:03 +0000632 # x0:x1 are 0-based slice indices; convert to 1-based line indices.
Guido van Rossumcf691932001-09-21 21:06:22 +0000633 x0 += 1
634 if x0 >= x1:
Tim Petersc377b162001-09-22 05:31:03 +0000635 return "line " + str(x0)
Guido van Rossumcf691932001-09-21 21:06:22 +0000636 else:
Tim Petersc377b162001-09-22 05:31:03 +0000637 return "lines %d-%d" % (x0, x1)
638
Guido van Rossumcf691932001-09-21 21:06:22 +0000639 for op, a0, a1, b0, b1 in tuples:
640 if op == 'equal':
641 pass
Tim Petersc377b162001-09-22 05:31:03 +0000642
Guido van Rossumcf691932001-09-21 21:06:22 +0000643 elif op == 'delete':
Tim Petersc377b162001-09-22 05:31:03 +0000644 print "***", pair(a0, a1), "of expected output missing:"
Guido van Rossumcf691932001-09-21 21:06:22 +0000645 for line in a[a0:a1]:
Tim Petersc377b162001-09-22 05:31:03 +0000646 print "-", line,
647
Guido van Rossumcf691932001-09-21 21:06:22 +0000648 elif op == 'replace':
Tim Petersc377b162001-09-22 05:31:03 +0000649 print "*** mismatch between", pair(a0, a1), "of expected", \
650 "output and", pair(b0, b1), "of actual output:"
651 for line in difflib.ndiff(a[a0:a1], b[b0:b1]):
652 print line,
653
Guido van Rossumcf691932001-09-21 21:06:22 +0000654 elif op == 'insert':
Tim Petersc377b162001-09-22 05:31:03 +0000655 print "***", pair(b0, b1), "of actual output doesn't appear", \
656 "in expected output after line", str(a1)+":"
Guido van Rossumcf691932001-09-21 21:06:22 +0000657 for line in b[b0:b1]:
Tim Petersc377b162001-09-22 05:31:03 +0000658 print "+", line,
659
Guido van Rossumcf691932001-09-21 21:06:22 +0000660 else:
661 print "get_opcodes() returned bad tuple?!?!", (op, a0, a1, b0, b1)
Tim Petersc377b162001-09-22 05:31:03 +0000662
Guido van Rossum0fcca4e2001-09-21 20:31:52 +0000663 print "*" * 70
Guido van Rossum152494a1996-12-20 03:12:20 +0000664
665def findtestdir():
666 if __name__ == '__main__':
Guido van Rossum41360a41998-03-26 19:42:58 +0000667 file = sys.argv[0]
Guido van Rossum152494a1996-12-20 03:12:20 +0000668 else:
Guido van Rossum41360a41998-03-26 19:42:58 +0000669 file = __file__
Guido van Rossum152494a1996-12-20 03:12:20 +0000670 testdir = os.path.dirname(file) or os.curdir
671 return testdir
672
Tim Petersc5000df2002-06-02 21:42:01 +0000673def removepy(name):
674 if name.endswith(os.extsep + "py"):
675 name = name[:-3]
676 return name
677
Guido van Rossum152494a1996-12-20 03:12:20 +0000678def count(n, word):
679 if n == 1:
Guido van Rossum41360a41998-03-26 19:42:58 +0000680 return "%d %s" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +0000681 else:
Guido van Rossum41360a41998-03-26 19:42:58 +0000682 return "%d %ss" % (n, word)
Guido van Rossum152494a1996-12-20 03:12:20 +0000683
Tim Petersa45da922001-08-12 03:45:50 +0000684def printlist(x, width=70, indent=4):
Tim Peters7c7efe92002-08-23 17:55:54 +0000685 """Print the elements of iterable x to stdout.
Tim Petersa45da922001-08-12 03:45:50 +0000686
687 Optional arg width (default 70) is the maximum line length.
688 Optional arg indent (default 4) is the number of blanks with which to
689 begin each line.
690 """
691
Tim Petersba78bc42002-07-04 19:45:06 +0000692 from textwrap import fill
693 blanks = ' ' * indent
694 print fill(' '.join(map(str, x)), width,
695 initial_indent=blanks, subsequent_indent=blanks)
Tim Petersa45da922001-08-12 03:45:50 +0000696
Tim Petersde14a302002-04-01 05:04:46 +0000697# Map sys.platform to a string containing the basenames of tests
698# expected to be skipped on that platform.
Tim Peters2a182db2002-10-09 01:07:11 +0000699#
700# Special cases:
701# test_pep277
702# The _ExpectedSkips constructor adds this to the set of expected
703# skips if not os.path.supports_unicode_filenames.
Tim Petersb4ee4eb2002-12-04 03:26:57 +0000704# test_socket_ssl
705# Controlled by test_socket_ssl.skip_expected. Requires the network
706# resource, and a socket module with ssl support.
Neal Norwitz55b61d22003-02-28 19:57:03 +0000707# test_timeout
708# Controlled by test_timeout.skip_expected. Requires the network
709# resource and a socket module.
Tim Petersde14a302002-04-01 05:04:46 +0000710
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000711_expectations = {
712 'win32':
713 """
Tim Petersc7c516a2003-09-20 22:06:13 +0000714 test__locale
Raymond Hettinger901dc982003-11-20 19:02:02 +0000715 test_applesingle
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000716 test_al
Skip Montanaro823ba282003-05-06 20:36:24 +0000717 test_bsddb185
Tim Peters78e35f92002-11-22 20:00:34 +0000718 test_bsddb3
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000719 test_cd
720 test_cl
721 test_commands
722 test_crypt
Tim Petersd7030572001-10-22 22:06:08 +0000723 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000724 test_dbm
725 test_dl
726 test_fcntl
727 test_fork1
728 test_gdbm
729 test_gl
730 test_grp
731 test_imgfile
Tim Petersfd8e6e52003-03-04 00:26:38 +0000732 test_ioctl
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000733 test_largefile
734 test_linuxaudiodev
735 test_mhlib
736 test_nis
737 test_openpty
Tim Petersefc4b122002-12-10 18:47:56 +0000738 test_ossaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000739 test_poll
Tim Peters003eb302003-02-17 21:48:48 +0000740 test_posix
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000741 test_pty
742 test_pwd
Tim Peters1e33ffa2002-04-23 23:09:02 +0000743 test_resource
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000744 test_signal
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000745 test_sunaudiodev
Tim Peterscea2cc42004-08-04 02:32:03 +0000746 test_threadsignals
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000747 test_timing
Tim Petersb84de022006-03-22 02:58:17 +0000748 test_wait3
749 test_wait4
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000750 """,
751 'linux2':
752 """
753 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000754 test_applesingle
Skip Montanaro823ba282003-05-06 20:36:24 +0000755 test_bsddb185
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000756 test_cd
757 test_cl
Guido van Rossumf66dacd2001-10-23 15:10:55 +0000758 test_curses
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000759 test_dl
760 test_gl
761 test_imgfile
762 test_largefile
Guido van Rossum4507ec72003-02-14 19:29:22 +0000763 test_linuxaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000764 test_nis
765 test_ntpath
Guido van Rossum4507ec72003-02-14 19:29:22 +0000766 test_ossaudiodev
Anthony Baxterc51ee692006-04-01 00:57:31 +0000767 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +0000768 test_startfile
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000769 test_sunaudiodev
Guido van Rossumf73e30c2001-08-12 02:22:19 +0000770 """,
Jack Jansen49a806e2001-08-28 14:49:00 +0000771 'mac':
Guido van Rossumaa782362001-09-02 03:58:41 +0000772 """
773 test_al
Jack Jansen67975142003-01-08 16:31:11 +0000774 test_atexit
Guido van Rossumaa782362001-09-02 03:58:41 +0000775 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000776 test_bsddb185
Jack Jansen67975142003-01-08 16:31:11 +0000777 test_bsddb3
778 test_bz2
Guido van Rossumaa782362001-09-02 03:58:41 +0000779 test_cd
780 test_cl
781 test_commands
782 test_crypt
Jack Jansenb3be2162001-11-30 14:16:36 +0000783 test_curses
Guido van Rossumaa782362001-09-02 03:58:41 +0000784 test_dbm
785 test_dl
786 test_fcntl
787 test_fork1
788 test_gl
789 test_grp
Jack Jansenc4d6bdd2003-03-07 15:38:11 +0000790 test_ioctl
Guido van Rossumaa782362001-09-02 03:58:41 +0000791 test_imgfile
792 test_largefile
793 test_linuxaudiodev
794 test_locale
795 test_mmap
796 test_nis
797 test_ntpath
798 test_openpty
Jack Jansen67975142003-01-08 16:31:11 +0000799 test_ossaudiodev
Guido van Rossumaa782362001-09-02 03:58:41 +0000800 test_poll
Jack Jansen67975142003-01-08 16:31:11 +0000801 test_popen
Guido van Rossumaa782362001-09-02 03:58:41 +0000802 test_popen2
Jack Jansen5bb97e62003-02-21 22:33:55 +0000803 test_posix
Guido van Rossumaa782362001-09-02 03:58:41 +0000804 test_pty
805 test_pwd
Jack Jansen67975142003-01-08 16:31:11 +0000806 test_resource
Guido van Rossumaa782362001-09-02 03:58:41 +0000807 test_signal
Anthony Baxterc51ee692006-04-01 00:57:31 +0000808 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +0000809 test_startfile
Guido van Rossumaa782362001-09-02 03:58:41 +0000810 test_sunaudiodev
811 test_sundry
Jack Jansenc4d6bdd2003-03-07 15:38:11 +0000812 test_tarfile
Guido van Rossumaa782362001-09-02 03:58:41 +0000813 test_timing
Guido van Rossumaa782362001-09-02 03:58:41 +0000814 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000815 'unixware7':
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000816 """
817 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000818 test_applesingle
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000819 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000820 test_bsddb185
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000821 test_cd
822 test_cl
823 test_dl
824 test_gl
825 test_imgfile
826 test_largefile
827 test_linuxaudiodev
828 test_minidom
829 test_nis
830 test_ntpath
831 test_openpty
832 test_pyexpat
833 test_sax
Anthony Baxter22495c02006-04-05 13:24:26 +0000834 test_startfile
Anthony Baxterc51ee692006-04-01 00:57:31 +0000835 test_sqlite
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000836 test_sunaudiodev
837 test_sundry
Martin v. Löwis0ace3262001-09-05 14:38:48 +0000838 """,
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000839 'openunix8':
840 """
841 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000842 test_applesingle
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000843 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000844 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000845 test_cd
846 test_cl
847 test_dl
848 test_gl
849 test_imgfile
850 test_largefile
851 test_linuxaudiodev
852 test_minidom
853 test_nis
854 test_ntpath
855 test_openpty
856 test_pyexpat
857 test_sax
Anthony Baxterc51ee692006-04-01 00:57:31 +0000858 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +0000859 test_startfile
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000860 test_sunaudiodev
861 test_sundry
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000862 """,
863 'sco_sv3':
864 """
865 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000866 test_applesingle
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000867 test_asynchat
868 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000869 test_bsddb185
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000870 test_cd
871 test_cl
872 test_dl
873 test_fork1
874 test_gettext
875 test_gl
876 test_imgfile
877 test_largefile
878 test_linuxaudiodev
879 test_locale
880 test_minidom
881 test_nis
882 test_ntpath
883 test_openpty
884 test_pyexpat
885 test_queue
886 test_sax
Anthony Baxterc51ee692006-04-01 00:57:31 +0000887 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +0000888 test_startfile
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000889 test_sunaudiodev
890 test_sundry
891 test_thread
892 test_threaded_import
893 test_threadedtempfile
894 test_threading
Martin v. Löwis21ee4092002-09-30 16:19:48 +0000895 """,
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000896 'riscos':
897 """
898 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000899 test_applesingle
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000900 test_asynchat
Martin v. Löwisa94568a2003-05-10 07:36:56 +0000901 test_atexit
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000902 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000903 test_bsddb185
Martin v. Löwisa94568a2003-05-10 07:36:56 +0000904 test_bsddb3
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000905 test_cd
906 test_cl
907 test_commands
908 test_crypt
909 test_dbm
910 test_dl
911 test_fcntl
912 test_fork1
913 test_gdbm
914 test_gl
915 test_grp
916 test_imgfile
917 test_largefile
918 test_linuxaudiodev
919 test_locale
920 test_mmap
921 test_nis
922 test_ntpath
923 test_openpty
924 test_poll
925 test_popen2
926 test_pty
927 test_pwd
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000928 test_strop
Anthony Baxterc51ee692006-04-01 00:57:31 +0000929 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +0000930 test_startfile
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000931 test_sunaudiodev
932 test_sundry
933 test_thread
934 test_threaded_import
935 test_threadedtempfile
936 test_threading
937 test_timing
Guido van Rossume2ae77b2001-10-24 20:42:55 +0000938 """,
Jack Jansen8a97f4a2001-12-05 23:27:32 +0000939 'darwin':
Jack Jansen398c2362001-12-02 21:41:36 +0000940 """
Brett Cannon2bfb94c2003-10-13 04:27:47 +0000941 test__locale
Jack Jansen398c2362001-12-02 21:41:36 +0000942 test_al
Jack Jansenacda3392002-12-30 23:03:13 +0000943 test_bsddb
Guido van Rossum9d427002002-12-03 10:24:56 +0000944 test_bsddb3
Jack Jansen398c2362001-12-02 21:41:36 +0000945 test_cd
946 test_cl
947 test_curses
948 test_dl
949 test_gdbm
950 test_gl
951 test_imgfile
952 test_largefile
953 test_linuxaudiodev
Jack Jansenacda3392002-12-30 23:03:13 +0000954 test_locale
Jack Jansen398c2362001-12-02 21:41:36 +0000955 test_minidom
956 test_nis
957 test_ntpath
Jack Jansenacda3392002-12-30 23:03:13 +0000958 test_ossaudiodev
Jack Jansen398c2362001-12-02 21:41:36 +0000959 test_poll
Anthony Baxterc51ee692006-04-01 00:57:31 +0000960 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +0000961 test_startfile
Jack Jansen398c2362001-12-02 21:41:36 +0000962 test_sunaudiodev
Jack Jansen398c2362001-12-02 21:41:36 +0000963 """,
Guido van Rossum11c3f092002-07-17 15:08:24 +0000964 'sunos5':
965 """
966 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000967 test_applesingle
Guido van Rossum11c3f092002-07-17 15:08:24 +0000968 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000969 test_bsddb185
Guido van Rossum11c3f092002-07-17 15:08:24 +0000970 test_cd
971 test_cl
972 test_curses
973 test_dbm
Guido van Rossum11c3f092002-07-17 15:08:24 +0000974 test_gdbm
975 test_gl
976 test_gzip
977 test_imgfile
978 test_linuxaudiodev
Guido van Rossum11c3f092002-07-17 15:08:24 +0000979 test_openpty
Anthony Baxterc51ee692006-04-01 00:57:31 +0000980 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +0000981 test_startfile
Guido van Rossum11c3f092002-07-17 15:08:24 +0000982 test_zipfile
983 test_zlib
Jeremy Hyltoned375e12002-07-17 15:56:55 +0000984 """,
Skip Montanarob3230212002-03-15 02:54:03 +0000985 'hp-ux11':
986 """
987 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +0000988 test_applesingle
Skip Montanarob3230212002-03-15 02:54:03 +0000989 test_bsddb
Skip Montanaro823ba282003-05-06 20:36:24 +0000990 test_bsddb185
Skip Montanarob3230212002-03-15 02:54:03 +0000991 test_cd
992 test_cl
993 test_curses
994 test_dl
995 test_gdbm
996 test_gl
997 test_gzip
998 test_imgfile
999 test_largefile
1000 test_linuxaudiodev
1001 test_locale
1002 test_minidom
1003 test_nis
1004 test_ntpath
1005 test_openpty
1006 test_pyexpat
1007 test_sax
Anthony Baxterc51ee692006-04-01 00:57:31 +00001008 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +00001009 test_startfile
Skip Montanarob3230212002-03-15 02:54:03 +00001010 test_sunaudiodev
Skip Montanarob3230212002-03-15 02:54:03 +00001011 test_zipfile
1012 test_zlib
1013 """,
Martin v. Löwisf90ae202002-06-11 06:22:31 +00001014 'atheos':
Tim Petersc411dba2002-07-16 21:35:23 +00001015 """
1016 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +00001017 test_applesingle
Skip Montanaro823ba282003-05-06 20:36:24 +00001018 test_bsddb185
Tim Petersc411dba2002-07-16 21:35:23 +00001019 test_cd
1020 test_cl
1021 test_curses
1022 test_dl
Tim Petersc411dba2002-07-16 21:35:23 +00001023 test_gdbm
1024 test_gl
1025 test_imgfile
1026 test_largefile
1027 test_linuxaudiodev
1028 test_locale
1029 test_mhlib
1030 test_mmap
Tim Petersc411dba2002-07-16 21:35:23 +00001031 test_nis
1032 test_poll
1033 test_popen2
1034 test_resource
Anthony Baxterc51ee692006-04-01 00:57:31 +00001035 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +00001036 test_startfile
Tim Petersc411dba2002-07-16 21:35:23 +00001037 test_sunaudiodev
Tim Petersc411dba2002-07-16 21:35:23 +00001038 """,
Jason Tishler25115942002-12-05 15:18:15 +00001039 'cygwin':
1040 """
1041 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +00001042 test_applesingle
Skip Montanaro823ba282003-05-06 20:36:24 +00001043 test_bsddb185
Tim Petersb0f89e02002-12-05 17:20:25 +00001044 test_bsddb3
Jason Tishler25115942002-12-05 15:18:15 +00001045 test_cd
1046 test_cl
1047 test_curses
1048 test_dbm
Jason Tishler25115942002-12-05 15:18:15 +00001049 test_gl
1050 test_imgfile
Jason Tishlerc23f39c2003-07-22 18:35:58 +00001051 test_ioctl
Jason Tishler25115942002-12-05 15:18:15 +00001052 test_largefile
1053 test_linuxaudiodev
1054 test_locale
Jason Tishler25115942002-12-05 15:18:15 +00001055 test_nis
Jason Tishler5c4ded22003-02-05 16:46:01 +00001056 test_ossaudiodev
Jason Tishler25115942002-12-05 15:18:15 +00001057 test_socketserver
Anthony Baxterc51ee692006-04-01 00:57:31 +00001058 test_sqlite
Jason Tishler25115942002-12-05 15:18:15 +00001059 test_sunaudiodev
Jason Tishler25115942002-12-05 15:18:15 +00001060 """,
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001061 'os2emx':
1062 """
1063 test_al
Guido van Rossum944a6c32003-11-20 22:11:29 +00001064 test_applesingle
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001065 test_audioop
Skip Montanaro823ba282003-05-06 20:36:24 +00001066 test_bsddb185
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001067 test_bsddb3
1068 test_cd
1069 test_cl
1070 test_commands
1071 test_curses
1072 test_dl
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001073 test_gl
1074 test_imgfile
1075 test_largefile
1076 test_linuxaudiodev
1077 test_mhlib
1078 test_mmap
1079 test_nis
1080 test_openpty
1081 test_ossaudiodev
1082 test_pty
1083 test_resource
1084 test_signal
Anthony Baxterc51ee692006-04-01 00:57:31 +00001085 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +00001086 test_startfile
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001087 test_sunaudiodev
Andrew MacIntyrefd07e7d2002-12-31 11:26:50 +00001088 """,
Guido van Rossum944a6c32003-11-20 22:11:29 +00001089 'freebsd4':
1090 """
1091 test_aepack
1092 test_al
1093 test_applesingle
1094 test_bsddb
1095 test_bsddb3
1096 test_cd
1097 test_cl
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001098 test_gdbm
Guido van Rossum944a6c32003-11-20 22:11:29 +00001099 test_gl
1100 test_imgfile
1101 test_linuxaudiodev
1102 test_locale
1103 test_macfs
1104 test_macostools
1105 test_nis
Guido van Rossum944a6c32003-11-20 22:11:29 +00001106 test_ossaudiodev
1107 test_pep277
1108 test_plistlib
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001109 test_pty
Guido van Rossum944a6c32003-11-20 22:11:29 +00001110 test_scriptpackages
1111 test_socket_ssl
1112 test_socketserver
Anthony Baxterc51ee692006-04-01 00:57:31 +00001113 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +00001114 test_startfile
Guido van Rossum944a6c32003-11-20 22:11:29 +00001115 test_sunaudiodev
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001116 test_tcl
Guido van Rossum944a6c32003-11-20 22:11:29 +00001117 test_timeout
1118 test_unicode_file
1119 test_urllibnet
1120 test_winreg
1121 test_winsound
Martin v. Löwis56f88112003-06-07 20:01:37 +00001122 """,
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001123 'aix5':
1124 """
1125 test_aepack
1126 test_al
1127 test_applesingle
1128 test_bsddb
1129 test_bsddb185
1130 test_bsddb3
1131 test_bz2
1132 test_cd
1133 test_cl
1134 test_dl
1135 test_gdbm
1136 test_gl
1137 test_gzip
1138 test_imgfile
1139 test_linuxaudiodev
1140 test_macfs
1141 test_macostools
1142 test_nis
1143 test_ossaudiodev
Anthony Baxterc51ee692006-04-01 00:57:31 +00001144 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +00001145 test_startfile
Guido van Rossum8ee3e5a2005-09-14 18:09:42 +00001146 test_sunaudiodev
1147 test_tcl
1148 test_winreg
1149 test_winsound
1150 test_zipimport
1151 test_zlib
1152 """,
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001153 'openbsd3':
1154 """
1155 test_aepack
1156 test_al
1157 test_applesingle
1158 test_bsddb
1159 test_bsddb3
1160 test_cd
1161 test_cl
1162 test_ctypes
1163 test_dl
1164 test_gdbm
1165 test_gl
1166 test_imgfile
1167 test_linuxaudiodev
1168 test_locale
1169 test_macfs
1170 test_macostools
1171 test_nis
1172 test_normalization
1173 test_ossaudiodev
1174 test_pep277
1175 test_plistlib
1176 test_scriptpackages
1177 test_tcl
Anthony Baxterc51ee692006-04-01 00:57:31 +00001178 test_sqlite
Anthony Baxter22495c02006-04-05 13:24:26 +00001179 test_startfile
Martin v. Löwis6da56f92006-03-26 10:02:34 +00001180 test_sunaudiodev
1181 test_unicode_file
1182 test_winreg
1183 test_winsound
1184 """,
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001185}
Martin v. Löwis32d0c1b2004-07-26 12:09:13 +00001186_expectations['freebsd5'] = _expectations['freebsd4']
Hye-Shik Changf64700a2004-08-18 15:13:41 +00001187_expectations['freebsd6'] = _expectations['freebsd4']
Hye-Shik Chang4e422812005-07-17 02:36:59 +00001188_expectations['freebsd7'] = _expectations['freebsd4']
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001189
Tim Petersb5b7b782001-08-12 01:20:39 +00001190class _ExpectedSkips:
1191 def __init__(self):
Tim Peters2a182db2002-10-09 01:07:11 +00001192 import os.path
Tim Petersb4ee4eb2002-12-04 03:26:57 +00001193 from test import test_socket_ssl
Neal Norwitz55b61d22003-02-28 19:57:03 +00001194 from test import test_timeout
Tim Peters1b445d32002-11-24 18:53:11 +00001195
Tim Peters7c7efe92002-08-23 17:55:54 +00001196 self.valid = False
Tim Petersde14a302002-04-01 05:04:46 +00001197 if sys.platform in _expectations:
Guido van Rossumf73e30c2001-08-12 02:22:19 +00001198 s = _expectations[sys.platform]
Raymond Hettingera690a992003-11-16 16:17:49 +00001199 self.expected = set(s.split())
Tim Peters1b445d32002-11-24 18:53:11 +00001200
Tim Peters2a182db2002-10-09 01:07:11 +00001201 if not os.path.supports_unicode_filenames:
1202 self.expected.add('test_pep277')
Tim Peters1b445d32002-11-24 18:53:11 +00001203
Tim Petersb4ee4eb2002-12-04 03:26:57 +00001204 if test_socket_ssl.skip_expected:
1205 self.expected.add('test_socket_ssl')
1206
Neal Norwitz55b61d22003-02-28 19:57:03 +00001207 if test_timeout.skip_expected:
1208 self.expected.add('test_timeout')
1209
Martin v. Löwisfba73692004-11-13 11:13:35 +00001210 if sys.maxint == 9223372036854775807L:
1211 self.expected.add('test_rgbimg')
1212 self.expected.add('test_imageop')
1213
Jack Jansen6afc5e02003-01-29 16:24:16 +00001214 if not sys.platform in ("mac", "darwin"):
Neal Norwitz7035c982003-03-29 22:01:17 +00001215 MAC_ONLY = ["test_macostools", "test_macfs", "test_aepack",
1216 "test_plistlib", "test_scriptpackages"]
1217 for skip in MAC_ONLY:
1218 self.expected.add(skip)
Tim Petersecd79eb2003-01-29 00:35:32 +00001219
1220 if sys.platform != "win32":
Neal Norwitz7035c982003-03-29 22:01:17 +00001221 WIN_ONLY = ["test_unicode_file", "test_winreg",
1222 "test_winsound"]
1223 for skip in WIN_ONLY:
1224 self.expected.add(skip)
Tim Petersf2715e02003-02-19 02:35:07 +00001225
Tim Peters7c7efe92002-08-23 17:55:54 +00001226 self.valid = True
Tim Petersb5b7b782001-08-12 01:20:39 +00001227
1228 def isvalid(self):
1229 "Return true iff _ExpectedSkips knows about the current platform."
1230 return self.valid
1231
1232 def getexpected(self):
1233 """Return set of test names we expect to skip on current platform.
1234
1235 self.isvalid() must be true.
1236 """
1237
1238 assert self.isvalid()
1239 return self.expected
1240
Guido van Rossum152494a1996-12-20 03:12:20 +00001241if __name__ == '__main__':
Barry Warsaw408b6d32002-07-30 23:27:12 +00001242 # Remove regrtest.py's own directory from the module search path. This
1243 # prevents relative imports from working, and relative imports will screw
1244 # up the testing framework. E.g. if both test.test_support and
1245 # test_support are imported, they will not contain the same globals, and
1246 # much of the testing framework relies on the globals in the
1247 # test.test_support module.
1248 mydir = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0])))
1249 i = pathlen = len(sys.path)
1250 while i >= 0:
1251 i -= 1
1252 if os.path.abspath(os.path.normpath(sys.path[i])) == mydir:
1253 del sys.path[i]
1254 if len(sys.path) == pathlen:
1255 print 'Could not find %r in sys.path to remove it' % mydir
Barry Warsaw08fca522001-08-20 22:33:46 +00001256 main()