Merge the trunk changes in. Breaks socket.ssl for now.
Merged revisions 57392-57619 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r57395 | georg.brandl | 2007-08-24 19:23:23 +0200 (Fri, 24 Aug 2007) | 2 lines
Bug #1011: fix rfc822.Message.getheader docs.
........
r57397 | georg.brandl | 2007-08-24 19:38:49 +0200 (Fri, 24 Aug 2007) | 2 lines
Patch #1006: port test_winreg to unittest.
........
r57398 | georg.brandl | 2007-08-24 19:46:54 +0200 (Fri, 24 Aug 2007) | 2 lines
Fix #1012: wrong URL to :mod:`site` in install/index.rst.
........
r57399 | georg.brandl | 2007-08-24 20:07:52 +0200 (Fri, 24 Aug 2007) | 2 lines
Patch #1008: port test_signal to unittest.
........
r57400 | georg.brandl | 2007-08-24 20:22:54 +0200 (Fri, 24 Aug 2007) | 2 lines
Port test_frozen to unittest.
........
r57401 | georg.brandl | 2007-08-24 20:27:43 +0200 (Fri, 24 Aug 2007) | 2 lines
Document new utility functions in test_support.
........
r57402 | georg.brandl | 2007-08-24 20:30:06 +0200 (Fri, 24 Aug 2007) | 2 lines
Remove test_rgbimg output file, there is no test_rgbimg.py.
........
r57403 | georg.brandl | 2007-08-24 20:35:27 +0200 (Fri, 24 Aug 2007) | 2 lines
Remove output file for test_ossaudiodev, also properly close the dsp object.
........
r57404 | georg.brandl | 2007-08-24 20:46:27 +0200 (Fri, 24 Aug 2007) | 2 lines
Convert test_linuxaudiodev to unittest. Fix a wrong finally clause in test_ossaudiodev.
........
r57406 | collin.winter | 2007-08-24 21:13:58 +0200 (Fri, 24 Aug 2007) | 1 line
Convert test_pkg to use unittest.
........
r57408 | georg.brandl | 2007-08-24 21:22:34 +0200 (Fri, 24 Aug 2007) | 2 lines
Catch the correct errors.
........
r57409 | georg.brandl | 2007-08-24 21:33:53 +0200 (Fri, 24 Aug 2007) | 2 lines
Port test_class to unittest. Patch #1671298.
........
r57415 | collin.winter | 2007-08-24 23:09:42 +0200 (Fri, 24 Aug 2007) | 1 line
Make test_structmembers pass when run with regrtests's -R flag.
........
r57455 | nick.coghlan | 2007-08-25 06:32:07 +0200 (Sat, 25 Aug 2007) | 1 line
Revert misguided attempt at fixing incompatibility between -m and -i switches (better fix coming soon)
........
r57456 | nick.coghlan | 2007-08-25 06:35:54 +0200 (Sat, 25 Aug 2007) | 1 line
Revert compile.c changes that shouldn't have been included in previous checkin
........
r57461 | nick.coghlan | 2007-08-25 12:50:41 +0200 (Sat, 25 Aug 2007) | 1 line
Fix bug 1764407 - the -i switch now does the right thing when using the -m switch
........
r57464 | guido.van.rossum | 2007-08-25 17:08:43 +0200 (Sat, 25 Aug 2007) | 4 lines
Server-side SSL and certificate validation, by Bill Janssen.
While cleaning up Bill's C style, I may have cleaned up some code
he didn't touch as well (in _ssl.c).
........
r57465 | neal.norwitz | 2007-08-25 18:41:36 +0200 (Sat, 25 Aug 2007) | 3 lines
Try to get this to build with Visual Studio by moving all the variable
declarations to the beginning of a scope.
........
r57466 | neal.norwitz | 2007-08-25 18:54:38 +0200 (Sat, 25 Aug 2007) | 1 line
Fix test so it is skipped properly if there is no SSL support.
........
r57467 | neal.norwitz | 2007-08-25 18:58:09 +0200 (Sat, 25 Aug 2007) | 2 lines
Fix a few more variables to try to get this to compile with Visual Studio.
........
r57473 | neal.norwitz | 2007-08-25 19:25:17 +0200 (Sat, 25 Aug 2007) | 1 line
Try to get this test to pass for systems that do not have SO_REUSEPORT
........
r57482 | gregory.p.smith | 2007-08-26 02:26:00 +0200 (Sun, 26 Aug 2007) | 7 lines
keep setup.py from listing unneeded hash modules (_md5, _sha*) as
missing when they were not built because _hashlib with openssl provided
their functionality instead.
don't build bsddb185 if bsddb was built.
........
r57483 | neal.norwitz | 2007-08-26 03:08:16 +0200 (Sun, 26 Aug 2007) | 1 line
Fix typo in docstring (missing c in reacquire)
........
r57484 | neal.norwitz | 2007-08-26 03:42:03 +0200 (Sun, 26 Aug 2007) | 2 lines
Spell check (also americanify behaviour, it's almost 3 times as common)
........
r57503 | neal.norwitz | 2007-08-26 08:29:57 +0200 (Sun, 26 Aug 2007) | 4 lines
Reap children before the test starts so hopefully SocketServer
won't find any old children left around which causes an exception
in collect_children() and the test to fail.
........
r57510 | neal.norwitz | 2007-08-26 20:50:39 +0200 (Sun, 26 Aug 2007) | 1 line
Fail gracefully if the cert files cannot be created
........
r57513 | guido.van.rossum | 2007-08-26 21:35:09 +0200 (Sun, 26 Aug 2007) | 4 lines
Bill Janssen wrote:
Here's a patch which makes test_ssl a better player in the buildbots
environment. I deep-ended on "try-except-else" clauses.
........
r57518 | neal.norwitz | 2007-08-26 23:40:16 +0200 (Sun, 26 Aug 2007) | 1 line
Get the test passing by commenting out some writes (should they be removed?)
........
r57522 | neal.norwitz | 2007-08-27 00:16:23 +0200 (Mon, 27 Aug 2007) | 3 lines
Catch IOError for when the device file doesn't exist or the user doesn't have
permission to write to the device.
........
r57524 | neal.norwitz | 2007-08-27 00:20:03 +0200 (Mon, 27 Aug 2007) | 5 lines
Another patch from Bill Janssen that:
1) Fixes the bug that two class names are initial-lower-case.
2) Replaces the poll waiting for the server to become ready with
a threading.Event signal.
........
r57536 | neal.norwitz | 2007-08-27 02:58:33 +0200 (Mon, 27 Aug 2007) | 1 line
Stop using string.join (from the module) to ease upgrade to py3k
........
r57537 | neal.norwitz | 2007-08-27 03:03:18 +0200 (Mon, 27 Aug 2007) | 1 line
Make a utility function for handling (printing) an error
........
r57538 | neal.norwitz | 2007-08-27 03:15:33 +0200 (Mon, 27 Aug 2007) | 4 lines
If we can't create a certificate, print a warning, but don't fail the test.
Modified patch from what Bill Janssen sent on python-3000.
........
r57539 | facundo.batista | 2007-08-27 03:15:34 +0200 (Mon, 27 Aug 2007) | 7 lines
Ignore test failures caused by 'resource temporarily unavailable'
exceptions raised in the test server thread, since SimpleXMLRPCServer
does not gracefully handle them. Changed number of requests handled
by tests server thread to one (was 2) because no tests require more
than one request. [GSoC - Alan McIntyre]
........
r57561 | guido.van.rossum | 2007-08-27 19:19:42 +0200 (Mon, 27 Aug 2007) | 8 lines
> Regardless, building a fixed test certificate and checking it in sounds like
> the better option. Then the openssl command in the test code can be turned
> into a comment describing how the test data was pregenerated.
Here's a patch that does that.
Bill
........
r57568 | guido.van.rossum | 2007-08-27 20:42:23 +0200 (Mon, 27 Aug 2007) | 26 lines
> Some of the code sets the error string in this directly before
> returning NULL, and other pieces of the code call PySSL_SetError,
> which creates the error string. I think some of the places which set
> the string directly probably shouldn't; instead, they should call
> PySSL_SetError to cons up the error name directly from the err code.
> However, PySSL_SetError only works after the construction of an ssl
> object, which means it can't be used there... I'll take a longer look
> at it and see if there's a reasonable fix.
Here's a patch which addresses this. It also fixes the indentation in
PySSL_SetError, bringing it into line with PEP 7, fixes a compile warning
about one of the OpenSSL macros, and makes the namespace a bit more
consistent. I've tested it on FC 7 and OS X 10.4.
% ./python ./Lib/test/regrtest.py -R :1: -u all test_ssl
test_ssl
beginning 6 repetitions
123456
......
1 test OK.
[29244 refs]
%
[GvR: slightly edited to enforce 79-char line length, even if it required
violating the style guide.]
........
r57570 | guido.van.rossum | 2007-08-27 21:11:11 +0200 (Mon, 27 Aug 2007) | 2 lines
Patch 10124 by Bill Janssen, docs for the new ssl code.
........
r57574 | guido.van.rossum | 2007-08-27 22:51:00 +0200 (Mon, 27 Aug 2007) | 3 lines
Patch # 1739906 by Christian Heimes -- add reduce to functools (importing
it from __builtin__).
........
r57575 | guido.van.rossum | 2007-08-27 22:52:10 +0200 (Mon, 27 Aug 2007) | 2 lines
News about functools.reduce.
........
r57611 | georg.brandl | 2007-08-28 10:29:08 +0200 (Tue, 28 Aug 2007) | 2 lines
Document rev. 57574.
........
r57612 | sean.reifschneider | 2007-08-28 11:07:54 +0200 (Tue, 28 Aug 2007) | 2 lines
Adding basic imputil documentation.
........
r57614 | georg.brandl | 2007-08-28 12:48:18 +0200 (Tue, 28 Aug 2007) | 2 lines
Fix some glitches.
........
r57616 | lars.gustaebel | 2007-08-28 14:31:09 +0200 (Tue, 28 Aug 2007) | 5 lines
TarFile.__init__() no longer fails if no name argument is passed and
the fileobj argument has no usable name attribute (e.g. StringIO).
(will backport to 2.5)
........
r57619 | thomas.wouters | 2007-08-28 17:28:19 +0200 (Tue, 28 Aug 2007) | 22 lines
Improve extended slicing support in builtin types and classes. Specifically:
- Specialcase extended slices that amount to a shallow copy the same way as
is done for simple slices, in the tuple, string and unicode case.
- Specialcase step-1 extended slices to optimize the common case for all
involved types.
- For lists, allow extended slice assignment of differing lengths as long
as the step is 1. (Previously, 'l[:2:1] = []' failed even though
'l[:2] = []' and 'l[:2:None] = []' do not.)
- Implement extended slicing for buffer, array, structseq, mmap and
UserString.UserString.
- Implement slice-object support (but not non-step-1 slice assignment) for
UserString.MutableString.
- Add tests for all new functionality.
........
diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py
new file mode 100644
index 0000000..35c6af9
--- /dev/null
+++ b/Lib/test/test_ssl.py
@@ -0,0 +1,372 @@
+# Test the support for SSL and sockets
+
+import sys
+import unittest
+from test import test_support
+import socket
+import errno
+import threading
+import subprocess
+import time
+import os
+import pprint
+import urllib
+import shutil
+import traceback
+
+# Optionally test SSL support, if we have it in the tested platform
+skip_expected = False
+try:
+ import ssl
+except ImportError:
+ skip_expected = True
+
+CERTFILE = None
+
+
+def handle_error(prefix):
+ exc_format = ' '.join(traceback.format_exception(*sys.exc_info()))
+ sys.stdout.write(prefix + exc_format)
+
+
+class BasicTests(unittest.TestCase):
+
+ def testRudeShutdown(self):
+ # Some random port to connect to.
+ PORT = [9934]
+
+ listener_ready = threading.Event()
+ listener_gone = threading.Event()
+
+ # `listener` runs in a thread. It opens a socket listening on
+ # PORT, and sits in an accept() until the main thread connects.
+ # Then it rudely closes the socket, and sets Event `listener_gone`
+ # to let the main thread know the socket is gone.
+ def listener():
+ s = socket.socket()
+ PORT[0] = test_support.bind_port(s, '', PORT[0])
+ s.listen(5)
+ listener_ready.set()
+ s.accept()
+ s = None # reclaim the socket object, which also closes it
+ listener_gone.set()
+
+ def connector():
+ listener_ready.wait()
+ s = socket.socket()
+ s.connect(('localhost', PORT[0]))
+ listener_gone.wait()
+ try:
+ ssl_sock = socket.ssl(s)
+ except socket.sslerror:
+ pass
+ else:
+ raise test_support.TestFailed(
+ 'connecting to closed SSL socket should have failed')
+
+ t = threading.Thread(target=listener)
+ t.start()
+ connector()
+ t.join()
+
+ def testSSLconnect(self):
+ import os
+ with test_support.transient_internet():
+ s = ssl.sslsocket(socket.socket(socket.AF_INET),
+ cert_reqs=ssl.CERT_NONE)
+ s.connect(("pop.gmail.com", 995))
+ c = s.getpeercert()
+ if c:
+ raise test_support.TestFailed("Peer cert %s shouldn't be here!")
+ s.close()
+
+ # this should fail because we have no verification certs
+ s = ssl.sslsocket(socket.socket(socket.AF_INET),
+ cert_reqs=ssl.CERT_REQUIRED)
+ try:
+ s.connect(("pop.gmail.com", 995))
+ except ssl.sslerror:
+ pass
+ finally:
+ s.close()
+
+class ConnectedTests(unittest.TestCase):
+
+ def testTLSecho (self):
+
+ s1 = socket.socket()
+ try:
+ s1.connect(('127.0.0.1', 10024))
+ except:
+ handle_error("connection failure:\n")
+ raise test_support.TestFailed("Can't connect to test server")
+ else:
+ try:
+ c1 = ssl.sslsocket(s1, ssl_version=ssl.PROTOCOL_TLSv1)
+ except:
+ handle_error("SSL handshake failure:\n")
+ raise test_support.TestFailed("Can't SSL-handshake with test server")
+ else:
+ if not c1:
+ raise test_support.TestFailed("Can't SSL-handshake with test server")
+ indata = "FOO\n"
+ c1.write(indata)
+ outdata = c1.read()
+ if outdata != indata.lower():
+ raise test_support.TestFailed("bad data <<%s>> received; expected <<%s>>\n" % (data, indata.lower()))
+ c1.close()
+
+ def testReadCert(self):
+
+ s2 = socket.socket()
+ try:
+ s2.connect(('127.0.0.1', 10024))
+ except:
+ handle_error("connection failure:\n")
+ raise test_support.TestFailed("Can't connect to test server")
+ else:
+ try:
+ c2 = ssl.sslsocket(s2, ssl_version=ssl.PROTOCOL_TLSv1,
+ cert_reqs=ssl.CERT_REQUIRED, ca_certs=CERTFILE)
+ except:
+ handle_error("SSL handshake failure:\n")
+ raise test_support.TestFailed("Can't SSL-handshake with test server")
+ else:
+ if not c2:
+ raise test_support.TestFailed("Can't SSL-handshake with test server")
+ cert = c2.getpeercert()
+ if not cert:
+ raise test_support.TestFailed("Can't get peer certificate.")
+ if not cert.has_key('subject'):
+ raise test_support.TestFailed(
+ "No subject field in certificate: %s." %
+ pprint.pformat(cert))
+ if not (cert['subject'].has_key('organizationName')):
+ raise test_support.TestFailed(
+ "No 'organizationName' field in certificate subject: %s." %
+ pprint.pformat(cert))
+ if (cert['subject']['organizationName'] !=
+ "Python Software Foundation"):
+ raise test_support.TestFailed(
+ "Invalid 'organizationName' field in certificate subject; "
+ "should be 'Python Software Foundation'.");
+ c2.close()
+
+
+class ThreadedEchoServer(threading.Thread):
+
+ class ConnectionHandler(threading.Thread):
+
+ def __init__(self, server, connsock):
+ self.server = server
+ self.running = False
+ self.sock = connsock
+ threading.Thread.__init__(self)
+ self.setDaemon(True)
+
+ def run (self):
+ self.running = True
+ try:
+ sslconn = ssl.sslsocket(self.sock, server_side=True,
+ certfile=self.server.certificate,
+ ssl_version=self.server.protocol,
+ cert_reqs=self.server.certreqs)
+ except:
+ # here, we want to stop the server, because this shouldn't
+ # happen in the context of our test case
+ handle_error("Test server failure:\n")
+ self.running = False
+ # normally, we'd just stop here, but for the test
+ # harness, we want to stop the server
+ self.server.stop()
+ return
+
+ while self.running:
+ try:
+ msg = sslconn.read()
+ if not msg:
+ # eof, so quit this handler
+ self.running = False
+ sslconn.close()
+ elif msg.strip() == 'over':
+ sslconn.close()
+ self.server.stop()
+ self.running = False
+ else:
+ if test_support.verbose:
+ sys.stdout.write("\nserver: %s\n" % msg.strip().lower())
+ sslconn.write(msg.lower())
+ except ssl.sslerror:
+ handle_error("Test server failure:\n")
+ sslconn.close()
+ self.running = False
+ # normally, we'd just stop here, but for the test
+ # harness, we want to stop the server
+ self.server.stop()
+ except:
+ handle_error('')
+
+ def __init__(self, port, certificate, ssl_version=None,
+ certreqs=None, cacerts=None):
+ if ssl_version is None:
+ ssl_version = ssl.PROTOCOL_TLSv1
+ if certreqs is None:
+ certreqs = ssl.CERT_NONE
+ self.certificate = certificate
+ self.protocol = ssl_version
+ self.certreqs = certreqs
+ self.cacerts = cacerts
+ self.sock = socket.socket()
+ self.flag = None
+ if hasattr(socket, 'SO_REUSEADDR'):
+ self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+ if hasattr(socket, 'SO_REUSEPORT'):
+ self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
+ self.sock.bind(('127.0.0.1', port))
+ self.active = False
+ threading.Thread.__init__(self)
+ self.setDaemon(False)
+
+ def start (self, flag=None):
+ self.flag = flag
+ threading.Thread.start(self)
+
+ def run (self):
+ self.sock.settimeout(0.5)
+ self.sock.listen(5)
+ self.active = True
+ if self.flag:
+ # signal an event
+ self.flag.set()
+ while self.active:
+ try:
+ newconn, connaddr = self.sock.accept()
+ if test_support.verbose:
+ sys.stdout.write('\nserver: new connection from ' + str(connaddr) + '\n')
+ handler = self.ConnectionHandler(self, newconn)
+ handler.start()
+ except socket.timeout:
+ pass
+ except KeyboardInterrupt:
+ self.stop()
+ except:
+ handle_error("Test server failure:\n")
+
+ def stop (self):
+ self.active = False
+ self.sock.close()
+
+CERTFILE_CONFIG_TEMPLATE = """
+# create RSA certs - Server
+
+[ req ]
+default_bits = 1024
+encrypt_key = yes
+distinguished_name = req_dn
+x509_extensions = cert_type
+
+[ req_dn ]
+countryName = Country Name (2 letter code)
+countryName_default = US
+countryName_min = 2
+countryName_max = 2
+
+stateOrProvinceName = State or Province Name (full name)
+stateOrProvinceName_default = %(state)s
+
+localityName = Locality Name (eg, city)
+localityName_default = %(city)s
+
+0.organizationName = Organization Name (eg, company)
+0.organizationName_default = %(organization)s
+
+organizationalUnitName = Organizational Unit Name (eg, section)
+organizationalUnitName_default = %(unit)s
+
+0.commonName = Common Name (FQDN of your server)
+0.commonName_default = %(common-name)s
+
+# To create a certificate for more than one name uncomment:
+# 1.commonName = DNS alias of your server
+# 2.commonName = DNS alias of your server
+# ...
+# See http://home.netscape.com/eng/security/ssl_2.0_certificate.html
+# to see how Netscape understands commonName.
+
+[ cert_type ]
+nsCertType = server
+"""
+
+def create_cert_files(hostname=None):
+
+ """This is the routine that was run to create the certificate
+ and private key contained in keycert.pem."""
+
+ import tempfile, socket, os
+ d = tempfile.mkdtemp()
+ # now create a configuration file for the CA signing cert
+ fqdn = hostname or socket.getfqdn()
+ crtfile = os.path.join(d, "cert.pem")
+ conffile = os.path.join(d, "ca.conf")
+ fp = open(conffile, "w")
+ fp.write(CERTFILE_CONFIG_TEMPLATE %
+ {'state': "Delaware",
+ 'city': "Wilmington",
+ 'organization': "Python Software Foundation",
+ 'unit': "SSL",
+ 'common-name': fqdn,
+ })
+ fp.close()
+ error = os.system(
+ "openssl req -batch -new -x509 -days 2000 -nodes -config %s "
+ "-keyout \"%s\" -out \"%s\" > /dev/null < /dev/null 2>&1" %
+ (conffile, crtfile, crtfile))
+ # now we have a self-signed server cert in crtfile
+ os.unlink(conffile)
+ if (os.WEXITSTATUS(error) or
+ not os.path.exists(crtfile) or os.path.getsize(crtfile) == 0):
+ if test_support.verbose:
+ sys.stdout.write("Unable to create certificate for test, "
+ + "error status %d\n" % (error >> 8))
+ crtfile = None
+ elif test_support.verbose:
+ sys.stdout.write(open(crtfile, 'r').read() + '\n')
+ return d, crtfile
+
+
+def test_main(verbose=False):
+ if skip_expected:
+ raise test_support.TestSkipped("socket module has no ssl support")
+
+ global CERTFILE
+ CERTFILE = os.path.join(os.path.dirname(__file__) or os.curdir,
+ "keycert.pem")
+ if not CERTFILE:
+ sys.__stdout__.write("Skipping test_ssl ConnectedTests; "
+ "couldn't create a certificate.\n")
+
+ tests = [BasicTests]
+
+ server = None
+ if CERTFILE and test_support.is_resource_enabled('network'):
+ server = ThreadedEchoServer(10024, CERTFILE)
+ flag = threading.Event()
+ server.start(flag)
+ # wait for it to start
+ flag.wait()
+ tests.append(ConnectedTests)
+
+ thread_info = test_support.threading_setup()
+
+ try:
+ test_support.run_unittest(*tests)
+ finally:
+ if server is not None and server.active:
+ server.stop()
+ # wait for it to stop
+ server.join()
+
+ test_support.threading_cleanup(*thread_info)
+
+if __name__ == "__main__":
+ test_main()