doc/pyOpenSSL.tex

\documentclass{howto}

\title{Python OpenSSL Manual}

\release{0.9}

\author{Jean-Paul Calderone}
\authoraddress{\email{exarkun@twistedmatrix.com}}

\usepackage[english]{babel}
\usepackage[T1]{fontenc}

\begin{document}

\maketitle

\begin{abstract}
\noindent
This module is a rather thin wrapper around (a subset of) the OpenSSL library.
With thin wrapper I mean that a lot of the object methods do nothing more than
calling a corresponding function in the OpenSSL library.
\end{abstract}

\tableofcontents


\section{Introduction \label{intro}}

The reason pyOpenSSL was created is that the SSL support in the socket module
in Python 2.1 (the contemporary version of Python when the pyOpenSSL project
was begun) was severely limited.  Other OpenSSL wrappers for Python at the time
were also limited, though in different ways.  Unfortunately, Python's standard
library SSL support has remained weak, although other packages (such as
M2Crypto\footnote{See \url{http://chandlerproject.org/Projects/MeTooCrypto}})
have made great advances and now equal or exceed pyOpenSSL's functionality.

The reason pyOpenSSL continues to be maintained is that there is a significant
user community around it, as well as a large amount of software which depends
on it.  It is a great benefit to many people for pyOpenSSL to continue to exist
and advance.

\section{Building and Installing \label{building}}

These instructions can also be found in the file \verb|INSTALL|.

I have tested this on Debian Linux systems (woody and sid), Solaris 2.6 and
2.7. Others have successfully compiled it on Windows and NT.

\subsection{Building the Module on a Unix System \label{building-unix}}

pyOpenSSL uses distutils, so there really shouldn't be any problems. To build
the library:
\begin{verbatim}
python setup.py build
\end{verbatim}

If your OpenSSL header files aren't in \verb|/usr/include|, you may need to
supply the \verb|-I| flag to let the setup script know where to look. The same
goes for the libraries of course, use the \verb|-L| flag. Note that
\verb|build| won't accept these flags, so you have to run first
\verb|build_ext| and then \verb|build|! Example:
\begin{verbatim}
python setup.py build_ext -I/usr/local/ssl/include -L/usr/local/ssl/lib
python setup.py build
\end{verbatim}

Now you should have a directory called \verb|OpenSSL| that contains e.g.
\verb|SSL.so| and \verb|__init__.py| somewhere in the build dicrectory,
so just:
\begin{verbatim}
python setup.py install
\end{verbatim}

If you, for some arcane reason, don't want the module to appear in the
\verb|site-packages| directory, use the \verb|--prefix| option.

You can, of course, do
\begin{verbatim}
python setup.py --help
\end{verbatim}

to find out more about how to use the script.

\subsection{Building the Module on a Windows System \label{building-windows}}

Big thanks to Itamar Shtull-Trauring and Oleg Orlov for their help with
Windows build instructions.  Same as for Unix systems, we have to separate
the \verb|build_ext| and the \verb|build|.

Building the library:

\begin{verbatim}
setup.py build_ext -I ...\openssl\inc32 -L ...\openssl\out32dll
setup.py build
\end{verbatim}

Where \verb|...\openssl| is of course the location of your OpenSSL installation.

Installation is the same as for Unix systems:
\begin{verbatim}
setup.py install
\end{verbatim}

And similarily, you can do
\begin{verbatim}
setup.py --help
\end{verbatim}

to get more information.


\section{\module{OpenSSL} --- Python interface to OpenSSL \label{openssl}}

\declaremodule{extension}{OpenSSL}
\modulesynopsis{Python interface to OpenSSL}

This package provides a high-level interface to the functions in the
OpenSSL library. The following modules are defined:

\begin{datadesc}{crypto}
Generic cryptographic module. Note that if anything is incomplete, this module is!
\end{datadesc}

\begin{datadesc}{rand}
An interface to the OpenSSL pseudo random number generator.
\end{datadesc}

\begin{datadesc}{SSL}
An interface to the SSL-specific parts of OpenSSL.
\end{datadesc}


% % % crypto moduleOpenSSL

\subsection{\module{crypto} --- Generic cryptographic module \label{openssl-crypto}}

\declaremodule{extension}{crypto}
\modulesynopsis{Generic cryptographic module}

\begin{datadesc}{X509Type}
See \class{X509}.
\end{datadesc}

\begin{classdesc}{X509}{}
A class representing X.509 certificates.
\end{classdesc}

\begin{datadesc}{X509NameType}
See \class{X509Name}.
\end{datadesc}

\begin{classdesc}{X509Name}{x509name}
A class representing X.509 Distinguished Names.

This constructor creates a copy of \var{x509name} which should be an
instance of \class{X509Name}.
\end{classdesc}

\begin{datadesc}{X509ReqType}
See \class{X509Req}.
\end{datadesc}

\begin{classdesc}{X509Req}{}
A class representing X.509 certificate requests.
\end{classdesc}

\begin{datadesc}{X509StoreType}
A Python type object representing the X509Store object type.
\end{datadesc}

\begin{datadesc}{PKeyType}
See \class{PKey}.
\end{datadesc}

\begin{classdesc}{PKey}{}
A class representing DSA or RSA keys.
\end{classdesc}

\begin{datadesc}{PKCS7Type}
A Python type object representing the PKCS7 object type.
\end{datadesc}

\begin{datadesc}{PKCS12Type}
A Python type object representing the PKCS12 object type.
\end{datadesc}

\begin{datadesc}{X509ExtensionType}
See \class{X509Extension}.
\end{datadesc}

\begin{classdesc}{X509Extension}{typename, critical, value}
A class representing an X.509 v3 certificate extensions.
\end{classdesc}

\begin{datadesc}{NetscapeSPKIType}
See \class{NetscapeSPKI}.
\end{datadesc}

\begin{classdesc}{NetscapeSPKI}{\optional{enc}}
A class representing Netscape SPKI objects.

If the \var{enc} argument is present, it should be a base64-encoded string
representing a NetscapeSPKI object, as returned by the \method{b64_encode}
method.
\end{classdesc}

\begin{datadesc}{FILETYPE_PEM}
\dataline{FILETYPE_ASN1}
File type constants.
\end{datadesc}

\begin{datadesc}{TYPE_RSA}
\dataline{TYPE_DSA}
Key type constants.
\end{datadesc}

\begin{excdesc}{Error}
Generic exception used in the \module{crypto} module.
\end{excdesc}

\begin{funcdesc}{dump_certificate}{type, cert}
Dump the certificate \var{cert} into a buffer string encoded with the type
\var{type}.
\end{funcdesc}

\begin{funcdesc}{dump_certificate_request}{type, req}
Dump the certificate request \var{req} into a buffer string encoded with the
type \var{type}.
\end{funcdesc}

\begin{funcdesc}{dump_privatekey}{type, pkey\optional{, cipher, passphrase}}
Dump the private key \var{pkey} into a buffer string encoded with the type
\var{type}, optionally (if \var{type} is \constant{FILETYPE_PEM}) encrypting it
using \var{cipher} and \var{passphrase}.

\var{passphrase} must be either a string or a callback for providing the
pass phrase.
\end{funcdesc}

\begin{funcdesc}{load_certificate}{type, buffer}
Load a certificate (X509) from the string \var{buffer} encoded with the
type \var{type}.
\end{funcdesc}

\begin{funcdesc}{load_certificate_request}{type, buffer}
Load a certificate request (X509Req) from the string \var{buffer} encoded with
the type \var{type}.
\end{funcdesc}

\begin{funcdesc}{load_privatekey}{type, buffer\optional{, passphrase}}
Load a private key (PKey) from the string \var{buffer} encoded with
the type \var{type} (must be one of \constant{FILETYPE_PEM} and
\constant{FILETYPE_ASN1}).

\var{passphrase} must be either a string or a callback for providing the
pass phrase.
\end{funcdesc}

\begin{funcdesc}{load_pkcs7_data}{type, buffer}
Load pkcs7 data from the string \var{buffer} encoded with the type \var{type}.
\end{funcdesc}

\begin{funcdesc}{load_pkcs12}{buffer\optional{, passphrase}}
Load pkcs12 data from the string \var{buffer}. If the pkcs12 structure is
encrypted, a \var{passphrase} must be included.
\end{funcdesc}

\subsubsection{X509 objects \label{openssl-x509}}

X509 objects have the following methods:

\begin{methoddesc}[X509]{get_issuer}{}
Return an X509Name object representing the issuer of the certificate.
\end{methoddesc}

\begin{methoddesc}[X509]{get_pubkey}{}
Return a PKey object representing the public key of the certificate.
\end{methoddesc}

\begin{methoddesc}[X509]{get_serial_number}{}
Return the certificate serial number.
\end{methoddesc}

\begin{methoddesc}[X509]{get_subject}{}
Return an X509Name object representing the subject of the certificate.
\end{methoddesc}

\begin{methoddesc}[X509]{get_version}{}
Return the certificate version.
\end{methoddesc}

\begin{methoddesc}[X509]{get_notBefore}{}
Return a string giving the time before which the certificate is not valid.  The
string is formatted as an ASN1 GENERALIZEDTIME:
\begin{verbatim}
                 YYYYMMDDhhmmssZ
                 YYYYMMDDhhmmss+hhmm
                 YYYYMMDDhhmmss-hhmm
\end{verbatim}
If no value exists for this field, \code{None} is returned.
\end{methoddesc}

\begin{methoddesc}[X509]{get_notAfter}{}
Return a string giving the time after which the certificate is not valid.  The
string is formatted as an ASN1 GENERALIZEDTIME:
\begin{verbatim}
                 YYYYMMDDhhmmssZ
                 YYYYMMDDhhmmss+hhmm
                 YYYYMMDDhhmmss-hhmm
\end{verbatim}
If no value exists for this field, \code{None} is returned.
\end{methoddesc}

\begin{methoddesc}[X509]{set_notBefore}{when}
Change the time before which the certificate is not valid.  \var{when} is a
string formatted as an ASN1 GENERALIZEDTIME:
\begin{verbatim}
                 YYYYMMDDhhmmssZ
                 YYYYMMDDhhmmss+hhmm
                 YYYYMMDDhhmmss-hhmm
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[X509]{set_notAfter}{when}
Change the time after which the certificate is not valid.  \var{when} is a
string formatted as an ASN1 GENERALIZEDTIME:
\begin{verbatim}
                 YYYYMMDDhhmmssZ
                 YYYYMMDDhhmmss+hhmm
                 YYYYMMDDhhmmss-hhmm
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[X509]{gmtime_adj_notBefore}{time}
Adjust the timestamp (in GMT) when the certificate starts being valid.
\end{methoddesc}

\begin{methoddesc}[X509]{gmtime_adj_notAfter}{time}
Adjust the timestamp (in GMT) when the certificate stops being valid.
\end{methoddesc}

\begin{methoddesc}[X509]{has_expired}{}
Checks the certificate's time stamp against current time. Returns true if the
certificate has expired and false otherwise.
\end{methoddesc}

\begin{methoddesc}[X509]{set_issuer}{issuer}
Set the issuer of the certificate to \var{issuer}.
\end{methoddesc}

\begin{methoddesc}[X509]{set_pubkey}{pkey}
Set the public key of the certificate to \var{pkey}.
\end{methoddesc}

\begin{methoddesc}[X509]{set_serial_number}{serialno}
Set the serial number of the certificate to \var{serialno}.
\end{methoddesc}

\begin{methoddesc}[X509]{set_subject}{subject}
Set the subject of the certificate to \var{subject}.
\end{methoddesc}

\begin{methoddesc}[X509]{set_version}{version}
Set the certificate version to \var{version}.
\end{methoddesc}

\begin{methoddesc}[X509]{sign}{pkey, digest}
Sign the certificate, using the key \var{pkey} and the message digest algorithm
identified by the string \var{digest}.
\end{methoddesc}

\begin{methoddesc}[X509]{subject_name_hash}{}
Return the hash of the certificate subject.
\end{methoddesc}

\begin{methoddesc}[X509]{digest}{digest_name}
Return a digest of the certificate, using the \var{digest_name} method.
\var{digest_name} must be a string describing a digest algorithm supported
by OpenSSL (by EVP_get_digestbyname, specifically).  For example,
\constant{"md5"} or \constant{"sha1"}.
\end{methoddesc}

\begin{methoddesc}[X509]{add_extensions}{extensions}
Add the extensions in the sequence \var{extensions} to the certificate.
\end{methoddesc}

\subsubsection{X509Name objects \label{openssl-x509name}}

X509Name objects have the following methods:

\begin{methoddesc}[X509Name]{hash}{}
Return an integer giving the first four bytes of the MD5 digest of the DER
representation of the name.
\end{methoddesc}

\begin{methoddesc}[X509Name]{der}{}
Return a string giving the DER representation of the name.
\end{methoddesc}

\begin{methoddesc}[X509Name]{get_components}{}
Return a list of two-tuples of strings giving the components of the name.
\end{methoddesc}

X509Name objects have the following members:

\begin{memberdesc}[X509Name]{countryName}
The country of the entity. \code{C} may be used as an alias for
\code{countryName}.
\end{memberdesc}

\begin{memberdesc}[X509Name]{stateOrProvinceName}
The state or province of the entity. \code{ST} may be used as an alias for
\code{stateOrProvinceName}·
\end{memberdesc}

\begin{memberdesc}[X509Name]{localityName}
The locality of the entity. \code{L} may be used as an alias for
\code{localityName}.
\end{memberdesc}

\begin{memberdesc}[X509Name]{organizationName}
The organization name of the entity. \code{O} may be used as an alias for
\code{organizationName}.
\end{memberdesc}

\begin{memberdesc}[X509Name]{organizationalUnitName}
The organizational unit of the entity. \code{OU} may be used as an alias for
\code{organizationalUnitName}.
\end{memberdesc}

\begin{memberdesc}[X509Name]{commonName}
The common name of the entity. \code{CN} may be used as an alias for
\code{commonName}.
\end{memberdesc}

\begin{memberdesc}[X509Name]{emailAddress}
The e-mail address of the entity.
\end{memberdesc}

\subsubsection{X509Req objects \label{openssl-x509req}}

X509Req objects have the following methods:

\begin{methoddesc}[X509Req]{get_pubkey}{}
Return a PKey object representing the public key of the certificate request.
\end{methoddesc}

\begin{methoddesc}[X509Req]{get_subject}{}
Return an X509Name object representing the subject of the certificate.
\end{methoddesc}

\begin{methoddesc}[X509Req]{set_pubkey}{pkey}
Set the public key of the certificate request to \var{pkey}.
\end{methoddesc}

\begin{methoddesc}[X509Req]{sign}{pkey, digest}
Sign the certificate request, using the key \var{pkey} and the message digest
algorithm identified by the string \var{digest}.
\end{methoddesc}

\begin{methoddesc}[X509Req]{verify}{pkey}
Verify a certificate request using the public key \var{pkey}.
\end{methoddesc}

\begin{methoddesc}[X509Req]{set_version}{version}
Set the version (RFC 2459, 4.1.2.1) of the certificate request to
\var{version}.
\end{methoddesc}

\begin{methoddesc}[X509Req]{get_version}{}
Get the version (RFC 2459, 4.1.2.1) of the certificate request.
\end{methoddesc}

\subsubsection{X509Store objects \label{openssl-x509store}}

The X509Store object has currently just one method:

\begin{methoddesc}[X509Store]{add_cert}{cert}
Add the certificate \var{cert} to the certificate store.
\end{methoddesc}

\subsubsection{PKey objects \label{openssl-pkey}}

The PKey object has the following methods:

\begin{methoddesc}[PKey]{bits}{}
Return the number of bits of the key.
\end{methoddesc}

\begin{methoddesc}[PKey]{generate_key}{type, bits}
Generate a public/private key pair of the type \var{type} (one of
\constant{TYPE_RSA} and \constant{TYPE_DSA}) with the size \var{bits}.
\end{methoddesc}

\begin{methoddesc}[PKey]{type}{}
Return the type of the key.
\end{methoddesc}

\subsubsection{PKCS7 objects \label{openssl-pkcs7}}

PKCS7 objects have the following methods:

\begin{methoddesc}[PKCS7]{type_is_signed}{}
FIXME
\end{methoddesc}

\begin{methoddesc}[PKCS7]{type_is_enveloped}{}
FIXME
\end{methoddesc}

\begin{methoddesc}[PKCS7]{type_is_signedAndEnveloped}{}
FIXME
\end{methoddesc}

\begin{methoddesc}[PKCS7]{type_is_data}{}
FIXME
\end{methoddesc}

\begin{methoddesc}[PKCS7]{get_type_name}{}
Get the type name of the PKCS7.
\end{methoddesc}

\subsubsection{PKCS12 objects \label{openssl-pkcs12}}

PKCS12 objects have the following methods:

\begin{methoddesc}[PKCS12]{get_certificate}{}
Return certificate portion of the PKCS12 structure.
\end{methoddesc}

\begin{methoddesc}[PKCS12]{get_privatekey}{}
Return private key portion of the PKCS12 structure
\end{methoddesc}

\begin{methoddesc}[PKCS12]{get_ca_certificates}{}
Return CA certificates within the PKCS12 object as a tuple. Returns
None if no CA certificates are present.
\end{methoddesc}

\subsubsection{X509Extension objects \label{openssl-509ext}}

X509Extension objects have several methods:

\begin{methoddesc}[X509Extension]{get_critical}{}
Return the critical field of the extension object.
\end{methoddesc}

\begin{methoddesc}[X509Extension]{get_short_name}{}
Return the short type name of the extension object.
\end{methoddesc}

\subsubsection{NetscapeSPKI objects \label{openssl-netscape-spki}}

NetscapeSPKI objects have the following methods:

\begin{methoddesc}[NetscapeSPKI]{b64_encode}{}
Return a base64-encoded string representation of the object.
\end{methoddesc}

\begin{methoddesc}[NetscapeSPKI]{get_pubkey}{}
Return the public key of object.
\end{methoddesc}

\begin{methoddesc}[NetscapeSPKI]{set_pubkey}{key}
Set the public key of the object to \var{key}.
\end{methoddesc}

\begin{methoddesc}[NetscapeSPKI]{sign}{key, digest_name}
Sign the NetscapeSPKI object using the given \var{key} and
\var{digest_name}.  \var{digest_name} must be a string describing a digest
algorithm supported by OpenSSL (by EVP_get_digestbyname, specifically).  For
example, \constant{"md5"} or \constant{"sha1"}.
\end{methoddesc}

\begin{methoddesc}[NetscapeSPKI]{verify}{key}
Verify the NetscapeSPKI object using the given \var{key}.
\end{methoddesc}


% % % rand module

\subsection{\module{rand} --- An interface to the OpenSSL pseudo random number generator \label{openssl-rand}}

\declaremodule{extension}{rand}
\modulesynopsis{An interface to the OpenSSL pseudo random number generator}

This module handles the OpenSSL pseudo random number generator (PRNG) and
declares the following:

\begin{funcdesc}{add}{string, entropy}
Mix bytes from \var{string} into the PRNG state. The \var{entropy} argument is
(the lower bound of) an estimate of how much randomness is contained in
\var{string}, measured in bytes. For more information, see e.g. \rfc{1750}.
\end{funcdesc}

\begin{funcdesc}{egd}{path\optional{, bytes}}
Query the Entropy Gathering Daemon\footnote{See
\url{http://www.lothar.com/tech/crypto/}} on socket \var{path} for \var{bytes}
bytes of random data and and uses \function{add} to seed the PRNG. The default
value of \var{bytes} is 255.
\end{funcdesc}

\begin{funcdesc}{load_file}{path\optional{, bytes}}
Read \var{bytes} bytes (or all of it, if \var{bytes} is negative) of data from
the file \var{path} to seed the PRNG. The default value of \var{bytes} is -1.
\end{funcdesc}

\begin{funcdesc}{screen}{}
Add the current contents of the screen to the PRNG state.
Availability: Windows.
\end{funcdesc}

\begin{funcdesc}{seed}{string}
This is equivalent to calling \function{add} with \var{entropy} as the length
of the string.
\end{funcdesc}

\begin{funcdesc}{status}{}
Returns true if the PRNG has been seeded with enough data, and false otherwise.
\end{funcdesc}

\begin{funcdesc}{write_file}{path}
Write a number of random bytes (currently 1024) to the file \var{path}. This
file can then be used with \function{load_file} to seed the PRNG again.
\end{funcdesc}



% % % SSL module

\subsection{\module{SSL} --- An interface to the SSL-specific parts of OpenSSL \label{openssl-ssl}}

\declaremodule{extension}{SSL}
\modulesynopsis{An interface to the SSL-specific parts of OpenSSL}

This module handles things specific to SSL. There are two objects defined:
Context, Connection.

\begin{datadesc}{SSLv2_METHOD}
\dataline{SSLv3_METHOD}
\dataline{SSLv23_METHOD}
\dataline{TLSv1_METHOD}
These constants represent the different SSL methods to use when creating a
context object.
\end{datadesc}

\begin{datadesc}{VERIFY_NONE}
\dataline{VERIFY_PEER}
\dataline{VERIFY_FAIL_IF_NO_PEER_CERT}
These constants represent the verification mode used by the Context
object's \method{set_verify} method.
\end{datadesc}

\begin{datadesc}{FILETYPE_PEM}
\dataline{FILETYPE_ASN1}
File type constants used with the \method{use_certificate_file} and
\method{use_privatekey_file} methods of Context objects.
\end{datadesc}

\begin{datadesc}{OP_SINGLE_DH_USE}
\dataline{OP_EPHEMERAL_RSA}
\dataline{OP_NO_SSLv2}
\dataline{OP_NO_SSLv3}
\dataline{OP_NO_TLSv1}
Constants used with \method{set_options} of Context objects.
\constant{OP_SINGLE_DH_USE} means to always create a new key when using ephemeral
Diffie-Hellman. \constant{OP_EPHEMERAL_RSA} means to always use ephemeral RSA keys
when doing RSA operations. \constant{OP_NO_SSLv2}, \constant{OP_NO_SSLv3} and
\constant{OP_NO_TLSv1} means to disable those specific protocols. This is
interesting if you're using e.g. \constant{SSLv23_METHOD} to get an SSLv2-compatible
handshake, but don't want to use SSLv2.
\end{datadesc}

\begin{datadesc}{ContextType}
See \class{Context}.
\end{datadesc}

\begin{classdesc}{Context}{method}
A class representing SSL contexts.  Contexts define the parameters of one or
more SSL connections.

\var{method} should be \constant{SSLv2_METHOD}, \constant{SSLv3_METHOD},
\constant{SSLv23_METHOD} or \constant{TLSv1_METHOD}.
\end{classdesc}

\begin{datadesc}{ConnectionType}
See \class{Connection}.
\end{datadesc}

\begin{classdesc}{Connection}{context, socket}
A class representing SSL connections.

\var{context} should be an instance of \class{Context} and \var{socket}
should be a socket \footnote{Actually, all that is required is an object
that \emph{behaves} like a socket, you could even use files, even though
it'd be tricky to get the handshakes right!} object.  \var{socket} may be
\var{None}; in this case, the Connection is created with a memory BIO: see
the \method{bio_read}, \method{bio_write}, and \method{bio_shutdown}
methods.
\end{classdesc}

\begin{excdesc}{Error}
This exception is used as a base class for the other SSL-related
exceptions, but may also be raised directly.

Whenever this exception is raised directly, it has a list of error messages
from the OpenSSL error queue, where each item is a tuple \code{(\var{lib},
\var{function}, \var{reason})}. Here \var{lib}, \var{function} and \var{reason}
are all strings, describing where and what the problem is. See \manpage{err}{3}
for more information.
\end{excdesc}

\begin{excdesc}{ZeroReturnError}
This exception matches the error return code \code{SSL_ERROR_ZERO_RETURN}, and
is raised when the SSL Connection has been closed. In SSL 3.0 and TLS 1.0, this
only occurs if a closure alert has occurred in the protocol, i.e. the
connection has been closed cleanly. Note that this does not necessarily
mean that the transport layer (e.g. a socket) has been closed.

It may seem a little strange that this is an exception, but it does match an
\code{SSL_ERROR} code, and is very convenient.
\end{excdesc}

\begin{excdesc}{WantReadError}
The operation did not complete; the same I/O method should be called again
later, with the same arguments. Any I/O method can lead to this since new
handshakes can occur at any time.

The wanted read is for "dirty" data sent over the network, not the 
"clean" data inside the tunnel.  For a socket based SSL connection, 
"read" means data coming at us over the network.  Until
that read succeeds, the attempted 
\method{OpenSSL.SSL.Connection.recv}, 
\method{OpenSSL.SSL.Connection.send}, or
\method{OpenSSL.SSL.Connection.do_handshake} is prevented or incomplete.
You probably want to \method{select()} on the socket before trying again.
\end{excdesc}

\begin{excdesc}{WantWriteError}
See \exception{WantReadError}.  The socket may be too full to
write more data.
\end{excdesc}

\begin{excdesc}{WantX509LookupError}
The operation did not complete because an application callback has asked to be
called again. The I/O method should be called again later, with the same
arguments. Note: This won't occur in this version, as there are no such
callbacks in this version.
\end{excdesc}

\begin{excdesc}{SysCallError}
The \exception{SysCallError} occurs when there's an I/O error and OpenSSL's
error queue does not contain any information. This can mean two things: An
error in the transport protocol, or an end of file that violates the protocol.
The parameter to the exception is always a pair \code{(\var{errnum},
\var{errstr})}.
\end{excdesc}


\subsubsection{Context objects \label{openssl-context}}

Context objects have the following methods:

\begin{methoddesc}[Context]{check_privatekey}{}
Check if the private key (loaded with \method{use_privatekey\optional{_file}})
matches the certificate (loaded with \method{use_certificate\optional{_file}}).
Returns \code{None} if they match, raises \exception{Error} otherwise.
\end{methoddesc}

\begin{methoddesc}[Context]{get_app_data}{}
Retrieve application data as set by \method{set_app_data}.
\end{methoddesc}

\begin{methoddesc}[Context]{get_cert_store}{}
Retrieve the certificate store (a X509Store object) that the context uses.
This can be used to add "trusted" certificates without using the.
\method{load_verify_locations()} method.
\end{methoddesc}

\begin{methoddesc}[Context]{get_timeout}{}
Retrieve session timeout, as set by \method{set_timeout}. The default is 300
seconds.
\end{methoddesc}

\begin{methoddesc}[Context]{get_verify_depth}{}
Retrieve the Context object's verify depth, as set by
\method{set_verify_depth}.
\end{methoddesc}

\begin{methoddesc}[Context]{get_verify_mode}{}
Retrieve the Context object's verify mode, as set by \method{set_verify}.
\end{methoddesc}

\begin{methoddesc}[Context]{load_client_ca}{pemfile}
Read a file with PEM-formatted certificates that will be sent to the client
when requesting a client certificate.
\end{methoddesc}

\begin{methoddesc}[Context]{load_verify_locations}{pemfile, capath}
Specify where CA certificates for verification purposes are located. These
are trusted certificates. Note that the certificates have to be in PEM
format.  If capath is passed, it must be a directory prepared using the
\code{c_rehash} tool included with OpenSSL.  Either, but not both, of
\var{pemfile} or \var{capath} may be \code{None}.
\end{methoddesc}

\begin{methoddesc}[Context]{set_default_verify_paths}{}
Specify that the platform provided CA certificates are to be used for
verification purposes.  This method may not work properly on OS X.
\end{methoddesc}

\begin{methoddesc}[Context]{load_tmp_dh}{dhfile}
Load parameters for Ephemeral Diffie-Hellman from \var{dhfile}.
\end{methoddesc}

\begin{methoddesc}[Context]{set_app_data}{data}
Associate \var{data} with this Context object. \var{data} can be retrieved
later using the \method{get_app_data} method.
\end{methoddesc}

\begin{methoddesc}[Context]{set_cipher_list}{ciphers}
Set the list of ciphers to be used in this context. See the OpenSSL manual for
more information (e.g. ciphers(1))
\end{methoddesc}

\begin{methoddesc}[Context]{set_info_callback}{callback}
Set the information callback to \var{callback}. This function will be called
from time to time during SSL handshakes.
\var{callback} should take three arguments: a Connection object and two
integers. The first integer specifies where in the SSL handshake the function
was called, and the other the return code from a (possibly failed) internal
function call.
\end{methoddesc}

\begin{methoddesc}[Context]{set_options}{options}
Add SSL options. Options you have set before are not cleared!
This method should be used with the \constant{OP_*} constants.
\end{methoddesc}

\begin{methoddesc}[Context]{set_passwd_cb}{callback\optional{, userdata}}
Set the passphrase callback to \var{callback}. This function will be called
when a private key with a passphrase is loaded. \var{callback} must accept
three positional arguments.  First, an integer giving the maximum length of
the passphrase it may return.  If the returned passphrase is longer than
this, it will be truncated.  Second, a boolean value which will be true if
the user should be prompted for the passphrase twice and the callback should
verify that the two values supplied are equal. Third, the value given as the
\var{userdata} parameter to \method{set_passwd_cb}.  If an error occurs,
\var{callback} should return a false value (e.g. an empty string).
\end{methoddesc}

\begin{methoddesc}[Context]{set_session_id}{name}
Set the context \var{name} within which a session can be reused for this
Context object. This is needed when doing session resumption, because there is
no way for a stored session to know which Context object it is associated with.
\var{name} may be any binary data.
\end{methoddesc}

\begin{methoddesc}[Context]{set_timeout}{timeout}
Set the timeout for newly created sessions for this Context object to
\var{timeout}. \var{timeout} must be given in (whole) seconds. The default
value is 300 seconds. See the OpenSSL manual for more information (e.g.
SSL_CTX_set_timeout(3)).
\end{methoddesc}

\begin{methoddesc}[Context]{set_verify}{mode, callback}
Set the verification flags for this Context object to \var{mode} and specify
that \var{callback} should be used for verification callbacks. \var{mode}
should be one of \constant{VERIFY_NONE} and \constant{VERIFY_PEER}. If
\constant{VERIFY_PEER} is used, \var{mode} can be OR:ed with
\constant{VERIFY_FAIL_IF_NO_PEER_CERT} and \constant{VERIFY_CLIENT_ONCE} to
further control the behaviour.
\var{callback} should take five arguments: A Connection object, an X509 object,
and three integer variables, which are in turn potential error number, error
depth and return code. \var{callback} should return true if verification passes
and false otherwise.
\end{methoddesc}

\begin{methoddesc}[Context]{set_verify_depth}{depth}
Set the maximum depth for the certificate chain verification that shall be
allowed for this Context object.
\end{methoddesc}

\begin{methoddesc}[Context]{use_certificate}{cert}
Use the certificate \var{cert} which has to be a X509 object.
\end{methoddesc}

\begin{methoddesc}[Context]{add_extra_chain_cert}{cert}
Adds the certificate \var{cert}, which has to be a X509 object, to the
certificate chain presented together with the certificate.
\end{methoddesc}

\begin{methoddesc}[Context]{use_certificate_chain_file}{file}
Load a certificate chain from \var{file} which must be PEM encoded.
\end{methoddesc}

\begin{methoddesc}[Context]{use_privatekey}{pkey}
Use the private key \var{pkey} which has to be a PKey object.
\end{methoddesc}

\begin{methoddesc}[Context]{use_certificate_file}{file\optional{, format}}
Load the first certificate found in \var{file}. The certificate must be in the
format specified by \var{format}, which is either \constant{FILETYPE_PEM} or
\constant{FILETYPE_ASN1}. The default is \constant{FILETYPE_PEM}.
\end{methoddesc}

\begin{methoddesc}[Context]{use_privatekey_file}{file\optional{, format}}
Load the first private key found in \var{file}. The private key must be in the
format specified by \var{format}, which is either \constant{FILETYPE_PEM} or
\constant{FILETYPE_ASN1}. The default is \constant{FILETYPE_PEM}.
\end{methoddesc}


\subsubsection{Connection objects \label{openssl-connection}}

Connection objects have the following methods:

\begin{methoddesc}[Connection]{accept}{}
Call the \method{accept} method of the underlying socket and set up SSL on the
returned socket, using the Context object supplied to this Connection object at
creation. Returns a pair \code{(\var{conn}, \var{address})}. where \var{conn}
is the new Connection object created, and \var{address} is as returned by the
socket's \method{accept}.
\end{methoddesc}

\begin{methoddesc}[Connection]{bind}{address}
Call the \method{bind} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{close}{}
Call the \method{close} method of the underlying socket. Note: If you want
correct SSL closure, you need to call the \method{shutdown} method first.
\end{methoddesc}

\begin{methoddesc}[Connection]{connect}{address}
Call the \method{connect} method of the underlying socket and set up SSL on the
socket, using the Context object supplied to this Connection object at
creation.
\end{methoddesc}

\begin{methoddesc}[Connection]{connect_ex}{address}
Call the \method{connect_ex} method of the underlying socket and set up SSL on
the socket, using the Context object supplied to this Connection object at
creation. Note that if the \method{connect_ex} method of the socket doesn't
return 0, SSL won't be initialized.
\end{methoddesc}

\begin{methoddesc}[Connection]{do_handshake}{}
Perform an SSL handshake (usually called after \method{renegotiate} or one of
\method{set_accept_state} or \method{set_accept_state}). This can raise the
same exceptions as \method{send} and \method{recv}.
\end{methoddesc}

\begin{methoddesc}[Connection]{fileno}{}
Retrieve the file descriptor number for the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{listen}{backlog}
Call the \method{listen} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{get_app_data}{}
Retrieve application data as set by \method{set_app_data}.
\end{methoddesc}

\begin{methoddesc}[Connection]{get_cipher_list}{}
Retrieve the list of ciphers used by the Connection object. WARNING: This API
has changed. It used to take an optional parameter and just return a string,
but not it returns the entire list in one go.
\end{methoddesc}

\begin{methoddesc}[Connection]{get_context}{}
Retrieve the Context object associated with this Connection.
\end{methoddesc}

\begin{methoddesc}[Connection]{get_peer_certificate}{}
Retrieve the other side's certificate (if any)
\end{methoddesc}

\begin{methoddesc}[Connection]{getpeername}{}
Call the \method{getpeername} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{getsockname}{}
Call the \method{getsockname} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{getsockopt}{level, optname\optional{, buflen}}
Call the \method{getsockopt} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{pending}{}
Retrieve the number of bytes that can be safely read from the SSL buffer
(\emph{not} the underlying transport buffer).
\end{methoddesc}

\begin{methoddesc}[Connection]{recv}{bufsize}
Receive data from the Connection. The return value is a string representing the
data received. The maximum amount of data to be received at once, is specified
by \var{bufsize}.
\end{methoddesc}

\begin{methoddesc}[Connection]{bio_write}{bytes}
If the Connection was created with a memory BIO, this method can be used to add
bytes to the read end of that memory BIO.  The Connection can then read the
bytes (for example, in response to a call to \method{recv}).
\end{methoddesc}

\begin{methoddesc}[Connection]{renegotiate}{}
Renegotiate the SSL session. Call this if you wish to change cipher suites or
anything like that.
\end{methoddesc}

\begin{methoddesc}[Connection]{send}{string}
Send the \var{string} data to the Connection.
\end{methoddesc}

\begin{methoddesc}[Connection]{bio_read}{bufsize}
If the Connection was created with a memory BIO, this method can be used to
read bytes from the write end of that memory BIO.  Many Connection methods will
add bytes which must be read in this manner or the buffer will eventually fill
up and the Connection will be able to take no further actions.
\end{methoddesc}

\begin{methoddesc}[Connection]{sendall}{string}
Send all of the \var{string} data to the Connection. This calls \method{send}
repeatedly until all data is sent. If an error occurs, it's impossible to tell
how much data has been sent.
\end{methoddesc}

\begin{methoddesc}[Connection]{set_accept_state}{}
Set the connection to work in server mode. The handshake will be handled
automatically by read/write.
\end{methoddesc}

\begin{methoddesc}[Connection]{set_app_data}{data}
Associate \var{data} with this Connection object. \var{data} can be retrieved
later using the \method{get_app_data} method.
\end{methoddesc}

\begin{methoddesc}[Connection]{set_connect_state}{}
Set the connection to work in client mode. The handshake will be handled
automatically by read/write.
\end{methoddesc}

\begin{methoddesc}[Connection]{setblocking}{flag}
Call the \method{setblocking} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{setsockopt}{level, optname, value}
Call the \method{setsockopt} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{shutdown}{}
Send the shutdown message to the Connection. Returns true if the shutdown
message exchange is completed and false otherwise (in which case you call
\method{recv()} or \method{send()} when the connection becomes
readable/writeable.
\end{methoddesc}

\begin{methoddesc}[Connection]{get_shutdown}{}
Get the shutdown state of the Connection.  Returns a bitvector of either or
both of \var{SENT_SHUTDOWN} and \var{RECEIVED_SHUTDOWN}.
\end{methoddesc}

\begin{methoddesc}[Connection]{set_shutdown}{state}
Set the shutdown state of the Connection.  \var{state} is a bitvector of
either or both of \var{SENT_SHUTDOWN} and \var{RECEIVED_SHUTDOWN}.
\end{methoddesc}

\begin{methoddesc}[Connection]{sock_shutdown}{how}
Call the \method{shutdown} method of the underlying socket.
\end{methoddesc}

\begin{methoddesc}[Connection]{bio_shutdown}{}
If the Connection was created with a memory BIO, this method can be used to
indicate that ``end of file'' has been reached on the read end of that memory
BIO.
\end{methoddesc}

\begin{methoddesc}[Connection]{state_string}{}
Retrieve a verbose string detailing the state of the Connection.
\end{methoddesc}

\begin{methoddesc}[Connection]{client_random}{}
Retrieve the random value used with the client hello message.
\end{methoddesc}

\begin{methoddesc}[Connection]{server_random}{}
Retrieve the random value used with the server hello message.
\end{methoddesc}

\begin{methoddesc}[Connection]{master_key}{}
Retrieve the value of the master key for this session.
\end{methoddesc}

\begin{methoddesc}[Connection]{want_read}{}
Checks if more data has to be read from the transport layer to complete an
operation.
\end{methoddesc}

\begin{methoddesc}[Connection]{want_write}{}
Checks if there is data to write to the transport layer to complete an
operation.
\end{methoddesc}



\section{Internals \label{internals}}

We ran into three main problems developing this: Exceptions, callbacks and
accessing socket methods. This is what this chapter is about.

\subsection{Exceptions \label{exceptions}}

We realized early that most of the exceptions would be raised by the I/O
functions of OpenSSL, so it felt natural to mimic OpenSSL's error code system,
translating them into Python exceptions. This naturally gives us the exceptions
\exception{SSL.ZeroReturnError}, \exception{SSL.WantReadError},
\exception{SSL.WantWriteError}, \exception{SSL.WantX509LookupError} and
\exception{SSL.SysCallError}.

For more information about this, see section \ref{openssl-ssl}.


\subsection{Callbacks \label{callbacks}}

There are a number of problems with callbacks. First of all, OpenSSL is written
as a C library, it's not meant to have Python callbacks, so a way around that
is needed. Another problem is thread support. A lot of the OpenSSL I/O
functions can block if the socket is in blocking mode, and then you want other
Python threads to be able to do other things. The real trouble is if you've
released the global CPython interpreter lock to do a potentially blocking
operation, and the operation calls a callback. Then we must take the GIL back,
since calling Python APIs without holding it is not allowed.

There are two solutions to the first problem, both of which are necessary. The
first solution to use is if the C callback allows ''userdata'' to be passed to
it (an arbitrary pointer normally). This is great! We can set our Python
function object as the real userdata and emulate userdata for the Python
function in another way. The other solution can be used if an object with an
''app_data'' system always is passed to the callback. For example, the SSL
object in OpenSSL has app_data functions and in e.g. the verification
callbacks, you can retrieve the related SSL object. What we do is to set our
wrapper \class{Connection} object as app_data for the SSL object, and we can
easily find the Python callback.

The other problem is solved using thread local variables.  Whenever the GIL is
released before calling into an OpenSSL API, the PyThreadState pointer returned
by \cfunction{PyEval_SaveState} is stored in a global thread local variable
(using Python's own TLS API, \cfunction{PyThread_set_key_value}).  When it is
necessary to re-acquire the GIL, either after the OpenSSL API returns or in a C
callback invoked by that OpenSSL API, the value of the thread local variable is
retrieved (\cfunction{PyThread_get_key_value}) and used to re-acquire the GIL.
This allows Python threads to execute while OpenSSL APIs are running and allows
use of any particular pyOpenSSL object from any Python thread, since there is
no per-thread state associated with any of these objects and since OpenSSL is
threadsafe (as long as properly initialized, as pyOpenSSL initializes it).


\subsection{Acessing Socket Methods \label{socket-methods}}

We quickly saw the benefit of wrapping socket methods in the
\class{SSL.Connection} class, for an easy transition into using SSL. The
problem here is that the \module{socket} module lacks a C API, and all the
methods are declared static. One approach would be to have \module{OpenSSL} as
a submodule to the \module{socket} module, placing all the code in
\file{socketmodule.c}, but this is obviously not a good solution, since you
might not want to import tonnes of extra stuff you're not going to use when
importing the \module{socket} module. The other approach is to somehow get a
pointer to the method to be called, either the C function, or a callable Python
object. This is not really a good solution either, since there's a lot of
lookups involved.

The way it works is that you have to supply a ``\class{socket}-like'' transport
object to the \class{SSL.Connection}. The only requirement of this object is
that it has a \method{fileno()} method that returns a file descriptor that's
valid at the C level (i.e. you can use the system calls read and write). If you
want to use the \method{connect()} or \method{accept()} methods of the
\class{SSL.Connection} object, the transport object has to supply such
methods too. Apart from them, any method lookups in the \class{SSL.Connection}
object that fail are passed on to the underlying transport object.

Future changes might be to allow Python-level transport objects, that instead
of having \method{fileno()} methods, have \method{read()} and \method{write()}
methods, so more advanced features of Python can be used. This would probably
entail some sort of OpenSSL ``BIOs'', but converting Python strings back and
forth is expensive, so this shouldn't be used unless necessary. Other nice
things would be to be able to pass in different transport objects for reading
and writing, but then the \method{fileno()} method of \class{SSL.Connection}
becomes virtually useless. Also, should the method resolution be used on the
read-transport or the write-transport?


\end{document}