SF patch 852995: add processors feature to urllib2
John J. Lee writes: "the patch makes it possible to implement
functionality like HTTP cookie handling, Refresh handling,
etc. etc. using handler objects. At the moment urllib2's handler
objects aren't quite up to the job, which results in a lot of
cut-n-paste and subclassing. I believe the changes are
backwards-compatible, with the exception of people who've
reimplemented build_opener()'s functionality -- those people would
need to call opener.add_handler(HTTPErrorProcessor).
The main change is allowing handlers to implement
methods like:
http_request(request)
http_response(request, response)
In addition to the usual
http_open(request)
http_error{_*}(...)
"
Note that the change isn't well documented at least in part because
handlers aren't well documented at all. Need to fix this.
Add a bunch of new tests. It appears that none of these tests
actually use the network, so they don't need to be guarded by a
resource flag.
diff --git a/Doc/lib/liburllib2.tex b/Doc/lib/liburllib2.tex
index 3f8ff3a..0020b6c 100644
--- a/Doc/lib/liburllib2.tex
+++ b/Doc/lib/liburllib2.tex
@@ -52,7 +52,7 @@
them, instances of them or subclasses of them:
\class{ProxyHandler}, \class{UnknownHandler}, \class{HTTPHandler},
\class{HTTPDefaultErrorHandler}, \class{HTTPRedirectHandler},
-\class{FTPHandler}, \class{FileHandler}
+\class{FTPHandler}, \class{FileHandler}, \class{HTTPErrorProcessor}.
If the Python installation has SSL support (\function{socket.ssl()}
exists), \class{HTTPSHandler} will also be added.
@@ -248,6 +248,15 @@
same functionality using only one header.
\end{methoddesc}
+\begin{methoddesc}[Request]{add_unredirected_header}{key, header}
+Add a header that will not be added to a redirected request.
+\end{methoddesc}
+
+\begin{methoddesc}[Request]{has_header}{header}
+Return whether the instance has the named header (checks both regular
+and unredirected).
+\end{methoddesc}
+
\begin{methoddesc}[Request]{get_full_url}{}
Return the URL given in the constructor.
\end{methoddesc}
@@ -286,6 +295,12 @@
\item \method{\var{protocol}_error_\var{type}()} ---
signal that the handler knows how to handle \var{type} errors from
\var{protocol}.
+ \item \method{\var{protocol}_request()} ---
+ signal that the handler knows how to pre-process \var{protocol}
+ requests.
+ \item \method{\var{protocol}_response()} ---
+ signal that the handler knows how to post-process \var{protocol}
+ responses.
\end{itemize}
\end{methoddesc}
@@ -620,6 +635,21 @@
\end{methoddesc}
+\subsection{HTTPErrorProcessor Objects \label{http-error-processor-objects}}
+
+\begin{methoddesc}[HTTPErrorProcessor]{unknown_open}{}
+Process HTTP error responses.
+
+For 200 error codes, the response object is returned immediately.
+
+For non-200 error codes, this simply passes the job on to the
+\method{\var{protocol}_error_\var{code}()} handler methods, via
+\method{OpenerDirector.error()}. Eventually,
+\class{urllib2.HTTPDefaultErrorHandler} will raise an
+\exception{HTTPError} if no other handler handles the error.
+\end{methoddesc}
+
+
\subsection{Examples \label{urllib2-examples}}
This example gets the python.org main page and displays the first 100