Merged revisions 55631-55794 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk
................
r55636 | neal.norwitz | 2007-05-29 00:06:39 -0700 (Tue, 29 May 2007) | 149 lines
Merged revisions 55506-55635 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r55507 | georg.brandl | 2007-05-22 07:28:17 -0700 (Tue, 22 May 2007) | 2 lines
Remove the "panel" module doc file which has been ignored since 1994.
........
r55522 | mark.hammond | 2007-05-22 19:04:28 -0700 (Tue, 22 May 2007) | 4 lines
Remove definition of PY_UNICODE_TYPE from pyconfig.h, allowing the
definition in unicodeobject.h to be used, giving us the desired
wchar_t in place of 'unsigned short'. As discussed on python-dev.
........
r55525 | neal.norwitz | 2007-05-22 23:35:32 -0700 (Tue, 22 May 2007) | 6 lines
Add -3 option to the interpreter to warn about features that are
deprecated and will be changed/removed in Python 3.0.
This patch is mostly from Anthony. I tweaked some format and added
a little doc.
........
r55527 | neal.norwitz | 2007-05-22 23:57:35 -0700 (Tue, 22 May 2007) | 1 line
Whitespace cleanup
........
r55528 | neal.norwitz | 2007-05-22 23:58:36 -0700 (Tue, 22 May 2007) | 1 line
Add a bunch more deprecation warnings for builtins that are going away in 3.0
........
r55549 | georg.brandl | 2007-05-24 09:49:29 -0700 (Thu, 24 May 2007) | 2 lines
shlex.split() now has an optional "posix" parameter.
........
r55550 | georg.brandl | 2007-05-24 10:33:33 -0700 (Thu, 24 May 2007) | 2 lines
Fix parameter passing.
........
r55555 | facundo.batista | 2007-05-24 10:50:54 -0700 (Thu, 24 May 2007) | 6 lines
Added an optional timeout parameter to urllib.ftpwrapper, with tests
(for this and a basic one, because there weren't any). Changed also
NEWS, but didn't find documentation for this function, assumed it
wasn't public...
........
r55563 | facundo.batista | 2007-05-24 13:01:59 -0700 (Thu, 24 May 2007) | 4 lines
Removed the .recv() in the test, is not necessary, and was
causing problems that didn't have anything to do with was
actually being tested...
........
r55564 | facundo.batista | 2007-05-24 13:51:19 -0700 (Thu, 24 May 2007) | 5 lines
Let's see if reading exactly what is written allow this live
test to pass (now I know why there were so few tests in ftp,
http, etc, :( ).
........
r55567 | facundo.batista | 2007-05-24 20:10:28 -0700 (Thu, 24 May 2007) | 4 lines
Trying to make the tests work in Windows and Solaris, everywhere
else just works
........
r55568 | facundo.batista | 2007-05-24 20:47:19 -0700 (Thu, 24 May 2007) | 4 lines
Fixing stupid error, and introducing a sleep, to see if the
other thread is awakened and finish sending data.
........
r55569 | facundo.batista | 2007-05-24 21:20:22 -0700 (Thu, 24 May 2007) | 4 lines
Commenting out the tests until find out who can test them in
one of the problematic enviroments.
........
r55570 | neal.norwitz | 2007-05-24 22:13:40 -0700 (Thu, 24 May 2007) | 2 lines
Get test passing again by commenting out the reference to the test class.
........
r55575 | vinay.sajip | 2007-05-25 00:05:59 -0700 (Fri, 25 May 2007) | 1 line
Updated docstring for SysLogHandler (#1720726).
........
r55576 | vinay.sajip | 2007-05-25 00:06:55 -0700 (Fri, 25 May 2007) | 1 line
Updated documentation for SysLogHandler (#1720726).
........
r55592 | brett.cannon | 2007-05-25 13:17:15 -0700 (Fri, 25 May 2007) | 3 lines
Remove direct call's to file's constructor and replace them with calls to
open() as ths is considered best practice.
........
r55601 | kristjan.jonsson | 2007-05-26 12:19:50 -0700 (Sat, 26 May 2007) | 1 line
Remove the rgbimgmodule from PCBuild8
........
r55602 | kristjan.jonsson | 2007-05-26 12:31:39 -0700 (Sat, 26 May 2007) | 1 line
Include <windows.h> after python.h, so that WINNT is properly set before windows.h is included. Fixes warnings in PC builds.
........
r55603 | walter.doerwald | 2007-05-26 14:04:13 -0700 (Sat, 26 May 2007) | 2 lines
Fix typo.
........
r55604 | peter.astrand | 2007-05-26 15:18:20 -0700 (Sat, 26 May 2007) | 1 line
Applied patch 1669481, slightly modified: Support close_fds on Win32
........
r55606 | neal.norwitz | 2007-05-26 21:08:54 -0700 (Sat, 26 May 2007) | 2 lines
Add the new function object attribute names from py3k.
........
r55617 | lars.gustaebel | 2007-05-27 12:49:30 -0700 (Sun, 27 May 2007) | 20 lines
Added errors argument to TarFile class that allows the user to
specify an error handling scheme for character conversion. Additional
scheme "utf-8" in read mode. Unicode input filenames are now
supported by design. The values of the pax_headers dictionary are now
limited to unicode objects.
Fixed: The prefix field is no longer used in PAX_FORMAT (in
conformance with POSIX).
Fixed: In read mode use a possible pax header size field.
Fixed: Strip trailing slashes from pax header name values.
Fixed: Give values in user-specified pax_headers precedence when
writing.
Added unicode tests. Added pax/regtype4 member to testtar.tar all
possible number fields in a pax header.
Added two chapters to the documentation about the different formats
tarfile.py supports and how unicode issues are handled.
........
r55618 | raymond.hettinger | 2007-05-27 22:23:22 -0700 (Sun, 27 May 2007) | 1 line
Explain when groupby() issues a new group.
........
r55634 | martin.v.loewis | 2007-05-28 21:01:29 -0700 (Mon, 28 May 2007) | 2 lines
Test pre-commit hook for a link to a .py file.
........
r55635 | martin.v.loewis | 2007-05-28 21:02:03 -0700 (Mon, 28 May 2007) | 2 lines
Revert 55634.
........
................
r55639 | neal.norwitz | 2007-05-29 00:58:11 -0700 (Tue, 29 May 2007) | 1 line
Remove sys.exc_{type,exc_value,exc_traceback}
................
r55641 | neal.norwitz | 2007-05-29 01:03:50 -0700 (Tue, 29 May 2007) | 1 line
Missed one sys.exc_type. I wonder why exc_{value,traceback} were already gone
................
r55642 | neal.norwitz | 2007-05-29 01:08:33 -0700 (Tue, 29 May 2007) | 1 line
Missed more doc for sys.exc_* attrs.
................
r55643 | neal.norwitz | 2007-05-29 01:18:19 -0700 (Tue, 29 May 2007) | 1 line
Remove sys.exc_clear()
................
r55665 | guido.van.rossum | 2007-05-29 19:45:43 -0700 (Tue, 29 May 2007) | 4 lines
Make None, True, False keywords.
We can now also delete all the other places that explicitly forbid
assignment to None, but I'm not going to bother right now.
................
r55666 | guido.van.rossum | 2007-05-29 20:01:51 -0700 (Tue, 29 May 2007) | 3 lines
Found another place that needs check for forbidden names.
Fixed test_syntax.py accordingly (it helped me find that one).
................
r55668 | guido.van.rossum | 2007-05-29 20:41:48 -0700 (Tue, 29 May 2007) | 2 lines
Mark None, True, False as keywords.
................
r55673 | neal.norwitz | 2007-05-29 23:28:25 -0700 (Tue, 29 May 2007) | 3 lines
Get the dis module working on modules again after changing dicts
to not return lists and also new-style classes. Add a test.
................
r55674 | neal.norwitz | 2007-05-29 23:35:45 -0700 (Tue, 29 May 2007) | 1 line
Umm, it helps to add the module that the test uses
................
r55675 | neal.norwitz | 2007-05-29 23:53:05 -0700 (Tue, 29 May 2007) | 4 lines
Try to fix up all the other places that were assigning to True/False.
There's at least one more problem in test.test_xmlrpc. I have other
changes in that file and that should be fixed soon (I hope).
................
r55679 | neal.norwitz | 2007-05-30 00:31:55 -0700 (Wed, 30 May 2007) | 1 line
Fix up another place that was assigning to True/False.
................
r55688 | brett.cannon | 2007-05-30 14:19:47 -0700 (Wed, 30 May 2007) | 2 lines
Ditch MimeWriter.
................
r55692 | brett.cannon | 2007-05-30 14:52:00 -0700 (Wed, 30 May 2007) | 2 lines
Remove the mimify module.
................
r55707 | guido.van.rossum | 2007-05-31 05:08:45 -0700 (Thu, 31 May 2007) | 2 lines
Backport the addition of show_code() to dis.py -- it's too handy.
................
r55708 | guido.van.rossum | 2007-05-31 06:22:57 -0700 (Thu, 31 May 2007) | 7 lines
Fix a fairly long-standing bug in the check for assignment to None (and other
keywords, these days). In 2.5, you could write foo(None=1) without getting
a SyntaxError (although foo()'s definition would have to use **kwds to avoid
getting a runtime error complaining about an unknown keyword of course).
This ought to be backported to 2.5.2 or at least 2.6.
................
r55724 | brett.cannon | 2007-05-31 19:32:41 -0700 (Thu, 31 May 2007) | 2 lines
Remove the cfmfile.
................
r55727 | neal.norwitz | 2007-05-31 22:19:44 -0700 (Thu, 31 May 2007) | 1 line
Remove reload() builtin.
................
r55729 | neal.norwitz | 2007-05-31 22:51:30 -0700 (Thu, 31 May 2007) | 59 lines
Merged revisions 55636-55728 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r55637 | georg.brandl | 2007-05-29 00:16:47 -0700 (Tue, 29 May 2007) | 2 lines
Fix rst markup.
........
r55638 | neal.norwitz | 2007-05-29 00:51:39 -0700 (Tue, 29 May 2007) | 1 line
Fix typo in doc
........
r55671 | neal.norwitz | 2007-05-29 21:53:41 -0700 (Tue, 29 May 2007) | 1 line
Fix indentation (whitespace only).
........
r55676 | thomas.heller | 2007-05-29 23:58:30 -0700 (Tue, 29 May 2007) | 1 line
Fix compiler warnings.
........
r55677 | thomas.heller | 2007-05-30 00:01:25 -0700 (Wed, 30 May 2007) | 2 lines
Correct the name of a field in the WIN32_FIND_DATAA and WIN32_FIND_DATAW structures.
Closes bug #1726026.
........
r55686 | brett.cannon | 2007-05-30 13:46:26 -0700 (Wed, 30 May 2007) | 2 lines
Have MimeWriter raise a DeprecationWarning as per PEP 4 and its documentation.
........
r55690 | brett.cannon | 2007-05-30 14:48:58 -0700 (Wed, 30 May 2007) | 3 lines
Have mimify raise a DeprecationWarning. The docs and PEP 4 have listed the
module as deprecated for a while.
........
r55696 | brett.cannon | 2007-05-30 15:24:28 -0700 (Wed, 30 May 2007) | 2 lines
Have md5 raise a DeprecationWarning as per PEP 4.
........
r55705 | neal.norwitz | 2007-05-30 21:14:22 -0700 (Wed, 30 May 2007) | 1 line
Add some spaces in the example code.
........
r55716 | brett.cannon | 2007-05-31 12:20:00 -0700 (Thu, 31 May 2007) | 2 lines
Have the sha module raise a DeprecationWarning as specified in PEP 4.
........
r55719 | brett.cannon | 2007-05-31 12:40:42 -0700 (Thu, 31 May 2007) | 2 lines
Cause buildtools to raise a DeprecationWarning.
........
r55721 | brett.cannon | 2007-05-31 13:01:11 -0700 (Thu, 31 May 2007) | 2 lines
Have cfmfile raise a DeprecationWarning as per PEP 4.
........
r55726 | neal.norwitz | 2007-05-31 21:56:47 -0700 (Thu, 31 May 2007) | 1 line
Mail if there is an installation failure.
........
................
r55730 | neal.norwitz | 2007-05-31 23:22:07 -0700 (Thu, 31 May 2007) | 2 lines
Remove the code that was missed in rev 55303.
................
r55738 | neal.norwitz | 2007-06-01 19:10:43 -0700 (Fri, 01 Jun 2007) | 1 line
Fix doc breakage
................
r55741 | neal.norwitz | 2007-06-02 00:41:58 -0700 (Sat, 02 Jun 2007) | 1 line
Remove timing module (plus some remnants of other modules).
................
r55742 | neal.norwitz | 2007-06-02 00:51:44 -0700 (Sat, 02 Jun 2007) | 1 line
Remove posixfile module (plus some remnants of other modules).
................
r55744 | neal.norwitz | 2007-06-02 10:18:56 -0700 (Sat, 02 Jun 2007) | 1 line
Fix doc breakage.
................
r55745 | neal.norwitz | 2007-06-02 11:32:16 -0700 (Sat, 02 Jun 2007) | 1 line
Make a whatsnew 3.0 template.
................
r55754 | neal.norwitz | 2007-06-03 23:24:18 -0700 (Sun, 03 Jun 2007) | 1 line
SF #1730441, os._execvpe raises UnboundLocal due to new try/except semantics
................
r55755 | neal.norwitz | 2007-06-03 23:26:00 -0700 (Sun, 03 Jun 2007) | 1 line
Get rid of extra whitespace
................
r55794 | guido.van.rossum | 2007-06-06 15:29:22 -0700 (Wed, 06 Jun 2007) | 3 lines
Make this compile in GCC 2.96, which does not allow interspersing
declarations and code.
................
diff --git a/Doc/lib/libtarfile.tex b/Doc/lib/libtarfile.tex
index 73c35ed..54683a7 100644
--- a/Doc/lib/libtarfile.tex
+++ b/Doc/lib/libtarfile.tex
@@ -133,24 +133,20 @@
\versionadded{2.6}
\end{excdesc}
+Each of the following constants defines a tar archive format that the
+\module{tarfile} module is able to create. See section \ref{tar-formats} for
+details.
+
\begin{datadesc}{USTAR_FORMAT}
- \POSIX{}.1-1988 (ustar) format. It supports filenames up to a length of
- at best 256 characters and linknames up to 100 characters. The maximum
- file size is 8 gigabytes. This is an old and limited but widely
- supported format.
+ \POSIX{}.1-1988 (ustar) format.
\end{datadesc}
\begin{datadesc}{GNU_FORMAT}
- GNU tar format. It supports arbitrarily long filenames and linknames and
- files bigger than 8 gigabytes. It is the defacto standard on GNU/Linux
- systems.
+ GNU tar format.
\end{datadesc}
\begin{datadesc}{PAX_FORMAT}
- \POSIX{}.1-2001 (pax) format. It is the most flexible format with
- virtually no limits. It supports long filenames and linknames, large files
- and stores pathnames in a portable way. However, not all tar
- implementations today are able to handle pax archives properly.
+ \POSIX{}.1-2001 (pax) format.
\end{datadesc}
\begin{datadesc}{DEFAULT_FORMAT}
@@ -175,15 +171,15 @@
The \class{TarFile} object provides an interface to a tar archive. A tar
archive is a sequence of blocks. An archive member (a stored file) is made up
-of a header block followed by data blocks. It is possible, to store a file in a
+of a header block followed by data blocks. It is possible to store a file in a
tar archive several times. Each archive member is represented by a
\class{TarInfo} object, see \citetitle{TarInfo Objects} (section
\ref{tarinfo-objects}) for details.
\begin{classdesc}{TarFile}{name=None, mode='r', fileobj=None,
format=DEFAULT_FORMAT, tarinfo=TarInfo, dereference=False,
- ignore_zeros=False, encoding=None, pax_headers=None, debug=0,
- errorlevel=0}
+ ignore_zeros=False, encoding=None, errors=None, pax_headers=None,
+ debug=0, errorlevel=0}
All following arguments are optional and can be accessed as instance
attributes as well.
@@ -231,18 +227,14 @@
If \code{2}, all \emph{non-fatal} errors are raised as \exception{TarError}
exceptions as well.
- The \var{encoding} argument defines the local character encoding. It
- defaults to the value from \function{sys.getfilesystemencoding()} or if
- that is \code{None} to \code{"ascii"}. \var{encoding} is used only in
- connection with the pax format which stores text data in \emph{UTF-8}. If
- it is not set correctly, character conversion will fail with a
- \exception{UnicodeError}.
+ The \var{encoding} and \var{errors} arguments control the way strings are
+ converted to unicode objects and vice versa. The default settings will work
+ for most users. See section \ref{tar-unicode} for in-depth information.
\versionadded{2.6}
- The \var{pax_headers} argument must be a dictionary whose elements are
- either unicode objects, numbers or strings that can be decoded to unicode
- using \var{encoding}. This information will be added to the archive as a
- pax global header.
+ The \var{pax_headers} argument is an optional dictionary of unicode strings
+ which will be added as a pax global header if \var{format} is
+ \constant{PAX_FORMAT}.
\versionadded{2.6}
\end{classdesc}
@@ -287,7 +279,7 @@
Extract all members from the archive to the current working directory
or directory \var{path}. If optional \var{members} is given, it must be
a subset of the list returned by \method{getmembers()}.
- Directory informations like owner, modification time and permissions are
+ Directory information like owner, modification time and permissions are
set after all members have been extracted. This is done to work around two
problems: A directory's modification time is reset each time a file is
created in it. And, if a directory's permissions do not allow writing,
@@ -365,6 +357,11 @@
\deprecated{2.6}{Use the \member{format} attribute instead.}
\end{memberdesc}
+\begin{memberdesc}{pax_headers}
+ A dictionary containing key-value pairs of pax global headers.
+ \versionadded{2.6}
+\end{memberdesc}
+
%-----------------
% TarInfo Objects
%-----------------
@@ -384,8 +381,8 @@
Create a \class{TarInfo} object.
\end{classdesc}
-\begin{methoddesc}{frombuf}{}
- Create and return a \class{TarInfo} object from a string buffer.
+\begin{methoddesc}{frombuf}{buf}
+ Create and return a \class{TarInfo} object from string buffer \var{buf}.
\versionadded[Raises \exception{HeaderError} if the buffer is
invalid.]{2.6}
\end{methoddesc}
@@ -396,10 +393,11 @@
\versionadded{2.6}
\end{methoddesc}
-\begin{methoddesc}{tobuf}{\optional{format}}
- Create a string buffer from a \class{TarInfo} object. See
- \class{TarFile}'s \member{format} argument for information.
- \versionchanged[The \var{format} parameter]{2.6}
+\begin{methoddesc}{tobuf}{\optional{format\optional{, encoding
+ \optional{, errors}}}}
+ Create a string buffer from a \class{TarInfo} object. For information
+ on the arguments see the constructor of the \class{TarFile} class.
+ \versionchanged[The arguments were added]{2.6}
\end{methoddesc}
A \code{TarInfo} object has the following public data attributes:
@@ -452,6 +450,12 @@
Group name.
\end{memberdesc}
+\begin{memberdesc}{pax_headers}
+ A dictionary containing key-value pairs of an associated pax
+ extended header.
+ \versionadded{2.6}
+\end{memberdesc}
+
A \class{TarInfo} object also provides some convenient query methods:
\begin{methoddesc}{isfile}{}
@@ -554,3 +558,103 @@
tar.extract(tarinfo)
tar.close()
\end{verbatim}
+
+%------------
+% Tar format
+%------------
+
+\subsection{Supported tar formats \label{tar-formats}}
+
+There are three tar formats that can be created with the \module{tarfile}
+module:
+
+\begin{itemize}
+
+\item
+The \POSIX{}.1-1988 ustar format (\constant{USTAR_FORMAT}). It supports
+filenames up to a length of at best 256 characters and linknames up to 100
+characters. The maximum file size is 8 gigabytes. This is an old and limited
+but widely supported format.
+
+\item
+The GNU tar format (\constant{GNU_FORMAT}). It supports long filenames and
+linknames, files bigger than 8 gigabytes and sparse files. It is the de facto
+standard on GNU/Linux systems. \module{tarfile} fully supports the GNU tar
+extensions for long names, sparse file support is read-only.
+
+\item
+The \POSIX{}.1-2001 pax format (\constant{PAX_FORMAT}). It is the most
+flexible format with virtually no limits. It supports long filenames and
+linknames, large files and stores pathnames in a portable way. However, not
+all tar implementations today are able to handle pax archives properly.
+
+The \emph{pax} format is an extension to the existing \emph{ustar} format. It
+uses extra headers for information that cannot be stored otherwise. There are
+two flavours of pax headers: Extended headers only affect the subsequent file
+header, global headers are valid for the complete archive and affect all
+following files. All the data in a pax header is encoded in \emph{UTF-8} for
+portability reasons.
+
+\end{itemize}
+
+There are some more variants of the tar format which can be read, but not
+created:
+
+\begin{itemize}
+
+\item
+The ancient V7 format. This is the first tar format from \UNIX{} Seventh
+Edition, storing only regular files and directories. Names must not be longer
+than 100 characters, there is no user/group name information. Some archives
+have miscalculated header checksums in case of fields with non-\ASCII{}
+characters.
+
+\item
+The SunOS tar extended format. This format is a variant of the \POSIX{}.1-2001
+pax format, but is not compatible.
+
+\end{itemize}
+
+%----------------
+% Unicode issues
+%----------------
+
+\subsection{Unicode issues \label{tar-unicode}}
+
+The tar format was originally conceived to make backups on tape drives with the
+main focus on preserving file system information. Nowadays tar archives are
+commonly used for file distribution and exchanging archives over networks. One
+problem of the original format (that all other formats are merely variants of)
+is that there is no concept of supporting different character encodings.
+For example, an ordinary tar archive created on a \emph{UTF-8} system cannot be
+read correctly on a \emph{Latin-1} system if it contains non-\ASCII{}
+characters. Names (i.e. filenames, linknames, user/group names) containing
+these characters will appear damaged. Unfortunately, there is no way to
+autodetect the encoding of an archive.
+
+The pax format was designed to solve this problem. It stores non-\ASCII{} names
+using the universal character encoding \emph{UTF-8}. When a pax archive is
+read, these \emph{UTF-8} names are converted to the encoding of the local
+file system.
+
+The details of unicode conversion are controlled by the \var{encoding} and
+\var{errors} keyword arguments of the \class{TarFile} class.
+
+The default value for \var{encoding} is the local character encoding. It is
+deduced from \function{sys.getfilesystemencoding()} and
+\function{sys.getdefaultencoding()}. In read mode, \var{encoding} is used
+exclusively to convert unicode names from a pax archive to strings in the local
+character encoding. In write mode, the use of \var{encoding} depends on the
+chosen archive format. In case of \constant{PAX_FORMAT}, input names that
+contain non-\ASCII{} characters need to be decoded before being stored as
+\emph{UTF-8} strings. The other formats do not make use of \var{encoding}
+unless unicode objects are used as input names. These are converted to
+8-bit character strings before they are added to the archive.
+
+The \var{errors} argument defines how characters are treated that cannot be
+converted to or from \var{encoding}. Possible values are listed in section
+\ref{codec-base-classes}. In read mode, there is an additional scheme
+\code{'utf-8'} which means that bad characters are replaced by their
+\emph{UTF-8} representation. This is the default scheme. In write mode the
+default value for \var{errors} is \code{'strict'} to ensure that name
+information is not altered unnoticed.