Merged revisions 85572-85573,85606,85609-85622,85624,85626-85627,85629,85631,85633,85635-85636,85638-85639,85641-85642 via svnmerge from
svn+ssh://svn.python.org/python/branches/py3k
........
r85572 | georg.brandl | 2010-10-16 20:51:05 +0200 (Sa, 16 Okt 2010) | 1 line
#10122: typo fix.
........
r85573 | georg.brandl | 2010-10-16 20:53:08 +0200 (Sa, 16 Okt 2010) | 1 line
#10124: typo fix.
........
r85606 | georg.brandl | 2010-10-17 08:32:59 +0200 (So, 17 Okt 2010) | 1 line
#10058: tweak wording about exception returns.
........
r85609 | georg.brandl | 2010-10-17 11:19:03 +0200 (So, 17 Okt 2010) | 1 line
#8556: use less confusing mapping key in example.
........
r85610 | georg.brandl | 2010-10-17 11:23:05 +0200 (So, 17 Okt 2010) | 1 line
#8686: remove potentially confusing wording that does not add any value.
........
r85611 | georg.brandl | 2010-10-17 11:33:24 +0200 (So, 17 Okt 2010) | 1 line
#8811: small fixes to sqlite3 docs.
........
r85612 | georg.brandl | 2010-10-17 11:37:54 +0200 (So, 17 Okt 2010) | 1 line
#8855: add shelve security warning.
........
r85613 | georg.brandl | 2010-10-17 11:43:35 +0200 (So, 17 Okt 2010) | 1 line
Fix hmac docs: it takes and returns bytes, except for hexdigest().
........
r85614 | georg.brandl | 2010-10-17 11:46:11 +0200 (So, 17 Okt 2010) | 1 line
#8968: add actual name of token constants.
........
r85615 | georg.brandl | 2010-10-17 12:05:13 +0200 (So, 17 Okt 2010) | 1 line
#459007: merge info from PC/getpathp.c and using/windows.rst to document the forming of sys.path under Windows.
........
r85616 | georg.brandl | 2010-10-17 12:07:29 +0200 (So, 17 Okt 2010) | 1 line
Fix copy-paste error in example.
........
r85617 | georg.brandl | 2010-10-17 12:09:06 +0200 (So, 17 Okt 2010) | 1 line
#5212: md5 weaknesses do not affect hmac, so remove the note about that.
........
r85618 | georg.brandl | 2010-10-17 12:14:38 +0200 (So, 17 Okt 2010) | 1 line
#9086: correct wrong terminology about linking with pythonXY.dll.
........
r85619 | georg.brandl | 2010-10-17 12:15:50 +0200 (So, 17 Okt 2010) | 1 line
Make file names consistent.
........
r85620 | georg.brandl | 2010-10-17 12:22:28 +0200 (So, 17 Okt 2010) | 1 line
Remove second parser module example; it referred to non-readily-available example files, and this kind of discovery is much better done with the AST nowadays anyway.
........
r85621 | georg.brandl | 2010-10-17 12:24:54 +0200 (So, 17 Okt 2010) | 1 line
#9105: move pickle warning to a bit more prominent location.
........
r85622 | georg.brandl | 2010-10-17 12:28:04 +0200 (So, 17 Okt 2010) | 1 line
#9112: document error() and exit() methods of ArgumentParser.
........
r85624 | georg.brandl | 2010-10-17 12:34:28 +0200 (So, 17 Okt 2010) | 1 line
Some markup and style fixes in argparse docs.
........
r85626 | georg.brandl | 2010-10-17 12:38:20 +0200 (So, 17 Okt 2010) | 1 line
#9117: fix syntax for class definition.
........
r85627 | georg.brandl | 2010-10-17 12:44:11 +0200 (So, 17 Okt 2010) | 1 line
#9138: reword introduction to classes in Python.
........
r85629 | georg.brandl | 2010-10-17 12:51:45 +0200 (So, 17 Okt 2010) | 1 line
#5962: clarify sys.exit() vs. threads.
........
r85631 | georg.brandl | 2010-10-17 12:53:54 +0200 (So, 17 Okt 2010) | 1 line
Fix capitalization.
........
r85633 | georg.brandl | 2010-10-17 12:59:41 +0200 (So, 17 Okt 2010) | 1 line
#9204: remove mentions of removed types in the types module.
........
r85635 | georg.brandl | 2010-10-17 13:03:22 +0200 (So, 17 Okt 2010) | 1 line
#5121: fix claims about default values leading to segfaults.
........
r85636 | georg.brandl | 2010-10-17 13:06:14 +0200 (So, 17 Okt 2010) | 1 line
#9237: document sys.call_tracing().
........
r85638 | georg.brandl | 2010-10-17 13:13:37 +0200 (So, 17 Okt 2010) | 1 line
Port changes to pickle docs apparently lost in py3k.
........
r85639 | georg.brandl | 2010-10-17 13:23:56 +0200 (So, 17 Okt 2010) | 1 line
Make twisted example a bit more logical.
........
r85641 | georg.brandl | 2010-10-17 13:29:07 +0200 (So, 17 Okt 2010) | 1 line
Fix documentation of dis.opmap direction.
........
r85642 | georg.brandl | 2010-10-17 13:36:28 +0200 (So, 17 Okt 2010) | 1 line
#9730: fix example.
........
diff --git a/Doc/library/audioop.rst b/Doc/library/audioop.rst
index d50a970..198c11a 100644
--- a/Doc/library/audioop.rst
+++ b/Doc/library/audioop.rst
@@ -230,8 +230,8 @@
def mul_stereo(sample, width, lfactor, rfactor):
lsample = audioop.tomono(sample, width, 1, 0)
rsample = audioop.tomono(sample, width, 0, 1)
- lsample = audioop.mul(sample, width, lfactor)
- rsample = audioop.mul(sample, width, rfactor)
+ lsample = audioop.mul(lsample, width, lfactor)
+ rsample = audioop.mul(rsample, width, rfactor)
lsample = audioop.tostereo(lsample, width, 1, 0)
rsample = audioop.tostereo(rsample, width, 0, 1)
return audioop.add(lsample, rsample, width)
diff --git a/Doc/library/base64.rst b/Doc/library/base64.rst
index e282ea0..0802e6c 100644
--- a/Doc/library/base64.rst
+++ b/Doc/library/base64.rst
@@ -157,12 +157,12 @@
An example usage of the module:
>>> import base64
- >>> encoded = base64.b64encode('data to be encoded')
+ >>> encoded = base64.b64encode(b'data to be encoded')
>>> encoded
b'ZGF0YSB0byBiZSBlbmNvZGVk'
>>> data = base64.b64decode(encoded)
>>> data
- 'data to be encoded'
+ b'data to be encoded'
.. seealso::
diff --git a/Doc/library/difflib.rst b/Doc/library/difflib.rst
index c9d1eee..3fd44a6 100644
--- a/Doc/library/difflib.rst
+++ b/Doc/library/difflib.rst
@@ -500,16 +500,11 @@
Return an upper bound on :meth:`ratio` relatively quickly.
- This isn't defined beyond that it is an upper bound on :meth:`ratio`, and
- is faster to compute.
-
.. method:: real_quick_ratio()
Return an upper bound on :meth:`ratio` very quickly.
- This isn't defined beyond that it is an upper bound on :meth:`ratio`, and
- is faster to compute than either :meth:`ratio` or :meth:`quick_ratio`.
The three methods that return the ratio of matching to total characters can give
different results due to differing levels of approximation, although
diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst
index 39c11f3..8a1b935 100644
--- a/Doc/library/dis.rst
+++ b/Doc/library/dis.rst
@@ -93,7 +93,7 @@
.. data:: opmap
- Dictionary mapping bytecodes to operation names.
+ Dictionary mapping operation names to bytecodes.
.. data:: cmp_op
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
index 4a19de0..1547f6d 100644
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -466,7 +466,7 @@
.. function:: getattr(object, name[, default])
- Return the value of the named attributed of *object*. *name* must be a string.
+ Return the value of the named attribute of *object*. *name* must be a string.
If the string is the name of one of the object's attributes, the result is the
value of that attribute. For example, ``getattr(x, 'foobar')`` is equivalent to
``x.foobar``. If the named attribute does not exist, *default* is returned if
diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst
index 6ffd00a..b2bd98d 100644
--- a/Doc/library/hmac.rst
+++ b/Doc/library/hmac.rst
@@ -2,7 +2,8 @@
========================================================
.. module:: hmac
- :synopsis: Keyed-Hashing for Message Authentication (HMAC) implementation for Python.
+ :synopsis: Keyed-Hashing for Message Authentication (HMAC) implementation
+ for Python.
.. moduleauthor:: Gerhard Häring <ghaering@users.sourceforge.net>
.. sectionauthor:: Gerhard Häring <ghaering@users.sourceforge.net>
@@ -12,37 +13,34 @@
.. function:: new(key, msg=None, digestmod=None)
- Return a new hmac object. If *msg* is present, the method call ``update(msg)``
- is made. *digestmod* is the digest constructor or module for the HMAC object to
- use. It defaults to the :func:`hashlib.md5` constructor.
+ Return a new hmac object. *key* is a bytes object giving the secret key. If
+ *msg* is present, the method call ``update(msg)`` is made. *digestmod* is
+ the digest constructor or module for the HMAC object to use. It defaults to
+ the :func:`hashlib.md5` constructor.
- .. note::
-
- The md5 hash has known weaknesses but remains the default for backwards
- compatibility. Choose a better one for your application.
An HMAC object has the following methods:
-
.. method:: hmac.update(msg)
- Update the hmac object with the string *msg*. Repeated calls are equivalent to
- a single call with the concatenation of all the arguments: ``m.update(a);
- m.update(b)`` is equivalent to ``m.update(a + b)``.
+ Update the hmac object with the bytes object *msg*. Repeated calls are
+ equivalent to a single call with the concatenation of all the arguments:
+ ``m.update(a); m.update(b)`` is equivalent to ``m.update(a + b)``.
.. method:: hmac.digest()
- Return the digest of the strings passed to the :meth:`update` method so far.
- This string will be the same length as the *digest_size* of the digest given to
- the constructor. It may contain non-ASCII characters, including NUL bytes.
+ Return the digest of the bytes passed to the :meth:`update` method so far.
+ This bytes object will be the same length as the *digest_size* of the digest
+ given to the constructor. It may contain non-ASCII bytes, including NUL
+ bytes.
.. method:: hmac.hexdigest()
- Like :meth:`digest` except the digest is returned as a string twice the length
- containing only hexadecimal digits. This may be used to exchange the value
- safely in email or other non-binary environments.
+ Like :meth:`digest` except the digest is returned as a string twice the
+ length containing only hexadecimal digits. This may be used to exchange the
+ value safely in email or other non-binary environments.
.. method:: hmac.copy()
@@ -55,4 +53,3 @@
Module :mod:`hashlib`
The Python module providing secure hash functions.
-
diff --git a/Doc/library/os.rst b/Doc/library/os.rst
index 371a902..910291f 100644
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -1421,15 +1421,15 @@
.. function:: _exit(n)
- Exit to the system with status *n*, without calling cleanup handlers, flushing
+ Exit the process with status *n*, without calling cleanup handlers, flushing
stdio buffers, etc.
Availability: Unix, Windows.
.. note::
- The standard way to exit is ``sys.exit(n)``. :func:`_exit` should normally only
- be used in the child process after a :func:`fork`.
+ The standard way to exit is ``sys.exit(n)``. :func:`_exit` should
+ normally only be used in the child process after a :func:`fork`.
The following exit codes are defined and can be used with :func:`_exit`,
although they are not required. These are typically used for system programs
diff --git a/Doc/library/parser.rst b/Doc/library/parser.rst
index b79e8d3..b69d1b9 100644
--- a/Doc/library/parser.rst
+++ b/Doc/library/parser.rst
@@ -318,22 +318,8 @@
Same as ``st2tuple(st, line_info)``.
-.. _st-examples:
-
-Examples
---------
-
-.. index:: builtin: compile
-
-The parser modules allows operations to be performed on the parse tree of Python
-source code before the :term:`bytecode` is generated, and provides for inspection of the
-parse tree for information gathering purposes. Two examples are presented. The
-simple example demonstrates emulation of the :func:`compile` built-in function
-and the complex example shows the use of a parse tree for information discovery.
-
-
-Emulation of :func:`compile`
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+Example: Emulation of :func:`compile`
+-------------------------------------
While many useful operations may take place between parsing and bytecode
generation, the simplest operation is to do nothing. For this purpose, using
@@ -367,320 +353,3 @@
def load_expression(source_string):
st = parser.expr(source_string)
return st, st.compile()
-
-
-Information Discovery
-^^^^^^^^^^^^^^^^^^^^^
-
-.. index::
- single: string; documentation
- single: docstrings
-
-Some applications benefit from direct access to the parse tree. The remainder
-of this section demonstrates how the parse tree provides access to module
-documentation defined in docstrings without requiring that the code being
-examined be loaded into a running interpreter via :keyword:`import`. This can
-be very useful for performing analyses of untrusted code.
-
-Generally, the example will demonstrate how the parse tree may be traversed to
-distill interesting information. Two functions and a set of classes are
-developed which provide programmatic access to high level function and class
-definitions provided by a module. The classes extract information from the
-parse tree and provide access to the information at a useful semantic level, one
-function provides a simple low-level pattern matching capability, and the other
-function defines a high-level interface to the classes by handling file
-operations on behalf of the caller. All source files mentioned here which are
-not part of the Python installation are located in the :file:`Demo/parser/`
-directory of the distribution.
-
-The dynamic nature of Python allows the programmer a great deal of flexibility,
-but most modules need only a limited measure of this when defining classes,
-functions, and methods. In this example, the only definitions that will be
-considered are those which are defined in the top level of their context, e.g.,
-a function defined by a :keyword:`def` statement at column zero of a module, but
-not a function defined within a branch of an :keyword:`if` ... :keyword:`else`
-construct, though there are some good reasons for doing so in some situations.
-Nesting of definitions will be handled by the code developed in the example.
-
-To construct the upper-level extraction methods, we need to know what the parse
-tree structure looks like and how much of it we actually need to be concerned
-about. Python uses a moderately deep parse tree so there are a large number of
-intermediate nodes. It is important to read and understand the formal grammar
-used by Python. This is specified in the file :file:`Grammar/Grammar` in the
-distribution. Consider the simplest case of interest when searching for
-docstrings: a module consisting of a docstring and nothing else. (See file
-:file:`docstring.py`.) ::
-
- """Some documentation.
- """
-
-Using the interpreter to take a look at the parse tree, we find a bewildering
-mass of numbers and parentheses, with the documentation buried deep in nested
-tuples. ::
-
- >>> import parser
- >>> import pprint
- >>> st = parser.suite(open('docstring.py').read())
- >>> tup = st.totuple()
- >>> pprint.pprint(tup)
- (257,
- (264,
- (265,
- (266,
- (267,
- (307,
- (287,
- (288,
- (289,
- (290,
- (292,
- (293,
- (294,
- (295,
- (296,
- (297,
- (298,
- (299,
- (300, (3, '"""Some documentation.\n"""'))))))))))))))))),
- (4, ''))),
- (4, ''),
- (0, ''))
-
-The numbers at the first element of each node in the tree are the node types;
-they map directly to terminal and non-terminal symbols in the grammar.
-Unfortunately, they are represented as integers in the internal representation,
-and the Python structures generated do not change that. However, the
-:mod:`symbol` and :mod:`token` modules provide symbolic names for the node types
-and dictionaries which map from the integers to the symbolic names for the node
-types.
-
-In the output presented above, the outermost tuple contains four elements: the
-integer ``257`` and three additional tuples. Node type ``257`` has the symbolic
-name :const:`file_input`. Each of these inner tuples contains an integer as the
-first element; these integers, ``264``, ``4``, and ``0``, represent the node
-types :const:`stmt`, :const:`NEWLINE`, and :const:`ENDMARKER`, respectively.
-Note that these values may change depending on the version of Python you are
-using; consult :file:`symbol.py` and :file:`token.py` for details of the
-mapping. It should be fairly clear that the outermost node is related primarily
-to the input source rather than the contents of the file, and may be disregarded
-for the moment. The :const:`stmt` node is much more interesting. In
-particular, all docstrings are found in subtrees which are formed exactly as
-this node is formed, with the only difference being the string itself. The
-association between the docstring in a similar tree and the defined entity
-(class, function, or module) which it describes is given by the position of the
-docstring subtree within the tree defining the described structure.
-
-By replacing the actual docstring with something to signify a variable component
-of the tree, we allow a simple pattern matching approach to check any given
-subtree for equivalence to the general pattern for docstrings. Since the
-example demonstrates information extraction, we can safely require that the tree
-be in tuple form rather than list form, allowing a simple variable
-representation to be ``['variable_name']``. A simple recursive function can
-implement the pattern matching, returning a Boolean and a dictionary of variable
-name to value mappings. (See file :file:`example.py`.) ::
-
- def match(pattern, data, vars=None):
- if vars is None:
- vars = {}
- if isinstance(pattern, list):
- vars[pattern[0]] = data
- return True, vars
- if not instance(pattern, tuple):
- return (pattern == data), vars
- if len(data) != len(pattern):
- return False, vars
- for pattern, data in zip(pattern, data):
- same, vars = match(pattern, data, vars)
- if not same:
- break
- return same, vars
-
-Using this simple representation for syntactic variables and the symbolic node
-types, the pattern for the candidate docstring subtrees becomes fairly readable.
-(See file :file:`example.py`.) ::
-
- import symbol
- import token
-
- DOCSTRING_STMT_PATTERN = (
- symbol.stmt,
- (symbol.simple_stmt,
- (symbol.small_stmt,
- (symbol.expr_stmt,
- (symbol.testlist,
- (symbol.test,
- (symbol.and_test,
- (symbol.not_test,
- (symbol.comparison,
- (symbol.expr,
- (symbol.xor_expr,
- (symbol.and_expr,
- (symbol.shift_expr,
- (symbol.arith_expr,
- (symbol.term,
- (symbol.factor,
- (symbol.power,
- (symbol.atom,
- (token.STRING, ['docstring'])
- )))))))))))))))),
- (token.NEWLINE, '')
- ))
-
-Using the :func:`match` function with this pattern, extracting the module
-docstring from the parse tree created previously is easy::
-
- >>> found, vars = match(DOCSTRING_STMT_PATTERN, tup[1])
- >>> found
- True
- >>> vars
- {'docstring': '"""Some documentation.\n"""'}
-
-Once specific data can be extracted from a location where it is expected, the
-question of where information can be expected needs to be answered. When
-dealing with docstrings, the answer is fairly simple: the docstring is the first
-:const:`stmt` node in a code block (:const:`file_input` or :const:`suite` node
-types). A module consists of a single :const:`file_input` node, and class and
-function definitions each contain exactly one :const:`suite` node. Classes and
-functions are readily identified as subtrees of code block nodes which start
-with ``(stmt, (compound_stmt, (classdef, ...`` or ``(stmt, (compound_stmt,
-(funcdef, ...``. Note that these subtrees cannot be matched by :func:`match`
-since it does not support multiple sibling nodes to match without regard to
-number. A more elaborate matching function could be used to overcome this
-limitation, but this is sufficient for the example.
-
-Given the ability to determine whether a statement might be a docstring and
-extract the actual string from the statement, some work needs to be performed to
-walk the parse tree for an entire module and extract information about the names
-defined in each context of the module and associate any docstrings with the
-names. The code to perform this work is not complicated, but bears some
-explanation.
-
-The public interface to the classes is straightforward and should probably be
-somewhat more flexible. Each "major" block of the module is described by an
-object providing several methods for inquiry and a constructor which accepts at
-least the subtree of the complete parse tree which it represents. The
-:class:`ModuleInfo` constructor accepts an optional *name* parameter since it
-cannot otherwise determine the name of the module.
-
-The public classes include :class:`ClassInfo`, :class:`FunctionInfo`, and
-:class:`ModuleInfo`. All objects provide the methods :meth:`get_name`,
-:meth:`get_docstring`, :meth:`get_class_names`, and :meth:`get_class_info`. The
-:class:`ClassInfo` objects support :meth:`get_method_names` and
-:meth:`get_method_info` while the other classes provide
-:meth:`get_function_names` and :meth:`get_function_info`.
-
-Within each of the forms of code block that the public classes represent, most
-of the required information is in the same form and is accessed in the same way,
-with classes having the distinction that functions defined at the top level are
-referred to as "methods." Since the difference in nomenclature reflects a real
-semantic distinction from functions defined outside of a class, the
-implementation needs to maintain the distinction. Hence, most of the
-functionality of the public classes can be implemented in a common base class,
-:class:`SuiteInfoBase`, with the accessors for function and method information
-provided elsewhere. Note that there is only one class which represents function
-and method information; this parallels the use of the :keyword:`def` statement
-to define both types of elements.
-
-Most of the accessor functions are declared in :class:`SuiteInfoBase` and do not
-need to be overridden by subclasses. More importantly, the extraction of most
-information from a parse tree is handled through a method called by the
-:class:`SuiteInfoBase` constructor. The example code for most of the classes is
-clear when read alongside the formal grammar, but the method which recursively
-creates new information objects requires further examination. Here is the
-relevant part of the :class:`SuiteInfoBase` definition from :file:`example.py`::
-
- class SuiteInfoBase:
- _docstring = ''
- _name = ''
-
- def __init__(self, tree = None):
- self._class_info = {}
- self._function_info = {}
- if tree:
- self._extract_info(tree)
-
- def _extract_info(self, tree):
- # extract docstring
- if len(tree) == 2:
- found, vars = match(DOCSTRING_STMT_PATTERN[1], tree[1])
- else:
- found, vars = match(DOCSTRING_STMT_PATTERN, tree[3])
- if found:
- self._docstring = eval(vars['docstring'])
- # discover inner definitions
- for node in tree[1:]:
- found, vars = match(COMPOUND_STMT_PATTERN, node)
- if found:
- cstmt = vars['compound']
- if cstmt[0] == symbol.funcdef:
- name = cstmt[2][1]
- self._function_info[name] = FunctionInfo(cstmt)
- elif cstmt[0] == symbol.classdef:
- name = cstmt[2][1]
- self._class_info[name] = ClassInfo(cstmt)
-
-After initializing some internal state, the constructor calls the
-:meth:`_extract_info` method. This method performs the bulk of the information
-extraction which takes place in the entire example. The extraction has two
-distinct phases: the location of the docstring for the parse tree passed in, and
-the discovery of additional definitions within the code block represented by the
-parse tree.
-
-The initial :keyword:`if` test determines whether the nested suite is of the
-"short form" or the "long form." The short form is used when the code block is
-on the same line as the definition of the code block, as in ::
-
- def square(x): "Square an argument."; return x ** 2
-
-while the long form uses an indented block and allows nested definitions::
-
- def make_power(exp):
- "Make a function that raises an argument to the exponent `exp`."
- def raiser(x, y=exp):
- return x ** y
- return raiser
-
-When the short form is used, the code block may contain a docstring as the
-first, and possibly only, :const:`small_stmt` element. The extraction of such a
-docstring is slightly different and requires only a portion of the complete
-pattern used in the more common case. As implemented, the docstring will only
-be found if there is only one :const:`small_stmt` node in the
-:const:`simple_stmt` node. Since most functions and methods which use the short
-form do not provide a docstring, this may be considered sufficient. The
-extraction of the docstring proceeds using the :func:`match` function as
-described above, and the value of the docstring is stored as an attribute of the
-:class:`SuiteInfoBase` object.
-
-After docstring extraction, a simple definition discovery algorithm operates on
-the :const:`stmt` nodes of the :const:`suite` node. The special case of the
-short form is not tested; since there are no :const:`stmt` nodes in the short
-form, the algorithm will silently skip the single :const:`simple_stmt` node and
-correctly not discover any nested definitions.
-
-Each statement in the code block is categorized as a class definition, function
-or method definition, or something else. For the definition statements, the
-name of the element defined is extracted and a representation object appropriate
-to the definition is created with the defining subtree passed as an argument to
-the constructor. The representation objects are stored in instance variables
-and may be retrieved by name using the appropriate accessor methods.
-
-The public classes provide any accessors required which are more specific than
-those provided by the :class:`SuiteInfoBase` class, but the real extraction
-algorithm remains common to all forms of code blocks. A high-level function can
-be used to extract the complete set of information from a source file. (See
-file :file:`example.py`.) ::
-
- def get_docs(fileName):
- import os
- import parser
-
- source = open(fileName).read()
- basename = os.path.basename(os.path.splitext(fileName)[0])
- st = parser.suite(source)
- return ModuleInfo(st.totuple(), basename)
-
-This provides an easy-to-use interface to the documentation of a module. If
-information is required which is not extracted by the code of this example, the
-code may be extended at clearly defined points to provide additional
-capabilities.
-
diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst
index 3358053..8a671a7 100644
--- a/Doc/library/pickle.rst
+++ b/Doc/library/pickle.rst
@@ -23,6 +23,12 @@
"serialization", "marshalling," [#]_ or "flattening", however, to avoid
confusion, the terms used here are "pickling" and "unpickling"..
+.. warning::
+
+ The :mod:`pickle` module is not intended to be secure against erroneous or
+ maliciously constructed data. Never unpickle data received from an untrusted
+ or unauthenticated source.
+
Relationship to other Python modules
------------------------------------
@@ -63,12 +69,6 @@
The :mod:`pickle` serialization format is guaranteed to be backwards compatible
across Python releases.
-.. warning::
-
- The :mod:`pickle` module is not intended to be secure against erroneous or
- maliciously constructed data. Never unpickle data received from an untrusted
- or unauthenticated source.
-
Note that serialization is a more primitive notion than persistence; although
:mod:`pickle` reads and writes file objects, it does not handle the issue of
naming persistent objects, nor the (even more complicated) issue of concurrent
@@ -427,33 +427,38 @@
obj.__dict__.update(attributes)
return obj
-.. index:: single: __getnewargs__() (copy protocol)
-
Classes can alter the default behaviour by providing one or several special
-methods. In protocol 2 and newer, classes that implements the
-:meth:`__getnewargs__` method can dictate the values passed to the
-:meth:`__new__` method upon unpickling. This is often needed for classes
-whose :meth:`__new__` method requires arguments.
+methods:
-.. index:: single: __getstate__() (copy protocol)
+.. method:: object.__getnewargs__()
-Classes can further influence how their instances are pickled; if the class
-defines the method :meth:`__getstate__`, it is called and the returned object is
-pickled as the contents for the instance, instead of the contents of the
-instance's dictionary. If the :meth:`__getstate__` method is absent, the
-instance's :attr:`__dict__` is pickled as usual.
+ In protocol 2 and newer, classes that implements the :meth:`__getnewargs__`
+ method can dictate the values passed to the :meth:`__new__` method upon
+ unpickling. This is often needed for classes whose :meth:`__new__` method
+ requires arguments.
-.. index:: single: __setstate__() (copy protocol)
-Upon unpickling, if the class defines :meth:`__setstate__`, it is called with
-the unpickled state. In that case, there is no requirement for the state object
-to be a dictionary. Otherwise, the pickled state must be a dictionary and its
-items are assigned to the new instance's dictionary.
+.. method:: object.__getstate__()
-.. note::
+ Classes can further influence how their instances are pickled; if the class
+ defines the method :meth:`__getstate__`, it is called and the returned object
+ is pickled as the contents for the instance, instead of the contents of the
+ instance's dictionary. If the :meth:`__getstate__` method is absent, the
+ instance's :attr:`__dict__` is pickled as usual.
- If :meth:`__getstate__` returns a false value, the :meth:`__setstate__`
- method will not be called.
+
+.. method:: object.__setstate__(state)
+
+ Upon unpickling, if the class defines :meth:`__setstate__`, it is called with
+ the unpickled state. In that case, there is no requirement for the state
+ object to be a dictionary. Otherwise, the pickled state must be a dictionary
+ and its items are assigned to the new instance's dictionary.
+
+ .. note::
+
+ If :meth:`__getstate__` returns a false value, the :meth:`__setstate__`
+ method will not be called upon unpickling.
+
Refer to the section :ref:`pickle-state` for more information about how to use
the methods :meth:`__getstate__` and :meth:`__setstate__`.
@@ -462,14 +467,12 @@
At unpickling time, some methods like :meth:`__getattr__`,
:meth:`__getattribute__`, or :meth:`__setattr__` may be called upon the
- instance. In case those methods rely on some internal invariant being
- true, the type should implement either :meth:`__getinitargs__` or
- :meth:`__getnewargs__` to establish such an invariant; otherwise, neither
- :meth:`__new__` nor :meth:`__init__` will be called.
+ instance. In case those methods rely on some internal invariant being true,
+ the type should implement :meth:`__getnewargs__` to establish such an
+ invariant; otherwise, neither :meth:`__new__` nor :meth:`__init__` will be
+ called.
-.. index::
- pair: copy; protocol
- single: __reduce__() (copy protocol)
+.. index:: pair: copy; protocol
As we shall see, pickle does not use directly the methods described above. In
fact, these methods are part of the copy protocol which implements the
@@ -480,58 +483,61 @@
Although powerful, implementing :meth:`__reduce__` directly in your classes is
error prone. For this reason, class designers should use the high-level
interface (i.e., :meth:`__getnewargs__`, :meth:`__getstate__` and
-:meth:`__setstate__`) whenever possible. We will show, however, cases where using
-:meth:`__reduce__` is the only option or leads to more efficient pickling or
-both.
+:meth:`__setstate__`) whenever possible. We will show, however, cases where
+using :meth:`__reduce__` is the only option or leads to more efficient pickling
+or both.
-The interface is currently defined as follows. The :meth:`__reduce__` method
-takes no argument and shall return either a string or preferably a tuple (the
-returned object is often referred to as the "reduce value").
+.. method:: object.__reduce__()
-If a string is returned, the string should be interpreted as the name of a
-global variable. It should be the object's local name relative to its module;
-the pickle module searches the module namespace to determine the object's
-module. This behaviour is typically useful for singletons.
+ The interface is currently defined as follows. The :meth:`__reduce__` method
+ takes no argument and shall return either a string or preferably a tuple (the
+ returned object is often referred to as the "reduce value").
-When a tuple is returned, it must be between two and five items long. Optional
-items can either be omitted, or ``None`` can be provided as their value. The
-semantics of each item are in order:
+ If a string is returned, the string should be interpreted as the name of a
+ global variable. It should be the object's local name relative to its
+ module; the pickle module searches the module namespace to determine the
+ object's module. This behaviour is typically useful for singletons.
-.. XXX Mention __newobj__ special-case?
+ When a tuple is returned, it must be between two and five items long.
+ Optional items can either be omitted, or ``None`` can be provided as their
+ value. The semantics of each item are in order:
-* A callable object that will be called to create the initial version of the
- object.
+ .. XXX Mention __newobj__ special-case?
-* A tuple of arguments for the callable object. An empty tuple must be given if
- the callable does not accept any argument.
+ * A callable object that will be called to create the initial version of the
+ object.
-* Optionally, the object's state, which will be passed to the object's
- :meth:`__setstate__` method as previously described. If the object has no
- such method then, the value must be a dictionary and it will be added to the
- object's :attr:`__dict__` attribute.
+ * A tuple of arguments for the callable object. An empty tuple must be given
+ if the callable does not accept any argument.
-* Optionally, an iterator (and not a sequence) yielding successive items. These
- items will be appended to the object either using ``obj.append(item)`` or, in
- batch, using ``obj.extend(list_of_items)``. This is primarily used for list
- subclasses, but may be used by other classes as long as they have
- :meth:`append` and :meth:`extend` methods with the appropriate signature.
- (Whether :meth:`append` or :meth:`extend` is used depends on which pickle
- protocol version is used as well as the number of items to append, so both
- must be supported.)
+ * Optionally, the object's state, which will be passed to the object's
+ :meth:`__setstate__` method as previously described. If the object has no
+ such method then, the value must be a dictionary and it will be added to
+ the object's :attr:`__dict__` attribute.
-* Optionally, an iterator (not a sequence) yielding successive key-value pairs.
- These items will be stored to the object using ``obj[key] = value``. This is
- primarily used for dictionary subclasses, but may be used by other classes as
- long as they implement :meth:`__setitem__`.
+ * Optionally, an iterator (and not a sequence) yielding successive items.
+ These items will be appended to the object either using
+ ``obj.append(item)`` or, in batch, using ``obj.extend(list_of_items)``.
+ This is primarily used for list subclasses, but may be used by other
+ classes as long as they have :meth:`append` and :meth:`extend` methods with
+ the appropriate signature. (Whether :meth:`append` or :meth:`extend` is
+ used depends on which pickle protocol version is used as well as the number
+ of items to append, so both must be supported.)
-.. index:: single: __reduce_ex__() (copy protocol)
+ * Optionally, an iterator (not a sequence) yielding successive key-value
+ pairs. These items will be stored to the object using ``obj[key] =
+ value``. This is primarily used for dictionary subclasses, but may be used
+ by other classes as long as they implement :meth:`__setitem__`.
-Alternatively, a :meth:`__reduce_ex__` method may be defined. The only
-difference is this method should take a single integer argument, the protocol
-version. When defined, pickle will prefer it over the :meth:`__reduce__`
-method. In addition, :meth:`__reduce__` automatically becomes a synonym for the
-extended version. The main use for this method is to provide
-backwards-compatible reduce values for older Python releases.
+
+.. method:: object.__reduce_ex__(protocol)
+
+ Alternatively, a :meth:`__reduce_ex__` method may be defined. The only
+ difference is this method should take a single integer argument, the protocol
+ version. When defined, pickle will prefer it over the :meth:`__reduce__`
+ method. In addition, :meth:`__reduce__` automatically becomes a synonym for
+ the extended version. The main use for this method is to provide
+ backwards-compatible reduce values for older Python releases.
.. _pickle-persistent:
diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst
index 0650b41..a2f9720 100644
--- a/Doc/library/shelve.rst
+++ b/Doc/library/shelve.rst
@@ -43,6 +43,11 @@
:meth:`close` explicitly when you don't need it any more, or use a
:keyword:`with` statement with :func:`contextlib.closing`.
+.. warning::
+
+ Because the :mod:`shelve` module is backed by :mod:`pickle`, it is insecure
+ to load a shelf from an untrusted source. Like with pickle, loading a shelf
+ can execute arbitrary code.
Shelf objects support all methods supported by dictionaries. This eases the
transition from dictionary based scripts to those requiring persistent storage.
diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst
index b76f6eb..0603738 100644
--- a/Doc/library/sqlite3.rst
+++ b/Doc/library/sqlite3.rst
@@ -255,23 +255,22 @@
.. method:: Connection.execute(sql, [parameters])
This is a nonstandard shortcut that creates an intermediate cursor object by
- calling the cursor method, then calls the cursor's
- :meth:`execute<Cursor.execute>` method with the parameters given.
+ calling the cursor method, then calls the cursor's :meth:`execute
+ <Cursor.execute>` method with the parameters given.
.. method:: Connection.executemany(sql, [parameters])
This is a nonstandard shortcut that creates an intermediate cursor object by
- calling the cursor method, then calls the cursor's
- :meth:`executemany<Cursor.executemany>` method with the parameters given.
+ calling the cursor method, then calls the cursor's :meth:`executemany
+ <Cursor.executemany>` method with the parameters given.
.. method:: Connection.executescript(sql_script)
This is a nonstandard shortcut that creates an intermediate cursor object by
- calling the cursor method, then calls the cursor's
- :meth:`executescript<Cursor.executescript>` method with the parameters
- given.
+ calling the cursor method, then calls the cursor's :meth:`executescript
+ <Cursor.executescript>` method with the parameters given.
.. method:: Connection.create_function(name, num_params, func)
@@ -435,7 +434,7 @@
.. class:: Cursor
- A SQLite database cursor has the following attributes and methods:
+ A :class:`Cursor` instance has the following attributes and methods.
.. method:: Cursor.execute(sql, [parameters])
@@ -853,4 +852,3 @@
The only exception is calling the :meth:`~Connection.interrupt` method, which
only makes sense to call from a different thread.
-
diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
index 5693ed5..4b7ae39 100644
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -1264,9 +1264,8 @@
dictionary inserted immediately after the ``'%'`` character. The mapping key
selects the value to be formatted from the mapping. For example:
-
- >>> print('%(language)s has %(#)03d quote types.' % \
- ... {'language': "Python", "#": 2})
+ >>> print('%(language)s has %(number)03d quote types.' %
+ ... {'language': "Python", "number": 2})
Python has 002 quote types.
In this case no ``*`` specifiers may occur in a format (since they require a
@@ -1877,12 +1876,12 @@
values are added as items to the dictionary. If a key is specified both in
the positional argument and as a keyword argument, the value associated with
the keyword is retained in the dictionary. For example, these all return a
- dictionary equal to ``{"one": 2, "two": 3}``:
+ dictionary equal to ``{"one": 1, "two": 2}``:
- * ``dict(one=2, two=3)``
- * ``dict({'one': 2, 'two': 3})``
- * ``dict(zip(('one', 'two'), (2, 3)))``
- * ``dict([['two', 3], ['one', 2]])``
+ * ``dict(one=1, two=2)``
+ * ``dict({'one': 1, 'two': 2})``
+ * ``dict(zip(('one', 'two'), (1, 2)))``
+ * ``dict([['two', 2], ['one', 1]])``
The first example only works for keys that are valid Python identifiers; the
others work with any valid keys.
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
index 3af5b5f..d83f137 100644
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -48,6 +48,13 @@
``modules.keys()`` only lists the imported modules.)
+.. function:: call_tracing(func, args)
+
+ Call ``func(*args)``, while tracing is enabled. The tracing state is saved,
+ and restored afterwards. This is intended to be called from a debugger from
+ a checkpoint, to recursively debug some other code.
+
+
.. data:: copyright
A string containing the copyright pertaining to the Python interpreter.
@@ -173,19 +180,25 @@
Exit from Python. This is implemented by raising the :exc:`SystemExit`
exception, so cleanup actions specified by finally clauses of :keyword:`try`
- statements are honored, and it is possible to intercept the exit attempt at an
- outer level. The optional argument *arg* can be an integer giving the exit
- status (defaulting to zero), or another type of object. If it is an integer,
- zero is considered "successful termination" and any nonzero value is considered
- "abnormal termination" by shells and the like. Most systems require it to be in
- the range 0-127, and produce undefined results otherwise. Some systems have a
- convention for assigning specific meanings to specific exit codes, but these are
- generally underdeveloped; Unix programs generally use 2 for command line syntax
- errors and 1 for all other kind of errors. If another type of object is passed,
- ``None`` is equivalent to passing zero, and any other object is printed to
- ``sys.stderr`` and results in an exit code of 1. In particular,
- ``sys.exit("some error message")`` is a quick way to exit a program when an
- error occurs.
+ statements are honored, and it is possible to intercept the exit attempt at
+ an outer level.
+
+ The optional argument *arg* can be an integer giving the exit status
+ (defaulting to zero), or another type of object. If it is an integer, zero
+ is considered "successful termination" and any nonzero value is considered
+ "abnormal termination" by shells and the like. Most systems require it to be
+ in the range 0-127, and produce undefined results otherwise. Some systems
+ have a convention for assigning specific meanings to specific exit codes, but
+ these are generally underdeveloped; Unix programs generally use 2 for command
+ line syntax errors and 1 for all other kind of errors. If another type of
+ object is passed, ``None`` is equivalent to passing zero, and any other
+ object is printed to :data:`stderr` and results in an exit code of 1. In
+ particular, ``sys.exit("some error message")`` is a quick way to exit a
+ program when an error occurs.
+
+ Since :func:`exit` ultimately "only" raises an exception, it will only exit
+ the process when called from the main thread, and the exception is not
+ intercepted.
.. data:: flags
diff --git a/Doc/library/token.rst b/Doc/library/token.rst
index bc5c19e..991762f 100644
--- a/Doc/library/token.rst
+++ b/Doc/library/token.rst
@@ -12,8 +12,8 @@
the language grammar. The specific numeric values which the names map to may
change between Python versions.
-This module also provides one data object and some functions. The functions
-mirror definitions in the Python C header files.
+The module also provides a mapping from numeric codes to names and some
+functions. The functions mirror definitions in the Python C header files.
.. data:: tok_name
@@ -38,6 +38,65 @@
Return true if *x* is the marker indicating the end of input.
+The token constants are:
+
+.. data:: ENDMARKER
+ NAME
+ NUMBER
+ STRING
+ NEWLINE
+ INDENT
+ DEDENT
+ LPAR
+ RPAR
+ LSQB
+ RSQB
+ COLON
+ COMMA
+ SEMI
+ PLUS
+ MINUS
+ STAR
+ SLASH
+ VBAR
+ AMPER
+ LESS
+ GREATER
+ EQUAL
+ DOT
+ PERCENT
+ BACKQUOTE
+ LBRACE
+ RBRACE
+ EQEQUAL
+ NOTEQUAL
+ LESSEQUAL
+ GREATEREQUAL
+ TILDE
+ CIRCUMFLEX
+ LEFTSHIFT
+ RIGHTSHIFT
+ DOUBLESTAR
+ PLUSEQUAL
+ MINEQUAL
+ STAREQUAL
+ SLASHEQUAL
+ PERCENTEQUAL
+ AMPEREQUAL
+ VBAREQUAL
+ CIRCUMFLEXEQUAL
+ LEFTSHIFTEQUAL
+ RIGHTSHIFTEQUAL
+ DOUBLESTAREQUAL
+ DOUBLESLASH
+ DOUBLESLASHEQUAL
+ AT
+ OP
+ ERRORTOKEN
+ N_TOKENS
+ NT_OFFSET
+
+
.. seealso::
Module :mod:`parser`