Merge issue 11662.
diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst
index 8bd3ca5..2164ec0 100644
--- a/Doc/library/argparse.rst
+++ b/Doc/library/argparse.rst
@@ -351,6 +351,11 @@
:class:`ArgumentParser` will see two ``-h/--help`` options (one in the parent
and one in the child) and raise an error.
+.. note::
+ You must fully initialize the parsers before passing them via ``parents=``.
+ If you change the parent parsers after the child parser, those changes will
+ not be reflected in the child.
+
formatter_class
^^^^^^^^^^^^^^^
@@ -1314,13 +1319,24 @@
Namespace(accumulate=<built-in function sum>, integers=[1, 2, 3, 4])
-Custom namespaces
-^^^^^^^^^^^^^^^^^
+The Namespace object
+^^^^^^^^^^^^^^^^^^^^
+
+By default, :meth:`parse_args` will return a new object of type :class:`Namespace`
+where the necessary attributes have been set. This class is deliberately simple,
+just an :class:`object` subclass with a readable string representation. If you
+prefer to have dict-like view of the attributes, you can use the standard Python
+idiom via :func:`vars`::
+
+ >>> parser = argparse.ArgumentParser()
+ >>> parser.add_argument('--foo')
+ >>> args = parser.parse_args(['--foo', 'BAR'])
+ >>> vars(args)
+ {'foo': 'BAR'}
It may also be useful to have an :class:`ArgumentParser` assign attributes to an
-already existing object, rather than the newly-created :class:`Namespace` object
-that is normally used. This can be achieved by specifying the ``namespace=``
-keyword argument::
+already existing object, rather than a new :class:`Namespace` object. This can
+be achieved by specifying the ``namespace=`` keyword argument::
>>> class C:
... pass
diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst
index 602f513..5f67759 100644
--- a/Doc/library/collections.rst
+++ b/Doc/library/collections.rst
@@ -964,10 +964,10 @@
ABCs - abstract base classes
----------------------------
-The collections module offers the following ABCs:
+The collections module offers the following :term:`ABCs <abstract base class>`:
========================= ===================== ====================== ====================================================
-ABC Inherits Abstract Methods Mixin Methods
+ABC Inherits from Abstract Methods Mixin Methods
========================= ===================== ====================== ====================================================
:class:`Container` ``__contains__``
:class:`Hashable` ``__hash__``
@@ -980,15 +980,15 @@
:class:`Iterable`, ``index``, and ``count``
:class:`Container`
-:class:`MutableSequence` :class:`Sequence` ``__setitem__`` Inherited Sequence methods and
+:class:`MutableSequence` :class:`Sequence` ``__setitem__``, Inherited :class:`Sequence` methods and
``__delitem__``, ``append``, ``reverse``, ``extend``, ``pop``,
- and ``insert`` ``remove``, and ``__iadd__``
+ ``insert`` ``remove``, and ``__iadd__``
:class:`Set` :class:`Sized`, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``,
:class:`Iterable`, ``__gt__``, ``__ge__``, ``__and__``, ``__or__``,
:class:`Container` ``__sub__``, ``__xor__``, and ``isdisjoint``
-:class:`MutableSet` :class:`Set` ``add`` and Inherited Set methods and
+:class:`MutableSet` :class:`Set` ``add``, Inherited :class:`Set` methods and
``discard`` ``clear``, ``pop``, ``remove``, ``__ior__``,
``__iand__``, ``__ixor__``, and ``__isub__``
@@ -996,19 +996,61 @@
:class:`Iterable`, ``get``, ``__eq__``, and ``__ne__``
:class:`Container`
-:class:`MutableMapping` :class:`Mapping` ``__setitem__`` and Inherited Mapping methods and
+:class:`MutableMapping` :class:`Mapping` ``__setitem__``, Inherited :class:`Mapping` methods and
``__delitem__`` ``pop``, ``popitem``, ``clear``, ``update``,
and ``setdefault``
:class:`MappingView` :class:`Sized` ``__len__``
-:class:`KeysView` :class:`MappingView`, ``__contains__``,
- :class:`Set` ``__iter__``
:class:`ItemsView` :class:`MappingView`, ``__contains__``,
:class:`Set` ``__iter__``
+:class:`KeysView` :class:`MappingView`, ``__contains__``,
+ :class:`Set` ``__iter__``
:class:`ValuesView` :class:`MappingView` ``__contains__``, ``__iter__``
========================= ===================== ====================== ====================================================
+
+.. class:: Container
+ Hashable
+ Sized
+ Callable
+
+ ABCs for classes that provide respectively the methods :meth:`__contains__`,
+ :meth:`__hash__`, :meth:`__len__`, and :meth:`__call__`.
+
+.. class:: Iterable
+
+ ABC for classes that provide the :meth:`__iter__` method.
+ See also the definition of :term:`iterable`.
+
+.. class:: Iterator
+
+ ABC for classes that provide the :meth:`__iter__` and :meth:`next` methods.
+ See also the definition of :term:`iterator`.
+
+.. class:: Sequence
+ MutableSequence
+
+ ABCs for read-only and mutable :term:`sequences <sequence>`.
+
+.. class:: Set
+ MutableSet
+
+ ABCs for read-only and mutable sets.
+
+.. class:: Mapping
+ MutableMapping
+
+ ABCs for read-only and mutable :term:`mappings <mapping>`.
+
+.. class:: MappingView
+ ItemsView
+ KeysView
+ ValuesView
+
+ ABCs for mapping, items, keys, and values :term:`views <view>`.
+
+
These ABCs allow us to ask classes or instances if they provide
particular functionality, for example::
diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst
index 55dd958..cb7a09c 100644
--- a/Doc/library/compileall.rst
+++ b/Doc/library/compileall.rst
@@ -68,6 +68,9 @@
.. versionchanged:: 3.2
Added the ``-i``, ``-b`` and ``-h`` options.
+There is no command-line option to control the optimization level used by the
+:func:`compile` function, because the Python interpreter itself already
+provides the option: :program:`python -O -m compileall`.
Public functions
----------------
diff --git a/Doc/library/http.cookiejar.rst b/Doc/library/http.cookiejar.rst
index e3af41f..9771496 100644
--- a/Doc/library/http.cookiejar.rst
+++ b/Doc/library/http.cookiejar.rst
@@ -722,7 +722,7 @@
import os, http.cookiejar, urllib.request
cj = http.cookiejar.MozillaCookieJar()
- cj.load(os.path.join(os.environ["HOME"], ".netscape/cookies.txt"))
+ cj.load(os.path.join(os.path.expanduser("~"), ".netscape", "cookies.txt"))
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open("http://example.com/")
diff --git a/Doc/library/readline.rst b/Doc/library/readline.rst
index c667317..ab55197 100644
--- a/Doc/library/readline.rst
+++ b/Doc/library/readline.rst
@@ -196,7 +196,7 @@
import os
import readline
- histfile = os.path.join(os.environ["HOME"], ".pyhist")
+ histfile = os.path.join(os.path.expanduser("~"), ".pyhist")
try:
readline.read_history_file(histfile)
except IOError:
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
index 88655bb..e03ac8a 100644
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -239,33 +239,22 @@
The struct sequence *flags* exposes the status of command line flags. The
attributes are read only.
- +------------------------------+------------------------------------------+
- | attribute | flag |
- +==============================+==========================================+
- | :const:`debug` | -d |
- +------------------------------+------------------------------------------+
- | :const:`division_warning` | -Q |
- +------------------------------+------------------------------------------+
- | :const:`inspect` | -i |
- +------------------------------+------------------------------------------+
- | :const:`interactive` | -i |
- +------------------------------+------------------------------------------+
- | :const:`optimize` | -O or -OO |
- +------------------------------+------------------------------------------+
- | :const:`dont_write_bytecode` | -B |
- +------------------------------+------------------------------------------+
- | :const:`no_user_site` | -s |
- +------------------------------+------------------------------------------+
- | :const:`no_site` | -S |
- +------------------------------+------------------------------------------+
- | :const:`ignore_environment` | -E |
- +------------------------------+------------------------------------------+
- | :const:`verbose` | -v |
- +------------------------------+------------------------------------------+
- | :const:`bytes_warning` | -b |
- +------------------------------+------------------------------------------+
- | :const:`quiet` | -q |
- +------------------------------+------------------------------------------+
+ ============================= =============================
+ attribute flag
+ ============================= =============================
+ :const:`debug` :option:`-d`
+ :const:`division_warning` :option:`-Q`
+ :const:`inspect` :option:`-i`
+ :const:`interactive` :option:`-i`
+ :const:`optimize` :option:`-O` or :option:`-OO`
+ :const:`dont_write_bytecode` :option:`-B`
+ :const:`no_user_site` :option:`-s`
+ :const:`no_site` :option:`-S`
+ :const:`ignore_environment` :option:`-E`
+ :const:`verbose` :option:`-v`
+ :const:`bytes_warning` :option:`-b`
+ :const:`quiet` :option:`-q`
+ ============================= =============================
.. versionchanged:: 3.2
Added ``quiet`` attribute for the new :option:`-q` flag.
diff --git a/Doc/tutorial/interactive.rst b/Doc/tutorial/interactive.rst
index ca0cfaf..5faaf96 100644
--- a/Doc/tutorial/interactive.rst
+++ b/Doc/tutorial/interactive.rst
@@ -123,10 +123,7 @@
# bound to the Esc key by default (you can change it - see readline docs).
#
# Store the file in ~/.pystartup, and set an environment variable to point
- # to it: "export PYTHONSTARTUP=/home/user/.pystartup" in bash.
- #
- # Note that PYTHONSTARTUP does *not* expand "~", so you have to put in the
- # full path to your home directory.
+ # to it: "export PYTHONSTARTUP=~/.pystartup" in bash.
import atexit
import os
diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst
index 20ce228..d411d85 100644
--- a/Doc/whatsnew/3.2.rst
+++ b/Doc/whatsnew/3.2.rst
@@ -1499,11 +1499,11 @@
>>> os.fsencode(filename)
b'Sehensw\xc3\xbcrdigkeiten'
-Some operating systems allow direct access to the unencoded bytes in the
+Some operating systems allow direct access to encoded bytes in the
environment. If so, the :attr:`os.supports_bytes_environ` constant will be
true.
-For direct access to unencoded environment variables (if available),
+For direct access to encoded environment variables (if available),
use the new :func:`os.getenvb` function or use :data:`os.environb`
which is a bytes version of :data:`os.environ`.
diff --git a/Lib/argparse.py b/Lib/argparse.py
index de3cd11..0ef9f4e 100644
--- a/Lib/argparse.py
+++ b/Lib/argparse.py
@@ -82,6 +82,7 @@
]
+import collections as _collections
import copy as _copy
import os as _os
import re as _re
@@ -1041,7 +1042,7 @@
self._prog_prefix = prog
self._parser_class = parser_class
- self._name_parser_map = {}
+ self._name_parser_map = _collections.OrderedDict()
self._choices_actions = []
super(_SubParsersAction, self).__init__(
@@ -1294,6 +1295,13 @@
if not _callable(type_func):
raise ValueError('%r is not callable' % type_func)
+ # raise an error if the metavar does not match the type
+ if hasattr(self, "_get_formatter"):
+ try:
+ self._get_formatter()._format_args(action, None)
+ except TypeError:
+ raise ValueError("length of metavar tuple does not match nargs")
+
return self._add_action(action)
def add_argument_group(self, *args, **kwargs):
diff --git a/Lib/concurrent/futures/process.py b/Lib/concurrent/futures/process.py
index 79c60c3..a899d5f 100644
--- a/Lib/concurrent/futures/process.py
+++ b/Lib/concurrent/futures/process.py
@@ -66,28 +66,17 @@
# workers to exit when their work queues are empty and then waits until the
# threads/processes finish.
-_thread_references = set()
+_threads_queues = weakref.WeakKeyDictionary()
_shutdown = False
def _python_exit():
global _shutdown
_shutdown = True
- for thread_reference in _thread_references:
- thread = thread_reference()
- if thread is not None:
- thread.join()
-
-def _remove_dead_thread_references():
- """Remove inactive threads from _thread_references.
-
- Should be called periodically to prevent memory leaks in scenarios such as:
- >>> while True:
- >>> ... t = ThreadPoolExecutor(max_workers=5)
- >>> ... t.map(int, ['1', '2', '3', '4', '5'])
- """
- for thread_reference in set(_thread_references):
- if thread_reference() is None:
- _thread_references.discard(thread_reference)
+ items = list(_threads_queues.items())
+ for t, q in items:
+ q.put(None)
+ for t, q in items:
+ t.join()
# Controls how many more calls than processes will be queued in the call queue.
# A smaller number will mean that processes spend more time idle waiting for
@@ -130,11 +119,15 @@
"""
while True:
try:
- call_item = call_queue.get(block=True, timeout=0.1)
+ call_item = call_queue.get(block=True)
except queue.Empty:
if shutdown.is_set():
return
else:
+ if call_item is None:
+ # Wake up queue management thread
+ result_queue.put(None)
+ return
try:
r = call_item.fn(*call_item.args, **call_item.kwargs)
except BaseException as e:
@@ -209,40 +202,56 @@
process workers that they should exit when their work queue is
empty.
"""
+ nb_shutdown_processes = 0
+ def shutdown_one_process():
+ """Tell a worker to terminate, which will in turn wake us again"""
+ nonlocal nb_shutdown_processes
+ call_queue.put(None)
+ nb_shutdown_processes += 1
while True:
_add_call_item_to_queue(pending_work_items,
work_ids_queue,
call_queue)
try:
- result_item = result_queue.get(block=True, timeout=0.1)
+ result_item = result_queue.get(block=True)
except queue.Empty:
- executor = executor_reference()
- # No more work items can be added if:
- # - The interpreter is shutting down OR
- # - The executor that owns this worker has been collected OR
- # - The executor that owns this worker has been shutdown.
- if _shutdown or executor is None or executor._shutdown_thread:
- # Since no new work items can be added, it is safe to shutdown
- # this thread if there are no pending work items.
- if not pending_work_items:
- shutdown_process_event.set()
-
- # If .join() is not called on the created processes then
- # some multiprocessing.Queue methods may deadlock on Mac OS
- # X.
- for p in processes:
- p.join()
- return
- del executor
+ pass
else:
- work_item = pending_work_items[result_item.work_id]
- del pending_work_items[result_item.work_id]
+ if result_item is not None:
+ work_item = pending_work_items[result_item.work_id]
+ del pending_work_items[result_item.work_id]
- if result_item.exception:
- work_item.future.set_exception(result_item.exception)
+ if result_item.exception:
+ work_item.future.set_exception(result_item.exception)
+ else:
+ work_item.future.set_result(result_item.result)
+ continue
+ # If we come here, we either got a timeout or were explicitly woken up.
+ # In either case, check whether we should start shutting down.
+ executor = executor_reference()
+ # No more work items can be added if:
+ # - The interpreter is shutting down OR
+ # - The executor that owns this worker has been collected OR
+ # - The executor that owns this worker has been shutdown.
+ if _shutdown or executor is None or executor._shutdown_thread:
+ # Since no new work items can be added, it is safe to shutdown
+ # this thread if there are no pending work items.
+ if not pending_work_items:
+ shutdown_process_event.set()
+
+ while nb_shutdown_processes < len(processes):
+ shutdown_one_process()
+ # If .join() is not called on the created processes then
+ # some multiprocessing.Queue methods may deadlock on Mac OS
+ # X.
+ for p in processes:
+ p.join()
+ return
else:
- work_item.future.set_result(result_item.result)
+ # Start shutting down by telling a process it can exit.
+ shutdown_one_process()
+ del executor
_system_limits_checked = False
_system_limited = None
@@ -279,7 +288,6 @@
worker processes will be created as the machine has processors.
"""
_check_system_limits()
- _remove_dead_thread_references()
if max_workers is None:
self._max_workers = multiprocessing.cpu_count()
@@ -304,10 +312,14 @@
self._pending_work_items = {}
def _start_queue_management_thread(self):
+ # When the executor gets lost, the weakref callback will wake up
+ # the queue management thread.
+ def weakref_cb(_, q=self._result_queue):
+ q.put(None)
if self._queue_management_thread is None:
self._queue_management_thread = threading.Thread(
target=_queue_manangement_worker,
- args=(weakref.ref(self),
+ args=(weakref.ref(self, weakref_cb),
self._processes,
self._pending_work_items,
self._work_ids,
@@ -316,7 +328,7 @@
self._shutdown_process_event))
self._queue_management_thread.daemon = True
self._queue_management_thread.start()
- _thread_references.add(weakref.ref(self._queue_management_thread))
+ _threads_queues[self._queue_management_thread] = self._result_queue
def _adjust_process_count(self):
for _ in range(len(self._processes), self._max_workers):
@@ -339,6 +351,8 @@
self._pending_work_items[self._queue_count] = w
self._work_ids.put(self._queue_count)
self._queue_count += 1
+ # Wake up queue management thread
+ self._result_queue.put(None)
self._start_queue_management_thread()
self._adjust_process_count()
@@ -348,8 +362,10 @@
def shutdown(self, wait=True):
with self._shutdown_lock:
self._shutdown_thread = True
- if wait:
- if self._queue_management_thread:
+ if self._queue_management_thread:
+ # Wake up queue management thread
+ self._result_queue.put(None)
+ if wait:
self._queue_management_thread.join()
# To reduce the risk of openning too many files, remove references to
# objects that use file descriptors.
diff --git a/Lib/concurrent/futures/thread.py b/Lib/concurrent/futures/thread.py
index 15736da..93b495f 100644
--- a/Lib/concurrent/futures/thread.py
+++ b/Lib/concurrent/futures/thread.py
@@ -25,28 +25,17 @@
# workers to exit when their work queues are empty and then waits until the
# threads finish.
-_thread_references = set()
+_threads_queues = weakref.WeakKeyDictionary()
_shutdown = False
def _python_exit():
global _shutdown
_shutdown = True
- for thread_reference in _thread_references:
- thread = thread_reference()
- if thread is not None:
- thread.join()
-
-def _remove_dead_thread_references():
- """Remove inactive threads from _thread_references.
-
- Should be called periodically to prevent memory leaks in scenarios such as:
- >>> while True:
- ... t = ThreadPoolExecutor(max_workers=5)
- ... t.map(int, ['1', '2', '3', '4', '5'])
- """
- for thread_reference in set(_thread_references):
- if thread_reference() is None:
- _thread_references.discard(thread_reference)
+ items = list(_threads_queues.items())
+ for t, q in items:
+ q.put(None)
+ for t, q in items:
+ t.join()
atexit.register(_python_exit)
@@ -72,18 +61,23 @@
try:
while True:
try:
- work_item = work_queue.get(block=True, timeout=0.1)
+ work_item = work_queue.get(block=True)
except queue.Empty:
- executor = executor_reference()
- # Exit if:
- # - The interpreter is shutting down OR
- # - The executor that owns the worker has been collected OR
- # - The executor that owns the worker has been shutdown.
- if _shutdown or executor is None or executor._shutdown:
- return
- del executor
+ pass
else:
- work_item.run()
+ if work_item is not None:
+ work_item.run()
+ continue
+ executor = executor_reference()
+ # Exit if:
+ # - The interpreter is shutting down OR
+ # - The executor that owns the worker has been collected OR
+ # - The executor that owns the worker has been shutdown.
+ if _shutdown or executor is None or executor._shutdown:
+ # Notice other workers
+ work_queue.put(None)
+ return
+ del executor
except BaseException as e:
_base.LOGGER.critical('Exception in worker', exc_info=True)
@@ -95,8 +89,6 @@
max_workers: The maximum number of threads that can be used to
execute the given calls.
"""
- _remove_dead_thread_references()
-
self._max_workers = max_workers
self._work_queue = queue.Queue()
self._threads = set()
@@ -117,19 +109,25 @@
submit.__doc__ = _base.Executor.submit.__doc__
def _adjust_thread_count(self):
+ # When the executor gets lost, the weakref callback will wake up
+ # the worker threads.
+ def weakref_cb(_, q=self._work_queue):
+ q.put(None)
# TODO(bquinlan): Should avoid creating new threads if there are more
# idle threads than items in the work queue.
if len(self._threads) < self._max_workers:
t = threading.Thread(target=_worker,
- args=(weakref.ref(self), self._work_queue))
+ args=(weakref.ref(self, weakref_cb),
+ self._work_queue))
t.daemon = True
t.start()
self._threads.add(t)
- _thread_references.add(weakref.ref(t))
+ _threads_queues[t] = self._work_queue
def shutdown(self, wait=True):
with self._shutdown_lock:
self._shutdown = True
+ self._work_queue.put(None)
if wait:
for t in self._threads:
t.join()
diff --git a/Lib/ctypes/test/test_as_parameter.py b/Lib/ctypes/test/test_as_parameter.py
index 835398f..475d595 100644
--- a/Lib/ctypes/test/test_as_parameter.py
+++ b/Lib/ctypes/test/test_as_parameter.py
@@ -187,6 +187,18 @@
self.assertEqual((s8i.a, s8i.b, s8i.c, s8i.d, s8i.e, s8i.f, s8i.g, s8i.h),
(9*2, 8*3, 7*4, 6*5, 5*6, 4*7, 3*8, 2*9))
+ def test_recursive_as_param(self):
+ from ctypes import c_int
+
+ class A(object):
+ pass
+
+ a = A()
+ a._as_parameter_ = a
+ with self.assertRaises(RuntimeError):
+ c_int.from_param(a)
+
+
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
class AsParamWrapper(object):
diff --git a/Lib/email/header.py b/Lib/email/header.py
index 8c32514..2562b30 100644
--- a/Lib/email/header.py
+++ b/Lib/email/header.py
@@ -66,9 +66,15 @@
otherwise a lower-case string containing the name of the character set
specified in the encoded string.
+ header may be a string that may or may not contain RFC2047 encoded words,
+ or it may be a Header object.
+
An email.errors.HeaderParseError may be raised when certain decoding error
occurs (e.g. a base64 decoding exception).
"""
+ # If it is a Header object, we can just return the chunks.
+ if hasattr(header, '_chunks'):
+ return list(header._chunks)
# If no encoding, just return the header with no charset.
if not ecre.search(header):
return [(header, None)]
diff --git a/Lib/email/test/test_email.py b/Lib/email/test/test_email.py
index 250a644..245f659 100644
--- a/Lib/email/test/test_email.py
+++ b/Lib/email/test/test_email.py
@@ -3918,6 +3918,20 @@
h.append(x, errors='replace')
eq(str(h), e)
+ def test_escaped_8bit_header(self):
+ x = b'Ynwp4dUEbay Auction Semiar- No Charge \x96 Earn Big'
+ x = x.decode('ascii', 'surrogateescape')
+ h = Header(x, charset=email.charset.UNKNOWN8BIT)
+ self.assertEqual(str(h),
+ 'Ynwp4dUEbay Auction Semiar- No Charge \uFFFD Earn Big')
+ self.assertEqual(email.header.decode_header(h), [(x, 'unknown-8bit')])
+
+ def test_modify_returned_list_does_not_change_header(self):
+ h = Header('test')
+ chunks = email.header.decode_header(h)
+ chunks.append(('ascii', 'test2'))
+ self.assertEqual(str(h), 'test')
+
def test_encoded_adjacent_nonencoded(self):
eq = self.assertEqual
h = Header()
diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py
index a37cb1d..ffc4e88 100644
--- a/Lib/idlelib/EditorWindow.py
+++ b/Lib/idlelib/EditorWindow.py
@@ -303,13 +303,13 @@
return "break"
def home_callback(self, event):
- if (event.state & 12) != 0 and event.keysym == "Home":
- # state&1==shift, state&4==control, state&8==alt
- return # <Modifier-Home>; fall back to class binding
-
+ if (event.state & 4) != 0 and event.keysym == "Home":
+ # state&4==Control. If <Control-Home>, use the Tk binding.
+ return
if self.text.index("iomark") and \
self.text.compare("iomark", "<=", "insert lineend") and \
self.text.compare("insert linestart", "<=", "iomark"):
+ # In Shell on input line, go to just after prompt
insertpt = int(self.text.index("iomark").split(".")[1])
else:
line = self.text.get("insert linestart", "insert lineend")
@@ -318,30 +318,27 @@
break
else:
insertpt=len(line)
-
lineat = int(self.text.index("insert").split('.')[1])
-
if insertpt == lineat:
insertpt = 0
-
dest = "insert linestart+"+str(insertpt)+"c"
-
if (event.state&1) == 0:
- # shift not pressed
+ # shift was not pressed
self.text.tag_remove("sel", "1.0", "end")
else:
if not self.text.index("sel.first"):
- self.text.mark_set("anchor","insert")
-
+ self.text.mark_set("my_anchor", "insert") # there was no previous selection
+ else:
+ if self.text.compare(self.text.index("sel.first"), "<", self.text.index("insert")):
+ self.text.mark_set("my_anchor", "sel.first") # extend back
+ else:
+ self.text.mark_set("my_anchor", "sel.last") # extend forward
first = self.text.index(dest)
- last = self.text.index("anchor")
-
+ last = self.text.index("my_anchor")
if self.text.compare(first,">",last):
first,last = last,first
-
self.text.tag_remove("sel", "1.0", "end")
self.text.tag_add("sel", first, last)
-
self.text.mark_set("insert", dest)
self.text.see("insert")
return "break"
diff --git a/Lib/idlelib/NEWS.txt b/Lib/idlelib/NEWS.txt
index 037b02a..deeb4a6 100644
--- a/Lib/idlelib/NEWS.txt
+++ b/Lib/idlelib/NEWS.txt
@@ -1,4 +1,15 @@
-What's New in IDLE 3.1?
+What's New in IDLE 3.1.4?
+=========================
+
+*Release date: XX-XXX-XX*
+
+- <Home> toggle failing on Tk 8.5, causing IDLE exits and strange selection
+ behavior. Issue 4676. Improve selection extension behaviour.
+- <Home> toggle non-functional when NumLock set on Windows. Issue 3851.
+
+
+
+What's New in IDLE 3.1b1?
=========================
*Release date: 27-Jun-09*
diff --git a/Lib/inspect.py b/Lib/inspect.py
index aa951d8..4899cbf 100644
--- a/Lib/inspect.py
+++ b/Lib/inspect.py
@@ -944,8 +944,14 @@
f_name, 'at most' if defaults else 'exactly', num_args,
'arguments' if num_args > 1 else 'argument', num_total))
elif num_args == 0 and num_total:
- raise TypeError('%s() takes no arguments (%d given)' %
- (f_name, num_total))
+ if varkw or kwonlyargs:
+ if num_pos:
+ # XXX: We should use num_pos, but Python also uses num_total:
+ raise TypeError('%s() takes exactly 0 positional arguments '
+ '(%d given)' % (f_name, num_total))
+ else:
+ raise TypeError('%s() takes no arguments (%d given)' %
+ (f_name, num_total))
for arg in itertools.chain(args, kwonlyargs):
if arg in named:
diff --git a/Lib/msilib/__init__.py b/Lib/msilib/__init__.py
index bd3e1f2..6eff89c 100644
--- a/Lib/msilib/__init__.py
+++ b/Lib/msilib/__init__.py
@@ -172,11 +172,10 @@
add_data(db, table, getattr(module, table))
def make_id(str):
- #str = str.replace(".", "_") # colons are allowed
- str = str.replace(" ", "_")
- str = str.replace("-", "_")
- if str[0] in string.digits:
- str = "_"+str
+ identifier_chars = string.ascii_letters + string.digits + "._"
+ str = "".join([c if c in identifier_chars else "_" for c in str])
+ if str[0] in (string.digits + "."):
+ str = "_" + str
assert re.match("^[A-Za-z_][A-Za-z0-9_.]*$", str), "FILE"+str
return str
@@ -284,19 +283,28 @@
[(feature.id, component)])
def make_short(self, file):
+ oldfile = file
+ file = file.replace('+', '_')
+ file = ''.join(c for c in file if not c in ' "/\[]:;=,')
parts = file.split(".")
- if len(parts)>1:
+ if len(parts) > 1:
+ prefix = "".join(parts[:-1]).upper()
suffix = parts[-1].upper()
+ if not prefix:
+ prefix = suffix
+ suffix = None
else:
+ prefix = file.upper()
suffix = None
- prefix = parts[0].upper()
- if len(prefix) <= 8 and (not suffix or len(suffix)<=3):
+ if len(parts) < 3 and len(prefix) <= 8 and file == oldfile and (
+ not suffix or len(suffix) <= 3):
if suffix:
file = prefix+"."+suffix
else:
file = prefix
- assert file not in self.short_names
else:
+ file = None
+ if file is None or file in self.short_names:
prefix = prefix[:6]
if suffix:
suffix = suffix[:3]
diff --git a/Lib/multiprocessing/sharedctypes.py b/Lib/multiprocessing/sharedctypes.py
index e83ce58..1e694da 100644
--- a/Lib/multiprocessing/sharedctypes.py
+++ b/Lib/multiprocessing/sharedctypes.py
@@ -80,7 +80,9 @@
type_ = typecode_to_type.get(typecode_or_type, typecode_or_type)
if isinstance(size_or_initializer, int):
type_ = type_ * size_or_initializer
- return _new_value(type_)
+ obj = _new_value(type_)
+ ctypes.memset(ctypes.addressof(obj), 0, ctypes.sizeof(obj))
+ return obj
else:
type_ = type_ * len(size_or_initializer)
result = _new_value(type_)
diff --git a/Lib/pydoc.py b/Lib/pydoc.py
index 9d3cdd5..dc398e3 100755
--- a/Lib/pydoc.py
+++ b/Lib/pydoc.py
@@ -165,7 +165,7 @@
no.append(x)
return yes, no
-def visiblename(name, all=None):
+def visiblename(name, all=None, obj=None):
"""Decide whether to show documentation on a variable."""
# Certain special names are redundant.
_hidden_names = ('__builtins__', '__doc__', '__file__', '__path__',
@@ -175,6 +175,9 @@
if name in _hidden_names: return 0
# Private names are hidden, but special names are displayed.
if name.startswith('__') and name.endswith('__'): return 1
+ # Namedtuples have public fields and methods with a single leading underscore
+ if name.startswith('_') and hasattr(obj, '_fields'):
+ return True
if all is not None:
# only document that which the programmer exported in __all__
return name in all
@@ -642,7 +645,7 @@
# if __all__ exists, believe it. Otherwise use old heuristic.
if (all is not None or
(inspect.getmodule(value) or object) is object):
- if visiblename(key, all):
+ if visiblename(key, all, object):
classes.append((key, value))
cdict[key] = cdict[value] = '#' + key
for key, value in classes:
@@ -658,13 +661,13 @@
# if __all__ exists, believe it. Otherwise use old heuristic.
if (all is not None or
inspect.isbuiltin(value) or inspect.getmodule(value) is object):
- if visiblename(key, all):
+ if visiblename(key, all, object):
funcs.append((key, value))
fdict[key] = '#-' + key
if inspect.isfunction(value): fdict[value] = fdict[key]
data = []
for key, value in inspect.getmembers(object, isdata):
- if visiblename(key, all):
+ if visiblename(key, all, object):
data.append((key, value))
doc = self.markup(getdoc(object), self.preformat, fdict, cdict)
@@ -789,7 +792,7 @@
attrs = [(name, kind, cls, value)
for name, kind, cls, value in classify_class_attrs(object)
- if visiblename(name)]
+ if visiblename(name, obj=object)]
mdict = {}
for key, kind, homecls, value in attrs:
@@ -1056,18 +1059,18 @@
# if __all__ exists, believe it. Otherwise use old heuristic.
if (all is not None
or (inspect.getmodule(value) or object) is object):
- if visiblename(key, all):
+ if visiblename(key, all, object):
classes.append((key, value))
funcs = []
for key, value in inspect.getmembers(object, inspect.isroutine):
# if __all__ exists, believe it. Otherwise use old heuristic.
if (all is not None or
inspect.isbuiltin(value) or inspect.getmodule(value) is object):
- if visiblename(key, all):
+ if visiblename(key, all, object):
funcs.append((key, value))
data = []
for key, value in inspect.getmembers(object, isdata):
- if visiblename(key, all):
+ if visiblename(key, all, object):
data.append((key, value))
modpkgs = []
@@ -1206,7 +1209,7 @@
attrs = [(name, kind, cls, value)
for name, kind, cls, value in classify_class_attrs(object)
- if visiblename(name)]
+ if visiblename(name, obj=object)]
while attrs:
if mro:
diff --git a/Lib/re.py b/Lib/re.py
index 92e4e4c..abd7ea2 100644
--- a/Lib/re.py
+++ b/Lib/re.py
@@ -224,8 +224,7 @@
if isinstance(pattern, str):
alphanum = _alphanum_str
s = list(pattern)
- for i in range(len(pattern)):
- c = pattern[i]
+ for i, c in enumerate(pattern):
if c not in alphanum:
if c == "\000":
s[i] = "\\000"
diff --git a/Lib/test/test_argparse.py b/Lib/test/test_argparse.py
index 03c95fa..e187653 100644
--- a/Lib/test/test_argparse.py
+++ b/Lib/test/test_argparse.py
@@ -2837,16 +2837,22 @@
parser = argparse.ArgumentParser(
*tester.parser_signature.args,
**tester.parser_signature.kwargs)
- for argument_sig in tester.argument_signatures:
+ for argument_sig in getattr(tester, 'argument_signatures', []):
parser.add_argument(*argument_sig.args,
**argument_sig.kwargs)
- group_signatures = tester.argument_group_signatures
- for group_sig, argument_sigs in group_signatures:
+ group_sigs = getattr(tester, 'argument_group_signatures', [])
+ for group_sig, argument_sigs in group_sigs:
group = parser.add_argument_group(*group_sig.args,
**group_sig.kwargs)
for argument_sig in argument_sigs:
group.add_argument(*argument_sig.args,
**argument_sig.kwargs)
+ subparsers_sigs = getattr(tester, 'subparsers_signatures', [])
+ if subparsers_sigs:
+ subparsers = parser.add_subparsers()
+ for subparser_sig in subparsers_sigs:
+ subparsers.add_parser(*subparser_sig.args,
+ **subparser_sig.kwargs)
return parser
def _test(self, tester, parser_text):
@@ -3940,6 +3946,77 @@
'''
version = ''
+class TestHelpSubparsersOrdering(HelpTestCase):
+ """Test ordering of subcommands in help matches the code"""
+ parser_signature = Sig(prog='PROG',
+ description='display some subcommands',
+ version='0.1')
+
+ subparsers_signatures = [Sig(name=name)
+ for name in ('a', 'b', 'c', 'd', 'e')]
+
+ usage = '''\
+ usage: PROG [-h] [-v] {a,b,c,d,e} ...
+ '''
+
+ help = usage + '''\
+
+ display some subcommands
+
+ positional arguments:
+ {a,b,c,d,e}
+
+ optional arguments:
+ -h, --help show this help message and exit
+ -v, --version show program's version number and exit
+ '''
+
+ version = '''\
+ 0.1
+ '''
+
+class TestHelpSubparsersWithHelpOrdering(HelpTestCase):
+ """Test ordering of subcommands in help matches the code"""
+ parser_signature = Sig(prog='PROG',
+ description='display some subcommands',
+ version='0.1')
+
+ subcommand_data = (('a', 'a subcommand help'),
+ ('b', 'b subcommand help'),
+ ('c', 'c subcommand help'),
+ ('d', 'd subcommand help'),
+ ('e', 'e subcommand help'),
+ )
+
+ subparsers_signatures = [Sig(name=name, help=help)
+ for name, help in subcommand_data]
+
+ usage = '''\
+ usage: PROG [-h] [-v] {a,b,c,d,e} ...
+ '''
+
+ help = usage + '''\
+
+ display some subcommands
+
+ positional arguments:
+ {a,b,c,d,e}
+ a a subcommand help
+ b b subcommand help
+ c c subcommand help
+ d d subcommand help
+ e e subcommand help
+
+ optional arguments:
+ -h, --help show this help message and exit
+ -v, --version show program's version number and exit
+ '''
+
+ version = '''\
+ 0.1
+ '''
+
+
# =====================================
# Optional/Positional constructor tests
# =====================================
@@ -4394,6 +4471,177 @@
self.assertEqual(NS(v=3, spam=True, badger="B"), args)
self.assertEqual(["C", "--foo", "4"], extras)
+# ==========================
+# add_argument metavar tests
+# ==========================
+
+class TestAddArgumentMetavar(TestCase):
+
+ EXPECTED_MESSAGE = "length of metavar tuple does not match nargs"
+
+ def do_test_no_exception(self, nargs, metavar):
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--foo", nargs=nargs, metavar=metavar)
+
+ def do_test_exception(self, nargs, metavar):
+ parser = argparse.ArgumentParser()
+ with self.assertRaises(ValueError) as cm:
+ parser.add_argument("--foo", nargs=nargs, metavar=metavar)
+ self.assertEqual(cm.exception.args[0], self.EXPECTED_MESSAGE)
+
+ # Unit tests for different values of metavar when nargs=None
+
+ def test_nargs_None_metavar_string(self):
+ self.do_test_no_exception(nargs=None, metavar="1")
+
+ def test_nargs_None_metavar_length0(self):
+ self.do_test_exception(nargs=None, metavar=tuple())
+
+ def test_nargs_None_metavar_length1(self):
+ self.do_test_no_exception(nargs=None, metavar=("1"))
+
+ def test_nargs_None_metavar_length2(self):
+ self.do_test_exception(nargs=None, metavar=("1", "2"))
+
+ def test_nargs_None_metavar_length3(self):
+ self.do_test_exception(nargs=None, metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=?
+
+ def test_nargs_optional_metavar_string(self):
+ self.do_test_no_exception(nargs="?", metavar="1")
+
+ def test_nargs_optional_metavar_length0(self):
+ self.do_test_exception(nargs="?", metavar=tuple())
+
+ def test_nargs_optional_metavar_length1(self):
+ self.do_test_no_exception(nargs="?", metavar=("1"))
+
+ def test_nargs_optional_metavar_length2(self):
+ self.do_test_exception(nargs="?", metavar=("1", "2"))
+
+ def test_nargs_optional_metavar_length3(self):
+ self.do_test_exception(nargs="?", metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=*
+
+ def test_nargs_zeroormore_metavar_string(self):
+ self.do_test_no_exception(nargs="*", metavar="1")
+
+ def test_nargs_zeroormore_metavar_length0(self):
+ self.do_test_exception(nargs="*", metavar=tuple())
+
+ def test_nargs_zeroormore_metavar_length1(self):
+ self.do_test_no_exception(nargs="*", metavar=("1"))
+
+ def test_nargs_zeroormore_metavar_length2(self):
+ self.do_test_no_exception(nargs="*", metavar=("1", "2"))
+
+ def test_nargs_zeroormore_metavar_length3(self):
+ self.do_test_exception(nargs="*", metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=+
+
+ def test_nargs_oneormore_metavar_string(self):
+ self.do_test_no_exception(nargs="+", metavar="1")
+
+ def test_nargs_oneormore_metavar_length0(self):
+ self.do_test_exception(nargs="+", metavar=tuple())
+
+ def test_nargs_oneormore_metavar_length1(self):
+ self.do_test_no_exception(nargs="+", metavar=("1"))
+
+ def test_nargs_oneormore_metavar_length2(self):
+ self.do_test_no_exception(nargs="+", metavar=("1", "2"))
+
+ def test_nargs_oneormore_metavar_length3(self):
+ self.do_test_exception(nargs="+", metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=...
+
+ def test_nargs_remainder_metavar_string(self):
+ self.do_test_no_exception(nargs="...", metavar="1")
+
+ def test_nargs_remainder_metavar_length0(self):
+ self.do_test_no_exception(nargs="...", metavar=tuple())
+
+ def test_nargs_remainder_metavar_length1(self):
+ self.do_test_no_exception(nargs="...", metavar=("1"))
+
+ def test_nargs_remainder_metavar_length2(self):
+ self.do_test_no_exception(nargs="...", metavar=("1", "2"))
+
+ def test_nargs_remainder_metavar_length3(self):
+ self.do_test_no_exception(nargs="...", metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=A...
+
+ def test_nargs_parser_metavar_string(self):
+ self.do_test_no_exception(nargs="A...", metavar="1")
+
+ def test_nargs_parser_metavar_length0(self):
+ self.do_test_exception(nargs="A...", metavar=tuple())
+
+ def test_nargs_parser_metavar_length1(self):
+ self.do_test_no_exception(nargs="A...", metavar=("1"))
+
+ def test_nargs_parser_metavar_length2(self):
+ self.do_test_exception(nargs="A...", metavar=("1", "2"))
+
+ def test_nargs_parser_metavar_length3(self):
+ self.do_test_exception(nargs="A...", metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=1
+
+ def test_nargs_1_metavar_string(self):
+ self.do_test_no_exception(nargs=1, metavar="1")
+
+ def test_nargs_1_metavar_length0(self):
+ self.do_test_exception(nargs=1, metavar=tuple())
+
+ def test_nargs_1_metavar_length1(self):
+ self.do_test_no_exception(nargs=1, metavar=("1"))
+
+ def test_nargs_1_metavar_length2(self):
+ self.do_test_exception(nargs=1, metavar=("1", "2"))
+
+ def test_nargs_1_metavar_length3(self):
+ self.do_test_exception(nargs=1, metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=2
+
+ def test_nargs_2_metavar_string(self):
+ self.do_test_no_exception(nargs=2, metavar="1")
+
+ def test_nargs_2_metavar_length0(self):
+ self.do_test_exception(nargs=2, metavar=tuple())
+
+ def test_nargs_2_metavar_length1(self):
+ self.do_test_no_exception(nargs=2, metavar=("1"))
+
+ def test_nargs_2_metavar_length2(self):
+ self.do_test_no_exception(nargs=2, metavar=("1", "2"))
+
+ def test_nargs_2_metavar_length3(self):
+ self.do_test_exception(nargs=2, metavar=("1", "2", "3"))
+
+ # Unit tests for different values of metavar when nargs=3
+
+ def test_nargs_3_metavar_string(self):
+ self.do_test_no_exception(nargs=3, metavar="1")
+
+ def test_nargs_3_metavar_length0(self):
+ self.do_test_exception(nargs=3, metavar=tuple())
+
+ def test_nargs_3_metavar_length1(self):
+ self.do_test_no_exception(nargs=3, metavar=("1"))
+
+ def test_nargs_3_metavar_length2(self):
+ self.do_test_exception(nargs=3, metavar=("1", "2"))
+
+ def test_nargs_3_metavar_length3(self):
+ self.do_test_no_exception(nargs=3, metavar=("1", "2", "3"))
+
# ============================
# from argparse import * tests
# ============================
diff --git a/Lib/test/test_inspect.py b/Lib/test/test_inspect.py
index 331d247..7d7e16e 100644
--- a/Lib/test/test_inspect.py
+++ b/Lib/test/test_inspect.py
@@ -632,6 +632,16 @@
self.assertEqualCallArgs(f, '2, c=4, **collections.UserDict(b=3)')
self.assertEqualCallArgs(f, 'b=2, **collections.UserDict(a=3, c=4)')
+ def test_varkw_only(self):
+ # issue11256:
+ f = self.makeCallable('**c')
+ self.assertEqualCallArgs(f, '')
+ self.assertEqualCallArgs(f, 'a=1')
+ self.assertEqualCallArgs(f, 'a=1, b=2')
+ self.assertEqualCallArgs(f, 'c=3, **{"a": 1, "b": 2}')
+ self.assertEqualCallArgs(f, '**collections.UserDict(a=1, b=2)')
+ self.assertEqualCallArgs(f, 'c=3, **collections.UserDict(a=1, b=2)')
+
def test_keyword_only(self):
f = self.makeCallable('a=3, *, c, d=2')
self.assertEqualCallArgs(f, 'c=3')
@@ -643,6 +653,11 @@
self.assertEqualException(f, 'a=3')
self.assertEqualException(f, 'd=4')
+ f = self.makeCallable('*, c, d=2')
+ self.assertEqualCallArgs(f, 'c=3')
+ self.assertEqualCallArgs(f, 'c=3, d=4')
+ self.assertEqualCallArgs(f, 'd=4, c=3')
+
def test_multiple_features(self):
f = self.makeCallable('a, b=2, *f, **g')
self.assertEqualCallArgs(f, '2, 3, 7')
@@ -656,6 +671,17 @@
'(4,[5,6])]), **collections.UserDict('
'y=9, z=10)')
+ f = self.makeCallable('a, b=2, *f, x, y=99, **g')
+ self.assertEqualCallArgs(f, '2, 3, x=8')
+ self.assertEqualCallArgs(f, '2, 3, x=8, *[(4,[5,6]), 7]')
+ self.assertEqualCallArgs(f, '2, x=8, *[3, (4,[5,6]), 7], y=9, z=10')
+ self.assertEqualCallArgs(f, 'x=8, *[2, 3, (4,[5,6])], y=9, z=10')
+ self.assertEqualCallArgs(f, 'x=8, *collections.UserList('
+ '[2, 3, (4,[5,6])]), q=0, **{"y":9, "z":10}')
+ self.assertEqualCallArgs(f, '2, x=8, *collections.UserList([3, '
+ '(4,[5,6])]), q=0, **collections.UserDict('
+ 'y=9, z=10)')
+
def test_errors(self):
f0 = self.makeCallable('')
f1 = self.makeCallable('a, b')
@@ -692,6 +718,13 @@
# - for functions and bound methods: unexpected keyword 'c'
# - for unbound methods: multiple values for keyword 'a'
#self.assertEqualException(f, '1, c=3, a=2')
+ # issue11256:
+ f3 = self.makeCallable('**c')
+ self.assertEqualException(f3, '1, 2')
+ self.assertEqualException(f3, '1, 2, a=1, b=2')
+ f4 = self.makeCallable('*, a, b=0')
+ self.assertEqualException(f3, '1, 2')
+ self.assertEqualException(f3, '1, 2, a=1, b=2')
class TestGetcallargsMethods(TestGetcallargsFunctions):
diff --git a/Lib/test/test_mailbox.py b/Lib/test/test_mailbox.py
index 1e4f887..45abcc8 100644
--- a/Lib/test/test_mailbox.py
+++ b/Lib/test/test_mailbox.py
@@ -869,8 +869,6 @@
self.assertFalse((perms & 0o111)) # Execute bits should all be off.
def test_reread(self):
- # Wait for 2 seconds
- time.sleep(2)
# Initially, the mailbox has not been read and the time is null.
assert getattr(self._box, '_last_read', None) is None
@@ -879,15 +877,21 @@
self._box._refresh()
assert getattr(self._box, '_last_read', None) is not None
- # Try calling _refresh() again; the modification times shouldn't have
- # changed, so the mailbox should not be re-reading. Re-reading causes
- # the ._toc attribute to be assigned a new dictionary object, so
- # we'll check that the ._toc attribute isn't a different object.
+ # Put the last modified times more than one second into the past
+ # (because mtime has a one second granularity, a refresh is done
+ # unconditionally if called for within the same second, just in case
+ # the mbox has changed).
+ for subdir in ('cur', 'new'):
+ os.utime(os.path.join(self._box._path, subdir),
+ (time.time()-5,)*2)
+
+ # Re-reading causes the ._toc attribute to be assigned a new dictionary
+ # object, so we'll check that the ._toc attribute isn't a different
+ # object.
orig_toc = self._box._toc
def refreshed():
return self._box._toc is not orig_toc
- time.sleep(1) # Wait 1sec to ensure time.time()'s value changes
self._box._refresh()
assert not refreshed()
diff --git a/Lib/test/test_msilib.py b/Lib/test/test_msilib.py
new file mode 100644
index 0000000..ccdaec7
--- /dev/null
+++ b/Lib/test/test_msilib.py
@@ -0,0 +1,46 @@
+""" Test suite for the code in msilib """
+import unittest
+import os
+from test.support import run_unittest, import_module
+msilib = import_module('msilib')
+
+class Test_make_id(unittest.TestCase):
+ #http://msdn.microsoft.com/en-us/library/aa369212(v=vs.85).aspx
+ """The Identifier data type is a text string. Identifiers may contain the
+ ASCII characters A-Z (a-z), digits, underscores (_), or periods (.).
+ However, every identifier must begin with either a letter or an
+ underscore.
+ """
+
+ def test_is_no_change_required(self):
+ self.assertEqual(
+ msilib.make_id("short"), "short")
+ self.assertEqual(
+ msilib.make_id("nochangerequired"), "nochangerequired")
+ self.assertEqual(
+ msilib.make_id("one.dot"), "one.dot")
+ self.assertEqual(
+ msilib.make_id("_"), "_")
+ self.assertEqual(
+ msilib.make_id("a"), "a")
+ #self.assertEqual(
+ # msilib.make_id(""), "")
+
+ def test_invalid_first_char(self):
+ self.assertEqual(
+ msilib.make_id("9.short"), "_9.short")
+ self.assertEqual(
+ msilib.make_id(".short"), "_.short")
+
+ def test_invalid_any_char(self):
+ self.assertEqual(
+ msilib.make_id(".s\x82ort"), "_.s_ort")
+ self.assertEqual (
+ msilib.make_id(".s\x82o?*+rt"), "_.s_o___rt")
+
+
+def test_main():
+ run_unittest(__name__)
+
+if __name__ == '__main__':
+ test_main()
diff --git a/Lib/test/test_multiprocessing.py b/Lib/test/test_multiprocessing.py
index 55b8cdd..4e48944 100644
--- a/Lib/test/test_multiprocessing.py
+++ b/Lib/test/test_multiprocessing.py
@@ -917,6 +917,21 @@
self.assertEqual(list(arr[:]), seq)
@unittest.skipIf(c_int is None, "requires _ctypes")
+ def test_array_from_size(self):
+ size = 10
+ # Test for zeroing (see issue #11675).
+ # The repetition below strengthens the test by increasing the chances
+ # of previously allocated non-zero memory being used for the new array
+ # on the 2nd and 3rd loops.
+ for _ in range(3):
+ arr = self.Array('i', size)
+ self.assertEqual(len(arr), size)
+ self.assertEqual(list(arr), [0] * size)
+ arr[:] = range(10)
+ self.assertEqual(list(arr), list(range(10)))
+ del arr
+
+ @unittest.skipIf(c_int is None, "requires _ctypes")
def test_rawarray(self):
self.test_array(raw=True)
diff --git a/Lib/test/test_pydoc.py b/Lib/test/test_pydoc.py
index 1d575cd..92f7963 100644
--- a/Lib/test/test_pydoc.py
+++ b/Lib/test/test_pydoc.py
@@ -12,9 +12,10 @@
import xml.etree
import textwrap
from io import StringIO
+from collections import namedtuple
from contextlib import contextmanager
from test.support import TESTFN, forget, rmtree, EnvironmentVarGuard, \
- reap_children, captured_output
+ reap_children, captured_output, captured_stdout
from test import pydoc_mod
@@ -373,6 +374,15 @@
finally:
pydoc.getpager = getpager_old
+ def test_namedtuple_public_underscore(self):
+ NT = namedtuple('NT', ['abc', 'def'], rename=True)
+ with captured_stdout() as help_io:
+ help(NT)
+ helptext = help_io.getvalue()
+ self.assertIn('_1', helptext)
+ self.assertIn('_replace', helptext)
+ self.assertIn('_asdict', helptext)
+
class TestDescriptions(unittest.TestCase):
diff --git a/Lib/test/test_re.py b/Lib/test/test_re.py
index e4b33c9..fe8bc34 100644
--- a/Lib/test/test_re.py
+++ b/Lib/test/test_re.py
@@ -1,7 +1,9 @@
from test.support import verbose, run_unittest
import re
from re import Scanner
-import sys, traceback
+import sys
+import string
+import traceback
from weakref import proxy
# Misc tests from Tim Peters' re.doc
@@ -411,31 +413,62 @@
self.assertEqual(re.search("\s(b)", " b").group(1), "b")
self.assertEqual(re.search("a\s", "a ").group(0), "a ")
- def test_re_escape(self):
- p=""
- self.assertEqual(re.escape(p), p)
- for i in range(0, 256):
- p = p + chr(i)
- self.assertEqual(re.match(re.escape(chr(i)), chr(i)) is not None,
- True)
- self.assertEqual(re.match(re.escape(chr(i)), chr(i)).span(), (0,1))
+ def assertMatch(self, pattern, text, match=None, span=None,
+ matcher=re.match):
+ if match is None and span is None:
+ # the pattern matches the whole text
+ match = text
+ span = (0, len(text))
+ elif match is None or span is None:
+ raise ValueError('If match is not None, span should be specified '
+ '(and vice versa).')
+ m = matcher(pattern, text)
+ self.assertTrue(m)
+ self.assertEqual(m.group(), match)
+ self.assertEqual(m.span(), span)
- pat=re.compile(re.escape(p))
- self.assertEqual(pat.match(p) is not None, True)
- self.assertEqual(pat.match(p).span(), (0,256))
+ def test_re_escape(self):
+ alnum_chars = string.ascii_letters + string.digits
+ p = ''.join(chr(i) for i in range(256))
+ for c in p:
+ if c in alnum_chars:
+ self.assertEqual(re.escape(c), c)
+ elif c == '\x00':
+ self.assertEqual(re.escape(c), '\\000')
+ else:
+ self.assertEqual(re.escape(c), '\\' + c)
+ self.assertMatch(re.escape(c), c)
+ self.assertMatch(re.escape(p), p)
def test_re_escape_byte(self):
- p=b""
- self.assertEqual(re.escape(p), p)
- for i in range(0, 256):
+ alnum_chars = (string.ascii_letters + string.digits).encode('ascii')
+ p = bytes(range(256))
+ for i in p:
b = bytes([i])
- p += b
- self.assertEqual(re.match(re.escape(b), b) is not None, True)
- self.assertEqual(re.match(re.escape(b), b).span(), (0,1))
+ if b in alnum_chars:
+ self.assertEqual(re.escape(b), b)
+ elif i == 0:
+ self.assertEqual(re.escape(b), b'\\000')
+ else:
+ self.assertEqual(re.escape(b), b'\\' + b)
+ self.assertMatch(re.escape(b), b)
+ self.assertMatch(re.escape(p), p)
- pat=re.compile(re.escape(p))
- self.assertEqual(pat.match(p) is not None, True)
- self.assertEqual(pat.match(p).span(), (0,256))
+ def test_re_escape_non_ascii(self):
+ s = 'xxx\u2620\u2620\u2620xxx'
+ s_escaped = re.escape(s)
+ self.assertEqual(s_escaped, 'xxx\\\u2620\\\u2620\\\u2620xxx')
+ self.assertMatch(s_escaped, s)
+ self.assertMatch('.%s+.' % re.escape('\u2620'), s,
+ 'x\u2620\u2620\u2620x', (2, 7), re.search)
+
+ def test_re_escape_non_ascii_bytes(self):
+ b = 'y\u2620y\u2620y'.encode('utf-8')
+ b_escaped = re.escape(b)
+ self.assertEqual(b_escaped, b'y\\\xe2\\\x98\\\xa0y\\\xe2\\\x98\\\xa0y')
+ self.assertMatch(b_escaped, b)
+ res = re.findall(re.escape('\u2620'.encode('utf-8')), b)
+ self.assertEqual(len(res), 2)
def pickle_test(self, pickle):
oldpat = re.compile('a(?:b|(c|e){1,2}?|d)+?(.)')
diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
index 01e670e..7ca3d92 100644
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -1226,6 +1226,7 @@
stdout=subprocess.PIPE,
bufsize=0)
f = p.stdout
+ self.addCleanup(f.close)
try:
self.assertEqual(f.read(4), b"appl")
self.assertIn(f, select.select([f], [], [], 0.0)[0])
diff --git a/Lib/test/test_urllibnet.py b/Lib/test/test_urllibnet.py
index c885103..32efb2b 100644
--- a/Lib/test/test_urllibnet.py
+++ b/Lib/test/test_urllibnet.py
@@ -12,6 +12,7 @@
class URLTimeoutTest(unittest.TestCase):
+ # XXX this test doesn't seem to test anything useful.
TIMEOUT = 30.0
@@ -24,7 +25,7 @@
def testURLread(self):
with support.transient_internet("www.python.org"):
f = urllib.request.urlopen("http://www.python.org/")
- x = f.read()
+ x = f.read()
class urlopenNetworkTests(unittest.TestCase):
"""Tests urllib.reqest.urlopen using the network.
@@ -43,8 +44,10 @@
def urlopen(self, *args, **kwargs):
resource = args[0]
- with support.transient_internet(resource):
- return urllib.request.urlopen(*args, **kwargs)
+ cm = support.transient_internet(resource)
+ cm.__enter__()
+ self.addCleanup(cm.__exit__, None, None, None)
+ return urllib.request.urlopen(*args, **kwargs)
def test_basic(self):
# Simple test expected to pass.
@@ -135,8 +138,10 @@
def urlretrieve(self, *args):
resource = args[0]
- with support.transient_internet(resource):
- return urllib.request.urlretrieve(*args)
+ cm = support.transient_internet(resource)
+ cm.__enter__()
+ self.addCleanup(cm.__exit__, None, None, None)
+ return urllib.request.urlretrieve(*args)
def test_basic(self):
# Test basic functionality.
diff --git a/Lib/test/test_xdrlib.py b/Lib/test/test_xdrlib.py
index 073448c..6004c9f 100644
--- a/Lib/test/test_xdrlib.py
+++ b/Lib/test/test_xdrlib.py
@@ -12,6 +12,7 @@
a = [b'what', b'is', b'hapnin', b'doctor']
p.pack_int(42)
+ p.pack_int(-17)
p.pack_uint(9)
p.pack_bool(True)
p.pack_bool(False)
@@ -29,6 +30,7 @@
self.assertEqual(up.get_position(), 0)
self.assertEqual(up.unpack_int(), 42)
+ self.assertEqual(up.unpack_int(), -17)
self.assertEqual(up.unpack_uint(), 9)
self.assertTrue(up.unpack_bool() is True)
diff --git a/Lib/xdrlib.py b/Lib/xdrlib.py
index b293e06..4e48677 100644
--- a/Lib/xdrlib.py
+++ b/Lib/xdrlib.py
@@ -50,7 +50,9 @@
def pack_uint(self, x):
self.__buf.write(struct.pack('>L', x))
- pack_int = pack_uint
+ def pack_int(self, x):
+ self.__buf.write(struct.pack('>l', x))
+
pack_enum = pack_int
def pack_bool(self, x):
diff --git a/Misc/ACKS b/Misc/ACKS
index f2bf3cd..028f99a 100644
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -331,6 +331,7 @@
Fabian Groffen
Eric Groo
Dag Gruneau
+Filip Gruszczyński
Michael Guravage
Lars Gustäbel
Thomas Güttler
@@ -354,6 +355,7 @@
Larry Hastings
Shane Hathaway
Rycharde Hawkes
+Ben Hayden
Jochen Hayek
Christian Heimes
Thomas Heller
@@ -568,6 +570,7 @@
Greg McFarlane
Alan McIntyre
Michael McLay
+Mark Mc Mahon
Gordon McMillan
Caolan McNamara
Andrew McNamara
diff --git a/Misc/NEWS b/Misc/NEWS
index 04c9fae..1047046 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -49,11 +49,34 @@
Library
-------
-- Issue #11662: Make urllib and urllib2 ignore redirections if the
- scheme is not HTTP, HTTPS or FTP (CVE-2011-1521).
+- Issue #11256: Fix inspect.getcallargs on functions that take only keyword
+ arguments.
+
+- Issue #11696: Fix ID generation in msilib.
+
+- Issue #9696: Fix exception incorrectly raised by xdrlib.Packer.pack_int when
+ trying to pack a negative (in-range) integer.
+
+- Issue #11675: multiprocessing.[Raw]Array objects created from an integer size
+ are now zeroed on creation. This matches the behaviour specified by the
+ documentation.
+
+- Issue #7639: Fix short file name generation in bdist_msi
+
+- Issue #11659: Fix ResourceWarning in test_subprocess introduced by #11459.
+ Patch by Ben Hayden.
+
+- Issue #11635: Don't use polling in worker threads and processes launched by
+ concurrent.futures.
- Issue #11628: cmp_to_key generated class should use __slots__
+- Issue #11666: let help() display named tuple attributes and methods
+ that start with a leading underscore.
+
+- Issue #11662: Make urllib and urllib2 ignore redirections if the
+ scheme is not HTTP, HTTPS or FTP (CVE-2011-1521).
+
- Issue #5537: Fix time2isoz() and time2netscape() functions of
httplib.cookiejar for expiration year greater than 2038 on 32-bit systems.
@@ -148,6 +171,15 @@
- Issue #10276: Fix the results of zlib.crc32() and zlib.adler32() on buffers
larger than 4GB. Patch by Nadeem Vawda.
+- Issue #9348: Raise an early error if argparse nargs and metavar don't match.
+
+- Issue #8982: Improve the documentation for the argparse Namespace object.
+
+- Issue #9343: Document that argparse parent parsers must be configured before
+ their children.
+
+- Issue #9026: Fix order of argparse sub-commands in help messages.
+
Build
-----
@@ -161,6 +193,12 @@
- Issue #11179: Make ccbench work under Python 3.1 and 2.7 again.
+Extensions
+----------
+
+- Issue #1838: Prevent segfault in ctypes, when _as_parameter_ on a class is set
+ to an instance of the class.
+
Tests
-----
diff --git a/Modules/_ctypes/_ctypes.c b/Modules/_ctypes/_ctypes.c
index f613613..8e85980 100644
--- a/Modules/_ctypes/_ctypes.c
+++ b/Modules/_ctypes/_ctypes.c
@@ -585,7 +585,10 @@
CDataType_from_param(PyObject *type, PyObject *value)
{
PyObject *as_parameter;
- if (1 == PyObject_IsInstance(value, type)) {
+ int res = PyObject_IsInstance(value, type);
+ if (res == -1)
+ return NULL;
+ if (res) {
Py_INCREF(value);
return value;
}
@@ -598,10 +601,14 @@
/* If we got a PyCArgObject, we must check if the object packed in it
is an instance of the type's dict->proto */
- if(dict && ob
- && PyObject_IsInstance(ob, dict->proto)) {
- Py_INCREF(value);
- return value;
+ if(dict && ob) {
+ res = PyObject_IsInstance(ob, dict->proto);
+ if (res == -1)
+ return NULL;
+ if (res) {
+ Py_INCREF(value);
+ return value;
+ }
}
ob_name = (ob) ? Py_TYPE(ob)->tp_name : "???";
PyErr_Format(PyExc_TypeError,
@@ -951,8 +958,7 @@
Py_INCREF(value); /* _byref steals a refcount */
return _byref(value);
case -1:
- PyErr_Clear();
- break;
+ return NULL;
default:
break;
}
@@ -1431,6 +1437,7 @@
c_wchar_p_from_param(PyObject *type, PyObject *value)
{
PyObject *as_parameter;
+ int res;
if (value == Py_None) {
Py_INCREF(Py_None);
return Py_None;
@@ -1451,7 +1458,10 @@
}
return (PyObject *)parg;
}
- if (PyObject_IsInstance(value, type)) {
+ res = PyObject_IsInstance(value, type);
+ if (res == -1)
+ return NULL;
+ if (res) {
Py_INCREF(value);
return value;
}
@@ -1492,6 +1502,7 @@
c_char_p_from_param(PyObject *type, PyObject *value)
{
PyObject *as_parameter;
+ int res;
if (value == Py_None) {
Py_INCREF(Py_None);
return Py_None;
@@ -1512,7 +1523,10 @@
}
return (PyObject *)parg;
}
- if (PyObject_IsInstance(value, type)) {
+ res = PyObject_IsInstance(value, type);
+ if (res == -1)
+ return NULL;
+ if (res) {
Py_INCREF(value);
return value;
}
@@ -1554,6 +1568,7 @@
{
StgDictObject *stgd;
PyObject *as_parameter;
+ int res;
/* None */
if (value == Py_None) {
@@ -1631,7 +1646,10 @@
return (PyObject *)parg;
}
/* c_void_p instance (or subclass) */
- if (PyObject_IsInstance(value, type)) {
+ res = PyObject_IsInstance(value, type);
+ if (res == -1)
+ return NULL;
+ if (res) {
/* c_void_p instances */
Py_INCREF(value);
return value;
@@ -1990,10 +2008,14 @@
PyCArgObject *parg;
struct fielddesc *fd;
PyObject *as_parameter;
+ int res;
/* If the value is already an instance of the requested type,
we can use it as is */
- if (1 == PyObject_IsInstance(value, type)) {
+ res = PyObject_IsInstance(value, type);
+ if (res == -1)
+ return NULL;
+ if (res) {
Py_INCREF(value);
return value;
}
@@ -2022,7 +2044,12 @@
as_parameter = PyObject_GetAttrString(value, "_as_parameter_");
if (as_parameter) {
+ if (Py_EnterRecursiveCall("while processing _as_parameter_")) {
+ Py_DECREF(as_parameter);
+ return NULL;
+ }
value = PyCSimpleType_from_param(type, as_parameter);
+ Py_LeaveRecursiveCall();
Py_DECREF(as_parameter);
return value;
}
@@ -2714,6 +2741,7 @@
Py_ssize_t size, char *ptr)
{
CDataObject *src;
+ int err;
if (setfunc)
return setfunc(ptr, value, size);
@@ -2754,7 +2782,10 @@
}
src = (CDataObject *)value;
- if (PyObject_IsInstance(value, type)) {
+ err = PyObject_IsInstance(value, type);
+ if (err == -1)
+ return NULL;
+ if (err) {
memcpy(ptr,
src->b_ptr,
size);
@@ -4749,14 +4780,17 @@
stgdict = PyObject_stgdict((PyObject *)self);
assert(stgdict); /* Cannot be NULL fr pointer instances */
assert(stgdict->proto);
- if (!CDataObject_Check(value)
- || 0 == PyObject_IsInstance(value, stgdict->proto)) {
- /* XXX PyObject_IsInstance could return -1! */
- PyErr_Format(PyExc_TypeError,
- "expected %s instead of %s",
- ((PyTypeObject *)(stgdict->proto))->tp_name,
- Py_TYPE(value)->tp_name);
- return -1;
+ if (!CDataObject_Check(value)) {
+ int res = PyObject_IsInstance(value, stgdict->proto);
+ if (res == -1)
+ return -1;
+ if (!res) {
+ PyErr_Format(PyExc_TypeError,
+ "expected %s instead of %s",
+ ((PyTypeObject *)(stgdict->proto))->tp_name,
+ Py_TYPE(value)->tp_name);
+ return -1;
+ }
}
dst = (CDataObject *)value;
diff --git a/Modules/getbuildinfo.c b/Modules/getbuildinfo.c
index ac51881..7069b6e 100644
--- a/Modules/getbuildinfo.c
+++ b/Modules/getbuildinfo.c
@@ -42,9 +42,9 @@
const char *
Py_GetBuildInfo(void)
{
- static char buildinfo[50 + sizeof HGVERSION +
- ((sizeof HGTAG > sizeof HGBRANCH) ?
- sizeof HGTAG : sizeof HGBRANCH)];
+ static char buildinfo[50 + sizeof(HGVERSION) +
+ ((sizeof(HGTAG) > sizeof(HGBRANCH)) ?
+ sizeof(HGTAG) : sizeof(HGBRANCH))];
const char *revision = _Py_hgversion();
const char *sep = *revision ? ":" : "";
const char *hgid = _Py_hgidentifier();
diff --git a/Objects/typeslots.inc b/Objects/typeslots.inc
index 5186dcf..0494a32 100644
--- a/Objects/typeslots.inc
+++ b/Objects/typeslots.inc
@@ -1,4 +1,4 @@
-/* Generated by typeslots.py $Revision$ */
+/* Generated by typeslots.py $Revision: 87806 $ */
0,
0,
offsetof(PyHeapTypeObject, as_mapping.mp_ass_subscript),
diff --git a/Tools/buildbot/clean-amd64.bat b/Tools/buildbot/clean-amd64.bat
index 9fb35e9..715805a 100644
--- a/Tools/buildbot/clean-amd64.bat
+++ b/Tools/buildbot/clean-amd64.bat
@@ -1,7 +1,10 @@
@rem Used by the buildbot "clean" step.
call "%VS90COMNTOOLS%\..\..\VC\vcvarsall.bat" x86_amd64
-cd PCbuild
@echo Deleting .pyc/.pyo files ...
del /s Lib\*.pyc Lib\*.pyo
+@echo Deleting test leftovers ...
+rmdir /s /q build
+cd PCbuild
vcbuild /clean pcbuild.sln "Release|x64"
vcbuild /clean pcbuild.sln "Debug|x64"
+cd ..
diff --git a/Tools/buildbot/clean.bat b/Tools/buildbot/clean.bat
index ec71804..5c8f33e 100644
--- a/Tools/buildbot/clean.bat
+++ b/Tools/buildbot/clean.bat
@@ -2,6 +2,9 @@
call "%VS90COMNTOOLS%vsvars32.bat"
@echo Deleting .pyc/.pyo files ...
del /s Lib\*.pyc Lib\*.pyo
+@echo Deleting test leftovers ...
+rmdir /s /q build
cd PCbuild
vcbuild /clean pcbuild.sln "Release|Win32"
vcbuild /clean pcbuild.sln "Debug|Win32"
+cd ..