Banch merge
diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst
index f4edced..c9e3864 100644
--- a/Doc/library/collections.rst
+++ b/Doc/library/collections.rst
@@ -293,7 +293,7 @@
Counter({'b': 4})
.. versionadded:: 3.3
- Added support for unary plus and unary minus.
+ Added support for unary plus, unary minus, and in-place multiset operations.
.. note::
diff --git a/Doc/library/functools.rst b/Doc/library/functools.rst
index 2316e80..4eaf54e 100644
--- a/Doc/library/functools.rst
+++ b/Doc/library/functools.rst
@@ -40,7 +40,7 @@
.. versionadded:: 3.2
-.. decorator:: lru_cache(maxsize=100)
+.. decorator:: lru_cache(maxsize=100, typed=False)
Decorator to wrap a function with a memoizing callable that saves up to the
*maxsize* most recent calls. It can save time when an expensive or I/O bound
@@ -52,6 +52,10 @@
If *maxsize* is set to None, the LRU feature is disabled and the cache
can grow without bound.
+ If *typed* is set to True, function arguments of different types will be
+ cached separately. For example, ``f(3)`` and ``f(3.0)`` will be treated
+ as distinct calls with distinct results.
+
To help measure the effectiveness of the cache and tune the *maxsize*
parameter, the wrapped function is instrumented with a :func:`cache_info`
function that returns a :term:`named tuple` showing *hits*, *misses*,
@@ -67,8 +71,8 @@
An `LRU (least recently used) cache
<http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used>`_ works
- best when more recent calls are the best predictors of upcoming calls (for
- example, the most popular articles on a news server tend to change daily).
+ best when the most recent calls are the best predictors of upcoming calls (for
+ example, the most popular articles on a news server tend to change each day).
The cache's size limit assures that the cache does not grow without bound on
long-running processes such as web servers.
@@ -111,6 +115,9 @@
.. versionadded:: 3.2
+ .. versionchanged:: 3.3
+ Added the *typed* option.
+
.. decorator:: total_ordering
Given a class defining one or more rich comparison ordering methods, this
diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst
index 68cfa6e..4998949 100644
--- a/Doc/library/logging.rst
+++ b/Doc/library/logging.rst
@@ -189,6 +189,9 @@
Logs a message with level :const:`WARNING` on this logger. The arguments are
interpreted as for :meth:`debug`.
+ .. note:: There is an obsolete method `warn()` which is functionally
+ identical to `warning()`. As `warn()` is deprecated, please do not use
+ it - use `warning()` instead.
.. method:: Logger.error(msg, *args, **kwargs)
@@ -880,8 +883,12 @@
.. function:: warning(msg, *args, **kwargs)
- Logs a message with level :const:`WARNING` on the root logger. The arguments are
- interpreted as for :func:`debug`.
+ Logs a message with level :const:`WARNING` on the root logger. The arguments
+ are interpreted as for :func:`debug`.
+
+ .. note:: There is an obsolete function `warn()` which is functionally
+ identical to `warning()`. As `warn()` is deprecated, please do not use
+ it - use `warning()` instead.
.. function:: error(msg, *args, **kwargs)
diff --git a/Doc/library/os.rst b/Doc/library/os.rst
index 6adfdb1..b2a16d9 100644
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -1369,6 +1369,8 @@
These constants are only available on Unix.
+ .. versionchanged:: 3.3
+ Add :data:`O_CLOEXEC` constant.
.. data:: O_BINARY
O_NOINHERIT
diff --git a/Doc/library/re.rst b/Doc/library/re.rst
index 412e6ee..331ffa7 100644
--- a/Doc/library/re.rst
+++ b/Doc/library/re.rst
@@ -80,7 +80,7 @@
characters either stand for classes of ordinary characters, or affect
how the regular expressions around them are interpreted. Regular
expression pattern strings may not contain null bytes, but can specify
-the null byte using the ``\number`` notation, e.g., ``'\x00'``.
+the null byte using a ``\number`` notation such as ``'\x00'``.
The special characters are:
@@ -161,30 +161,36 @@
raw strings for all but the simplest expressions.
``[]``
- Used to indicate a set of characters. Characters can be listed individually, or
- a range of characters can be indicated by giving two characters and separating
- them by a ``'-'``. Special characters are not active inside sets. For example,
- ``[akm$]`` will match any of the characters ``'a'``, ``'k'``,
- ``'m'``, or ``'$'``; ``[a-z]`` will match any lowercase letter, and
- ``[a-zA-Z0-9]`` matches any letter or digit. Character classes such
- as ``\w`` or ``\S`` (defined below) are also acceptable inside a
- range, although the characters they match depends on whether
- :const:`ASCII` or :const:`LOCALE` mode is in force. If you want to
- include a ``']'`` or a ``'-'`` inside a set, precede it with a
- backslash, or place it as the first character. The pattern ``[]]``
- will match ``']'``, for example.
+ Used to indicate a set of characters. In a set:
- You can match the characters not within a range by :dfn:`complementing` the set.
- This is indicated by including a ``'^'`` as the first character of the set;
- ``'^'`` elsewhere will simply match the ``'^'`` character. For example,
- ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any
- character except ``'^'``.
+ * Characters can be listed individually, e.g. ``[amk]`` will match ``'a'``,
+ ``'m'``, or ``'k'``.
- Note that inside ``[]`` the special forms and special characters lose
- their meanings and only the syntaxes described here are valid. For
- example, ``+``, ``*``, ``(``, ``)``, and so on are treated as
- literals inside ``[]``, and backreferences cannot be used inside
- ``[]``.
+ * Ranges of characters can be indicated by giving two characters and separating
+ them by a ``'-'``, for example ``[a-z]`` will match any lowercase ASCII letter,
+ ``[0-5][0-9]`` will match all the two-digits numbers from ``00`` to ``59``, and
+ ``[0-9A-Fa-f]`` will match any hexadecimal digit. If ``-`` is escaped (e.g.
+ ``[a\-z]``) or if it's placed as the first or last character (e.g. ``[a-]``),
+ it will match a literal ``'-'``.
+
+ * Special characters lose their special meaning inside sets. For example,
+ ``[(+*)]`` will match any of the literal characters ``'('``, ``'+'``,
+ ``'*'``, or ``')'``.
+
+ * Character classes such as ``\w`` or ``\S`` (defined below) are also accepted
+ inside a set, although the characters they match depends on whether
+ :const:`ASCII` or :const:`LOCALE` mode is in force.
+
+ * Characters that are not within a range can be matched by :dfn:`complementing`
+ the set. If the first character of the set is ``'^'``, all the characters
+ that are *not* in the set will be matched. For example, ``[^5]`` will match
+ any character except ``'5'``, and ``[^^]`` will match any character except
+ ``'^'``. ``^`` has no special meaning if it's not the first character in
+ the set.
+
+ * To match a literal ``']'`` inside a set, precede it with a backslash, or
+ place it at the beginning of the set. For example, both ``[()[\]{}]`` and
+ ``[]()[{}]`` will both match a parenthesis.
``'|'``
``A|B``, where A and B can be arbitrary REs, creates a regular expression that
@@ -405,7 +411,7 @@
\r \t \v \x
\\
-Octal escapes are included in a limited form: If the first digit is a 0, or if
+Octal escapes are included in a limited form. If the first digit is a 0, or if
there are three octal digits, it is considered an octal escape. Otherwise, it is
a group reference. As for string literals, octal escapes are always at most
three digits in length.
@@ -413,8 +419,8 @@
.. _matching-searching:
-Matching vs Searching
----------------------
+Matching vs. Searching
+----------------------
.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
@@ -595,8 +601,7 @@
['', '...', 'words', ', ', 'words', '...', '']
That way, separator components are always found at the same relative
- indices within the result list (e.g., if there's one capturing group
- in the separator, the 0th, the 2nd and so forth).
+ indices within the result list.
Note that *split* will never split a string on an empty pattern match.
For example:
@@ -716,7 +721,7 @@
--------------------------
Compiled regular expression objects support the following methods and
-attributes.
+attributes:
.. method:: regex.search(string[, pos[, endpos]])
@@ -735,7 +740,7 @@
The optional parameter *endpos* limits how far the string will be searched; it
will be as if the string is *endpos* characters long, so only the characters
from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
- than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
+ than *pos*, no match will be found; otherwise, if *rx* is a compiled regular
expression object, ``rx.search(string, 0, 50)`` is equivalent to
``rx.search(string[:50], 0)``.
@@ -823,8 +828,8 @@
Match Objects
-------------
-Match objects always have a boolean value of :const:`True`, so that you can test
-whether e.g. :func:`match` resulted in a match with a simple if statement. They
+Match objects always have a boolean value of :const:`True`. This lets you
+use a simple if-statement to test whether a match was found. Match objects
support the following methods and attributes:
@@ -1001,7 +1006,7 @@
---------------------------
-Checking For a Pair
+Checking for a Pair
^^^^^^^^^^^^^^^^^^^
In this example, we'll use the following helper function to display match
@@ -1109,7 +1114,7 @@
If you create regular expressions that require the engine to perform a lot of
recursion, you may encounter a :exc:`RuntimeError` exception with the message
-``maximum recursion limit`` exceeded. For example, ::
+``maximum recursion limit exceeded``. For example, ::
>>> s = 'Begin ' + 1000*'a very long string ' + 'end'
>>> re.match('Begin (\w| )*? end', s).end()
diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
index 783d544..2a14c46 100644
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -1805,6 +1805,12 @@
Wherever one of these methods needs to interpret the bytes as characters
(e.g. the :func:`is...` methods), the ASCII character set is assumed.
+.. versionadded:: 3.3
+ The functions :func:`count`, :func:`find`, :func:`index`,
+ :func:`rfind` and :func:`rindex` have additional semantics compared to
+ the corresponding string functions: They also accept an integer in
+ range 0 to 255 (a byte) as their first argument.
+
.. note::
The methods on bytes and bytearray objects don't accept strings as their
diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst
index a20bda6..48751e8 100644
--- a/Doc/whatsnew/3.2.rst
+++ b/Doc/whatsnew/3.2.rst
@@ -2362,7 +2362,7 @@
(Patch by Florent Xicluna in :issue:`7622` and :issue:`7462`.)
-* String to integer conversions now work two "digits" at a time, reducing the
+* Integer to string conversions now work two "digits" at a time, reducing the
number of division and modulo operations.
(:issue:`6713` by Gawain Bolton, Mark Dickinson, and Victor Stinner.)
diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py
index 3e864b6..68b63a8 100644
--- a/Lib/collections/__init__.py
+++ b/Lib/collections/__init__.py
@@ -683,6 +683,69 @@
'''
return Counter() - self
+ def _keep_positive(self):
+ '''Internal method to strip elements with a negative or zero count'''
+ nonpositive = [elem for elem, count in self.items() if not count > 0]
+ for elem in nonpositive:
+ del self[elem]
+ return self
+
+ def __iadd__(self, other):
+ '''Inplace add from another counter, keeping only positive counts.
+
+ >>> c = Counter('abbb')
+ >>> c += Counter('bcc')
+ >>> c
+ Counter({'b': 4, 'c': 2, 'a': 1})
+
+ '''
+ for elem, count in other.items():
+ self[elem] += count
+ return self._keep_positive()
+
+ def __isub__(self, other):
+ '''Inplace subtract counter, but keep only results with positive counts.
+
+ >>> c = Counter('abbbc')
+ >>> c -= Counter('bccd')
+ >>> c
+ Counter({'b': 2, 'a': 1})
+
+ '''
+ for elem, count in other.items():
+ self[elem] -= count
+ return self._keep_positive()
+
+ def __ior__(self, other):
+ '''Inplace union is the maximum of value from either counter.
+
+ >>> c = Counter('abbb')
+ >>> c |= Counter('bcc')
+ >>> c
+ Counter({'b': 3, 'c': 2, 'a': 1})
+
+ '''
+ for elem, other_count in other.items():
+ count = self[elem]
+ if other_count > count:
+ self[elem] = other_count
+ return self._keep_positive()
+
+ def __iand__(self, other):
+ '''Inplace intersection is the minimum of corresponding counts.
+
+ >>> c = Counter('abbb')
+ >>> c &= Counter('bcc')
+ >>> c
+ Counter({'b': 1})
+
+ '''
+ for elem, count in self.items():
+ other_count = other[elem]
+ if other_count < count:
+ self[elem] = other_count
+ return self._keep_positive()
+
########################################################################
### ChainMap (helper for configparser and string.Template)
diff --git a/Lib/fnmatch.py b/Lib/fnmatch.py
index 726fbe5..3df20d8 100644
--- a/Lib/fnmatch.py
+++ b/Lib/fnmatch.py
@@ -36,8 +36,8 @@
return fnmatchcase(name, pat)
@functools.lru_cache(maxsize=250)
-def _compile_pattern(pat, is_bytes=False):
- if is_bytes:
+def _compile_pattern(pat):
+ if isinstance(pat, bytes):
pat_str = str(pat, 'ISO-8859-1')
res_str = translate(pat_str)
res = bytes(res_str, 'ISO-8859-1')
@@ -49,7 +49,7 @@
"""Return the subset of the list NAMES that match PAT."""
result = []
pat = os.path.normcase(pat)
- match = _compile_pattern(pat, isinstance(pat, bytes))
+ match = _compile_pattern(pat)
if os.path is posixpath:
# normcase on posix is NOP. Optimize it away from the loop.
for name in names:
@@ -67,7 +67,7 @@
This is a version of fnmatch() which doesn't case-normalize
its arguments.
"""
- match = _compile_pattern(pat, isinstance(pat, bytes))
+ match = _compile_pattern(pat)
return match(name) is not None
diff --git a/Lib/functools.py b/Lib/functools.py
index 038f284..1abb37a 100644
--- a/Lib/functools.py
+++ b/Lib/functools.py
@@ -121,12 +121,16 @@
_CacheInfo = namedtuple("CacheInfo", "hits misses maxsize currsize")
-def lru_cache(maxsize=100):
+def lru_cache(maxsize=100, typed=False):
"""Least-recently-used cache decorator.
If *maxsize* is set to None, the LRU features are disabled and the cache
can grow without bound.
+ If *typed* is True, arguments of different types will be cached separately.
+ For example, f(3.0) and f(3) will be treated as distinct calls with
+ distinct results.
+
Arguments to the cached function must be hashable.
View the cache statistics named tuple (hits, misses, maxsize, currsize) with
@@ -142,7 +146,7 @@
# to allow the implementation to change (including a possible C version).
def decorating_function(user_function,
- tuple=tuple, sorted=sorted, len=len, KeyError=KeyError):
+ *, tuple=tuple, sorted=sorted, map=map, len=len, type=type, KeyError=KeyError):
hits = misses = 0
kwd_mark = (object(),) # separates positional and keyword args
@@ -156,7 +160,12 @@
nonlocal hits, misses
key = args
if kwds:
- key += kwd_mark + tuple(sorted(kwds.items()))
+ sorted_items = tuple(sorted(kwds.items()))
+ key += kwd_mark + sorted_items
+ if typed:
+ key += tuple(map(type, args))
+ if kwds:
+ key += tuple(type(v) for k, v in sorted_items)
try:
result = cache[key]
hits += 1
@@ -177,7 +186,12 @@
nonlocal hits, misses
key = args
if kwds:
- key += kwd_mark + tuple(sorted(kwds.items()))
+ sorted_items = tuple(sorted(kwds.items()))
+ key += kwd_mark + sorted_items
+ if typed:
+ key += tuple(map(type, args))
+ if kwds:
+ key += tuple(type(v) for k, v in sorted_items)
with lock:
try:
result = cache[key]
diff --git a/Lib/importlib/_bootstrap.py b/Lib/importlib/_bootstrap.py
index 824a31c..1e22438 100644
--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -81,7 +81,9 @@
def _write_atomic(path, data):
- """Best-effort function to write data to a path atomically."""
+ """Best-effort function to write data to a path atomically.
+ Be prepared to handle a FileExistsError if concurrent writing of the
+ temporary file is attempted."""
if not sys.platform.startswith('win'):
# On POSIX-like platforms, renaming is atomic
path_tmp = path + '.tmp'
@@ -516,12 +518,10 @@
raise
try:
_write_atomic(path, data)
- except OSError as exc:
- # Don't worry if you can't write bytecode.
- if exc.errno == errno.EACCES:
- return
- else:
- raise
+ except (PermissionError, FileExistsError):
+ # Don't worry if you can't write bytecode or someone is writing
+ # it at the same time.
+ pass
class _SourcelessFileLoader(_FileLoader, _LoaderBasics):
diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py
index 8406df3..6e0394f 100644
--- a/Lib/logging/__init__.py
+++ b/Lib/logging/__init__.py
@@ -1243,7 +1243,10 @@
if self.isEnabledFor(WARNING):
self._log(WARNING, msg, args, **kwargs)
- warn = warning
+ def warn(self, msg, *args, **kwargs):
+ warnings.warn("The 'warn' method is deprecated, "
+ "use 'warning' instead", PendingDeprecationWarning, 2)
+ self.warning(msg, *args, **kwargs)
def error(self, msg, *args, **kwargs):
"""
@@ -1556,7 +1559,10 @@
"""
self.log(WARNING, msg, *args, **kwargs)
- warn = warning
+ def warn(self, msg, *args, **kwargs):
+ warnings.warn("The 'warn' method is deprecated, "
+ "use 'warning' instead", PendingDeprecationWarning, 2)
+ self.warning(msg, *args, **kwargs)
def error(self, msg, *args, **kwargs):
"""
@@ -1766,7 +1772,10 @@
basicConfig()
root.warning(msg, *args, **kwargs)
-warn = warning
+def warn(msg, *args, **kwargs):
+ warnings.warn("The 'warn' function is deprecated, "
+ "use 'warning' instead", PendingDeprecationWarning, 2)
+ warning(msg, *args, **kwargs)
def info(msg, *args, **kwargs):
"""
diff --git a/Lib/re.py b/Lib/re.py
index cdf5976..4b90b3f 100644
--- a/Lib/re.py
+++ b/Lib/re.py
@@ -207,7 +207,7 @@
def purge():
"Clear the regular expression caches"
- _compile_typed.cache_clear()
+ _compile.cache_clear()
_compile_repl.cache_clear()
def template(pattern, flags=0):
@@ -253,11 +253,8 @@
_pattern_type = type(sre_compile.compile("", 0))
+@functools.lru_cache(maxsize=500, typed=True)
def _compile(pattern, flags):
- return _compile_typed(type(pattern), pattern, flags)
-
-@functools.lru_cache(maxsize=500)
-def _compile_typed(text_bytes_type, pattern, flags):
# internal: compile pattern
if isinstance(pattern, _pattern_type):
if flags:
diff --git a/Lib/smtpd.py b/Lib/smtpd.py
index 32f45ae..df835b2 100755
--- a/Lib/smtpd.py
+++ b/Lib/smtpd.py
@@ -678,6 +678,16 @@
if __name__ == '__main__':
options = parseargs()
# Become nobody
+ classname = options.classname
+ if "." in classname:
+ lastdot = classname.rfind(".")
+ mod = __import__(classname[:lastdot], globals(), locals(), [""])
+ classname = classname[lastdot+1:]
+ else:
+ import __main__ as mod
+ class_ = getattr(mod, classname)
+ proxy = class_((options.localhost, options.localport),
+ (options.remotehost, options.remoteport))
if options.setuid:
try:
import pwd
@@ -691,16 +701,6 @@
if e.errno != errno.EPERM: raise
print('Cannot setuid "nobody"; try running with -n option.', file=sys.stderr)
sys.exit(1)
- classname = options.classname
- if "." in classname:
- lastdot = classname.rfind(".")
- mod = __import__(classname[:lastdot], globals(), locals(), [""])
- classname = classname[lastdot+1:]
- else:
- import __main__ as mod
- class_ = getattr(mod, classname)
- proxy = class_((options.localhost, options.localport),
- (options.remotehost, options.remoteport))
try:
asyncore.loop()
except KeyboardInterrupt:
diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
index 36c7aa4..276e683 100644
--- a/Lib/sysconfig.py
+++ b/Lib/sysconfig.py
@@ -323,7 +323,6 @@
"""Generate the Python module containing build-time variables."""
import pprint
vars = {}
- destfile = os.path.join(os.path.dirname(__file__), '_sysconfigdata.py')
# load the installed Makefile:
makefile = get_makefile_filename()
try:
@@ -348,7 +347,11 @@
# the scripts are in another directory.
if _PYTHON_BUILD:
vars['LDSHARED'] = vars['BLDSHARED']
+
+ destfile = os.path.join(os.path.dirname(__file__), '_sysconfigdata.py')
with open(destfile, 'w', encoding='utf8') as f:
+ f.write('# system configuration generated and used by'
+ ' the sysconfig module\n')
f.write('build_time_vars = ')
pprint.pprint(vars, stream=f)
diff --git a/Lib/test/string_tests.py b/Lib/test/string_tests.py
index 049929c..43ecf2a 100644
--- a/Lib/test/string_tests.py
+++ b/Lib/test/string_tests.py
@@ -28,6 +28,11 @@
# Change in subclasses to change the behaviour of fixtesttype()
type2test = None
+ # Whether the "contained items" of the container are integers in
+ # range(0, 256) (i.e. bytes, bytearray) or strings of length 1
+ # (str)
+ contains_bytes = False
+
# All tests pass their arguments to the testing methods
# as str objects. fixtesttype() can be used to propagate
# these arguments to the appropriate type
@@ -117,7 +122,11 @@
self.checkequal(0, '', 'count', 'xx', sys.maxsize, 0)
self.checkraises(TypeError, 'hello', 'count')
- self.checkraises(TypeError, 'hello', 'count', 42)
+
+ if self.contains_bytes:
+ self.checkequal(0, 'hello', 'count', 42)
+ else:
+ self.checkraises(TypeError, 'hello', 'count', 42)
# For a variety of combinations,
# verify that str.count() matches an equivalent function
@@ -163,7 +172,11 @@
self.checkequal( 2, 'rrarrrrrrrrra', 'find', 'a', None, 6)
self.checkraises(TypeError, 'hello', 'find')
- self.checkraises(TypeError, 'hello', 'find', 42)
+
+ if self.contains_bytes:
+ self.checkequal(-1, 'hello', 'find', 42)
+ else:
+ self.checkraises(TypeError, 'hello', 'find', 42)
self.checkequal(0, '', 'find', '')
self.checkequal(-1, '', 'find', '', 1, 1)
@@ -217,7 +230,11 @@
self.checkequal( 2, 'rrarrrrrrrrra', 'rfind', 'a', None, 6)
self.checkraises(TypeError, 'hello', 'rfind')
- self.checkraises(TypeError, 'hello', 'rfind', 42)
+
+ if self.contains_bytes:
+ self.checkequal(-1, 'hello', 'rfind', 42)
+ else:
+ self.checkraises(TypeError, 'hello', 'rfind', 42)
# For a variety of combinations,
# verify that str.rfind() matches __contains__
@@ -264,7 +281,11 @@
self.checkequal( 2, 'rrarrrrrrrrra', 'index', 'a', None, 6)
self.checkraises(TypeError, 'hello', 'index')
- self.checkraises(TypeError, 'hello', 'index', 42)
+
+ if self.contains_bytes:
+ self.checkraises(ValueError, 'hello', 'index', 42)
+ else:
+ self.checkraises(TypeError, 'hello', 'index', 42)
def test_rindex(self):
self.checkequal(12, 'abcdefghiabc', 'rindex', '')
@@ -286,7 +307,11 @@
self.checkequal( 2, 'rrarrrrrrrrra', 'rindex', 'a', None, 6)
self.checkraises(TypeError, 'hello', 'rindex')
- self.checkraises(TypeError, 'hello', 'rindex', 42)
+
+ if self.contains_bytes:
+ self.checkraises(ValueError, 'hello', 'rindex', 42)
+ else:
+ self.checkraises(TypeError, 'hello', 'rindex', 42)
def test_lower(self):
self.checkequal('hello', 'HeLLo', 'lower')
diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py
index d32a44b..7acfde5 100644
--- a/Lib/test/test_bytes.py
+++ b/Lib/test/test_bytes.py
@@ -293,10 +293,27 @@
def test_count(self):
b = self.type2test(b'mississippi')
+ i = 105
+ p = 112
+ w = 119
+
self.assertEqual(b.count(b'i'), 4)
self.assertEqual(b.count(b'ss'), 2)
self.assertEqual(b.count(b'w'), 0)
+ self.assertEqual(b.count(i), 4)
+ self.assertEqual(b.count(w), 0)
+
+ self.assertEqual(b.count(b'i', 6), 2)
+ self.assertEqual(b.count(b'p', 6), 2)
+ self.assertEqual(b.count(b'i', 1, 3), 1)
+ self.assertEqual(b.count(b'p', 7, 9), 1)
+
+ self.assertEqual(b.count(i, 6), 2)
+ self.assertEqual(b.count(p, 6), 2)
+ self.assertEqual(b.count(i, 1, 3), 1)
+ self.assertEqual(b.count(p, 7, 9), 1)
+
def test_startswith(self):
b = self.type2test(b'hello')
self.assertFalse(self.type2test().startswith(b"anything"))
@@ -327,35 +344,81 @@
def test_find(self):
b = self.type2test(b'mississippi')
+ i = 105
+ w = 119
+
self.assertEqual(b.find(b'ss'), 2)
- self.assertEqual(b.find(b'ss', 3), 5)
- self.assertEqual(b.find(b'ss', 1, 7), 2)
- self.assertEqual(b.find(b'ss', 1, 3), -1)
self.assertEqual(b.find(b'w'), -1)
self.assertEqual(b.find(b'mississippian'), -1)
+ self.assertEqual(b.find(i), 1)
+ self.assertEqual(b.find(w), -1)
+
+ self.assertEqual(b.find(b'ss', 3), 5)
+ self.assertEqual(b.find(b'ss', 1, 7), 2)
+ self.assertEqual(b.find(b'ss', 1, 3), -1)
+
+ self.assertEqual(b.find(i, 6), 7)
+ self.assertEqual(b.find(i, 1, 3), 1)
+ self.assertEqual(b.find(w, 1, 3), -1)
+
def test_rfind(self):
b = self.type2test(b'mississippi')
+ i = 105
+ w = 119
+
self.assertEqual(b.rfind(b'ss'), 5)
- self.assertEqual(b.rfind(b'ss', 3), 5)
- self.assertEqual(b.rfind(b'ss', 0, 6), 2)
self.assertEqual(b.rfind(b'w'), -1)
self.assertEqual(b.rfind(b'mississippian'), -1)
+ self.assertEqual(b.rfind(i), 10)
+ self.assertEqual(b.rfind(w), -1)
+
+ self.assertEqual(b.rfind(b'ss', 3), 5)
+ self.assertEqual(b.rfind(b'ss', 0, 6), 2)
+
+ self.assertEqual(b.rfind(i, 1, 3), 1)
+ self.assertEqual(b.rfind(i, 3, 9), 7)
+ self.assertEqual(b.rfind(w, 1, 3), -1)
+
def test_index(self):
- b = self.type2test(b'world')
- self.assertEqual(b.index(b'w'), 0)
- self.assertEqual(b.index(b'orl'), 1)
- self.assertRaises(ValueError, b.index, b'worm')
- self.assertRaises(ValueError, b.index, b'ldo')
+ b = self.type2test(b'mississippi')
+ i = 105
+ w = 119
+
+ self.assertEqual(b.index(b'ss'), 2)
+ self.assertRaises(ValueError, b.index, b'w')
+ self.assertRaises(ValueError, b.index, b'mississippian')
+
+ self.assertEqual(b.index(i), 1)
+ self.assertRaises(ValueError, b.index, w)
+
+ self.assertEqual(b.index(b'ss', 3), 5)
+ self.assertEqual(b.index(b'ss', 1, 7), 2)
+ self.assertRaises(ValueError, b.index, b'ss', 1, 3)
+
+ self.assertEqual(b.index(i, 6), 7)
+ self.assertEqual(b.index(i, 1, 3), 1)
+ self.assertRaises(ValueError, b.index, w, 1, 3)
def test_rindex(self):
- # XXX could be more rigorous
- b = self.type2test(b'world')
- self.assertEqual(b.rindex(b'w'), 0)
- self.assertEqual(b.rindex(b'orl'), 1)
- self.assertRaises(ValueError, b.rindex, b'worm')
- self.assertRaises(ValueError, b.rindex, b'ldo')
+ b = self.type2test(b'mississippi')
+ i = 105
+ w = 119
+
+ self.assertEqual(b.rindex(b'ss'), 5)
+ self.assertRaises(ValueError, b.rindex, b'w')
+ self.assertRaises(ValueError, b.rindex, b'mississippian')
+
+ self.assertEqual(b.rindex(i), 10)
+ self.assertRaises(ValueError, b.rindex, w)
+
+ self.assertEqual(b.rindex(b'ss', 3), 5)
+ self.assertEqual(b.rindex(b'ss', 0, 6), 2)
+
+ self.assertEqual(b.rindex(i, 1, 3), 1)
+ self.assertEqual(b.rindex(i, 3, 9), 7)
+ self.assertRaises(ValueError, b.rindex, w, 1, 3)
def test_replace(self):
b = self.type2test(b'mississippi')
@@ -552,6 +615,14 @@
self.assertEqual(True, b.startswith(h, None, -2))
self.assertEqual(False, b.startswith(x, None, None))
+ def test_integer_arguments_out_of_byte_range(self):
+ b = self.type2test(b'hello')
+
+ for method in (b.count, b.find, b.index, b.rfind, b.rindex):
+ self.assertRaises(ValueError, method, -1)
+ self.assertRaises(ValueError, method, 256)
+ self.assertRaises(ValueError, method, 9999)
+
def test_find_etc_raise_correct_error_messages(self):
# issue 11828
b = self.type2test(b'hello')
@@ -1161,9 +1232,11 @@
class ByteArrayAsStringTest(FixedStringTest):
type2test = bytearray
+ contains_bytes = True
class BytesAsStringTest(FixedStringTest):
type2test = bytes
+ contains_bytes = True
class SubclassTest(unittest.TestCase):
diff --git a/Lib/test/test_collections.py b/Lib/test/test_collections.py
index 04c4d97..ec209389 100644
--- a/Lib/test/test_collections.py
+++ b/Lib/test/test_collections.py
@@ -932,6 +932,27 @@
set_result = setop(set(p.elements()), set(q.elements()))
self.assertEqual(counter_result, dict.fromkeys(set_result, 1))
+ def test_inplace_operations(self):
+ elements = 'abcd'
+ for i in range(1000):
+ # test random pairs of multisets
+ p = Counter(dict((elem, randrange(-2,4)) for elem in elements))
+ p.update(e=1, f=-1, g=0)
+ q = Counter(dict((elem, randrange(-2,4)) for elem in elements))
+ q.update(h=1, i=-1, j=0)
+ for inplace_op, regular_op in [
+ (Counter.__iadd__, Counter.__add__),
+ (Counter.__isub__, Counter.__sub__),
+ (Counter.__ior__, Counter.__or__),
+ (Counter.__iand__, Counter.__and__),
+ ]:
+ c = p.copy()
+ c_id = id(c)
+ regular_result = regular_op(c, q)
+ inplace_result = inplace_op(c, q)
+ self.assertEqual(inplace_result, regular_result)
+ self.assertEqual(id(inplace_result), c_id)
+
def test_subtract(self):
c = Counter(a=-5, b=0, c=5, d=10, e=15,g=40)
c.subtract(a=1, b=2, c=-3, d=10, e=20, f=30, h=-50)
diff --git a/Lib/test/test_functools.py b/Lib/test/test_functools.py
index a31c92e..c4d9fe6 100644
--- a/Lib/test/test_functools.py
+++ b/Lib/test/test_functools.py
@@ -734,6 +734,22 @@
with self.assertRaises(IndexError):
func(15)
+ def test_lru_with_types(self):
+ for maxsize in (None, 100):
+ @functools.lru_cache(maxsize=maxsize, typed=True)
+ def square(x):
+ return x * x
+ self.assertEqual(square(3), 9)
+ self.assertEqual(type(square(3)), type(9))
+ self.assertEqual(square(3.0), 9.0)
+ self.assertEqual(type(square(3.0)), type(9.0))
+ self.assertEqual(square(x=3), 9)
+ self.assertEqual(type(square(x=3)), type(9))
+ self.assertEqual(square(x=3.0), 9.0)
+ self.assertEqual(type(square(x=3.0)), type(9.0))
+ self.assertEqual(square.cache_info().hits, 4)
+ self.assertEqual(square.cache_info().misses, 4)
+
def test_main(verbose=None):
test_classes = (
TestPartial,
diff --git a/Misc/NEWS b/Misc/NEWS
index 8774334..55b261e 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,13 @@
Core and Builtins
-----------------
+- Issue #12170: The count(), find(), rfind(), index() and rindex() methods
+ of bytes and bytearray objects now accept an integer between 0 and 255
+ as their first argument. Patch by Petri Lehtinen.
+
+- Issue #12604: VTRACE macro expanded to no-op in _sre.c to avoid compiler
+ warnings. Patch by Josh Triplett and Petri Lehtinen.
+
- Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
error handlers on all Windows versions. The MBCS codec is now supporting all
error handlers, instead of only replace to encode and ignore to decode.
@@ -319,6 +326,16 @@
Library
-------
+- Issue #13235: Added PendingDeprecationWarning to warn() method and function.
+
+- Issue #9168: now smtpd is able to bind privileged port.
+
+- Issue #12529: fix cgi.parse_header issue on strings with double-quotes and
+ semicolons together. Patch by Ben Darnell and Petri Lehtinen.
+
+- Issue #13227: functools.lru_cache() now has a option to distinguish
+ calls with different argument types.
+
- Issue #6090: zipfile raises a ValueError when a document with a timestamp
earlier than 1980 is provided. Patch contributed by Petri Lehtinen.
@@ -488,6 +505,7 @@
in os.kill().
- Add support for unary plus and unary minus to collections.Counter().
+ Issue #13121: Also add support for inplace math operators.
- Issue #12683: urlparse updated to include svn as schemes that uses relative
paths. (svn from 1.5 onwards support relative path).
diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c
index 0c9add0..6f0b28d 100644
--- a/Objects/bytearrayobject.c
+++ b/Objects/bytearrayobject.c
@@ -1071,24 +1071,41 @@
bytearray_find_internal(PyByteArrayObject *self, PyObject *args, int dir)
{
PyObject *subobj;
+ char byte;
Py_buffer subbuf;
+ const char *sub;
+ Py_ssize_t sub_len;
Py_ssize_t start=0, end=PY_SSIZE_T_MAX;
Py_ssize_t res;
- if (!stringlib_parse_args_finds("find/rfind/index/rindex",
- args, &subobj, &start, &end))
+ if (!stringlib_parse_args_finds_byte("find/rfind/index/rindex",
+ args, &subobj, &byte, &start, &end))
return -2;
- if (_getbuffer(subobj, &subbuf) < 0)
- return -2;
+
+ if (subobj) {
+ if (_getbuffer(subobj, &subbuf) < 0)
+ return -2;
+
+ sub = subbuf.buf;
+ sub_len = subbuf.len;
+ }
+ else {
+ sub = &byte;
+ sub_len = 1;
+ }
+
if (dir > 0)
res = stringlib_find_slice(
PyByteArray_AS_STRING(self), PyByteArray_GET_SIZE(self),
- subbuf.buf, subbuf.len, start, end);
+ sub, sub_len, start, end);
else
res = stringlib_rfind_slice(
PyByteArray_AS_STRING(self), PyByteArray_GET_SIZE(self),
- subbuf.buf, subbuf.len, start, end);
- PyBuffer_Release(&subbuf);
+ sub, sub_len, start, end);
+
+ if (subobj)
+ PyBuffer_Release(&subbuf);
+
return res;
}
@@ -1121,23 +1138,39 @@
bytearray_count(PyByteArrayObject *self, PyObject *args)
{
PyObject *sub_obj;
- const char *str = PyByteArray_AS_STRING(self);
+ const char *str = PyByteArray_AS_STRING(self), *sub;
+ Py_ssize_t sub_len;
+ char byte;
Py_ssize_t start = 0, end = PY_SSIZE_T_MAX;
+
Py_buffer vsub;
PyObject *count_obj;
- if (!stringlib_parse_args_finds("count", args, &sub_obj, &start, &end))
+ if (!stringlib_parse_args_finds_byte("count", args, &sub_obj, &byte,
+ &start, &end))
return NULL;
- if (_getbuffer(sub_obj, &vsub) < 0)
- return NULL;
+ if (sub_obj) {
+ if (_getbuffer(sub_obj, &vsub) < 0)
+ return NULL;
+
+ sub = vsub.buf;
+ sub_len = vsub.len;
+ }
+ else {
+ sub = &byte;
+ sub_len = 1;
+ }
ADJUST_INDICES(start, end, PyByteArray_GET_SIZE(self));
count_obj = PyLong_FromSsize_t(
- stringlib_count(str + start, end - start, vsub.buf, vsub.len, PY_SSIZE_T_MAX)
+ stringlib_count(str + start, end - start, sub, sub_len, PY_SSIZE_T_MAX)
);
- PyBuffer_Release(&vsub);
+
+ if (sub_obj)
+ PyBuffer_Release(&vsub);
+
return count_obj;
}
diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c
index 17e31b9..7438a70 100644
--- a/Objects/bytesobject.c
+++ b/Objects/bytesobject.c
@@ -1230,31 +1230,42 @@
bytes_find_internal(PyBytesObject *self, PyObject *args, int dir)
{
PyObject *subobj;
+ char byte;
+ Py_buffer subbuf;
const char *sub;
Py_ssize_t sub_len;
Py_ssize_t start=0, end=PY_SSIZE_T_MAX;
+ Py_ssize_t res;
- if (!stringlib_parse_args_finds("find/rfind/index/rindex",
- args, &subobj, &start, &end))
+ if (!stringlib_parse_args_finds_byte("find/rfind/index/rindex",
+ args, &subobj, &byte, &start, &end))
return -2;
- if (PyBytes_Check(subobj)) {
- sub = PyBytes_AS_STRING(subobj);
- sub_len = PyBytes_GET_SIZE(subobj);
+ if (subobj) {
+ if (_getbuffer(subobj, &subbuf) < 0)
+ return -2;
+
+ sub = subbuf.buf;
+ sub_len = subbuf.len;
}
- else if (PyObject_AsCharBuffer(subobj, &sub, &sub_len))
- /* XXX - the "expected a character buffer object" is pretty
- confusing for a non-expert. remap to something else ? */
- return -2;
+ else {
+ sub = &byte;
+ sub_len = 1;
+ }
if (dir > 0)
- return stringlib_find_slice(
+ res = stringlib_find_slice(
PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self),
sub, sub_len, start, end);
else
- return stringlib_rfind_slice(
+ res = stringlib_rfind_slice(
PyBytes_AS_STRING(self), PyBytes_GET_SIZE(self),
sub, sub_len, start, end);
+
+ if (subobj)
+ PyBuffer_Release(&subbuf);
+
+ return res;
}
@@ -1480,23 +1491,38 @@
PyObject *sub_obj;
const char *str = PyBytes_AS_STRING(self), *sub;
Py_ssize_t sub_len;
+ char byte;
Py_ssize_t start = 0, end = PY_SSIZE_T_MAX;
- if (!stringlib_parse_args_finds("count", args, &sub_obj, &start, &end))
+ Py_buffer vsub;
+ PyObject *count_obj;
+
+ if (!stringlib_parse_args_finds_byte("count", args, &sub_obj, &byte,
+ &start, &end))
return NULL;
- if (PyBytes_Check(sub_obj)) {
- sub = PyBytes_AS_STRING(sub_obj);
- sub_len = PyBytes_GET_SIZE(sub_obj);
+ if (sub_obj) {
+ if (_getbuffer(sub_obj, &vsub) < 0)
+ return NULL;
+
+ sub = vsub.buf;
+ sub_len = vsub.len;
}
- else if (PyObject_AsCharBuffer(sub_obj, &sub, &sub_len))
- return NULL;
+ else {
+ sub = &byte;
+ sub_len = 1;
+ }
ADJUST_INDICES(start, end, PyBytes_GET_SIZE(self));
- return PyLong_FromSsize_t(
+ count_obj = PyLong_FromSsize_t(
stringlib_count(str + start, end - start, sub, sub_len, PY_SSIZE_T_MAX)
);
+
+ if (sub_obj)
+ PyBuffer_Release(&vsub);
+
+ return count_obj;
}
diff --git a/Objects/genobject.c b/Objects/genobject.c
index b7765ef..c6612e2 100644
--- a/Objects/genobject.c
+++ b/Objects/genobject.c
@@ -232,8 +232,9 @@
/* First, check the traceback argument, replacing None with
NULL. */
- if (tb == Py_None)
+ if (tb == Py_None) {
tb = NULL;
+ }
else if (tb != NULL && !PyTraceBack_Check(tb)) {
PyErr_SetString(PyExc_TypeError,
"throw() third argument must be a traceback object");
@@ -244,9 +245,8 @@
Py_XINCREF(val);
Py_XINCREF(tb);
- if (PyExceptionClass_Check(typ)) {
+ if (PyExceptionClass_Check(typ))
PyErr_NormalizeException(&typ, &val, &tb);
- }
else if (PyExceptionInstance_Check(typ)) {
/* Raising an instance. The value should be a dummy. */
@@ -262,10 +262,9 @@
typ = PyExceptionInstance_Class(typ);
Py_INCREF(typ);
- if (tb == NULL) {
+ if (tb == NULL)
/* Returns NULL if there's no traceback */
tb = PyException_GetTraceback(val);
- }
}
}
else {
diff --git a/Objects/stringlib/find.h b/Objects/stringlib/find.h
index 7cce156..00eaf1b 100644
--- a/Objects/stringlib/find.h
+++ b/Objects/stringlib/find.h
@@ -167,4 +167,47 @@
return 0;
}
+#else /* !STRINGLIB_IS_UNICODE */
+
+/*
+Wraps stringlib_parse_args_finds() and additionally checks whether the
+first argument is an integer in range(0, 256).
+
+If this is the case, writes the integer value to the byte parameter
+and sets subobj to NULL. Otherwise, sets the first argument to subobj
+and doesn't touch byte. The other parameters are similar to those of
+stringlib_parse_args_finds().
+*/
+
+Py_LOCAL_INLINE(int)
+STRINGLIB(parse_args_finds_byte)(const char *function_name, PyObject *args,
+ PyObject **subobj, char *byte,
+ Py_ssize_t *start, Py_ssize_t *end)
+{
+ PyObject *tmp_subobj;
+ Py_ssize_t ival;
+
+ if(!STRINGLIB(parse_args_finds)(function_name, args, &tmp_subobj,
+ start, end))
+ return 0;
+
+ ival = PyNumber_AsSsize_t(tmp_subobj, PyExc_ValueError);
+ if (ival == -1 && PyErr_Occurred()) {
+ PyErr_Clear();
+ *subobj = tmp_subobj;
+ }
+ else {
+ /* The first argument was an integer */
+ if(ival < 0 || ival > 255) {
+ PyErr_SetString(PyExc_ValueError, "byte must be in range(0, 256)");
+ return 0;
+ }
+
+ *subobj = NULL;
+ *byte = (char)ival;
+ }
+
+ return 1;
+}
+
#endif /* STRINGLIB_IS_UNICODE */