Branch merge
diff --git a/Doc/library/array.rst b/Doc/library/array.rst
index d563cce..75f680e 100644
--- a/Doc/library/array.rst
+++ b/Doc/library/array.rst
@@ -14,37 +14,49 @@
:dfn:`type code`, which is a single character. The following type codes are
defined:
-+-----------+----------------+-------------------+-----------------------+
-| Type code | C Type | Python Type | Minimum size in bytes |
-+===========+================+===================+=======================+
-| ``'b'`` | signed char | int | 1 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'B'`` | unsigned char | int | 1 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'u'`` | Py_UNICODE | Unicode character | 2 (see note) |
-+-----------+----------------+-------------------+-----------------------+
-| ``'h'`` | signed short | int | 2 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'H'`` | unsigned short | int | 2 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'i'`` | signed int | int | 2 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'I'`` | unsigned int | int | 2 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'l'`` | signed long | int | 4 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'L'`` | unsigned long | int | 4 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'f'`` | float | float | 4 |
-+-----------+----------------+-------------------+-----------------------+
-| ``'d'`` | double | float | 8 |
-+-----------+----------------+-------------------+-----------------------+
++-----------+--------------------+-------------------+-----------------------+-------+
+| Type code | C Type | Python Type | Minimum size in bytes | Notes |
++===========+====================+===================+=======================+=======+
+| ``'b'`` | signed char | int | 1 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'B'`` | unsigned char | int | 1 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'u'`` | Py_UNICODE | Unicode character | 2 | \(1) |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'h'`` | signed short | int | 2 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'H'`` | unsigned short | int | 2 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'i'`` | signed int | int | 2 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'I'`` | unsigned int | int | 2 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'l'`` | signed long | int | 4 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'L'`` | unsigned long | int | 4 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'q'`` | signed long long | int | 8 | \(2) |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'Q'`` | unsigned long long | int | 8 | \(2) |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'f'`` | float | float | 4 | |
++-----------+--------------------+-------------------+-----------------------+-------+
+| ``'d'`` | double | float | 8 | |
++-----------+--------------------+-------------------+-----------------------+-------+
-.. note::
+Notes:
- The ``'u'`` typecode corresponds to Python's unicode character. On narrow
+(1)
+ The ``'u'`` type code corresponds to Python's unicode character. On narrow
Unicode builds this is 2-bytes, on wide builds this is 4-bytes.
+(2)
+ The ``'q'`` and ``'Q'`` type codes are available only if
+ the platform C compiler used to build Python supports C :ctype:`long long`,
+ or, on Windows, :ctype:`__int64`.
+
+ .. versionadded:: 3.3
+
The actual representation of values is determined by the machine architecture
(strictly speaking, by the C implementation). The actual size can be accessed
through the :attr:`itemsize` attribute.
diff --git a/Doc/library/math.rst b/Doc/library/math.rst
index d68cf11..62c0f34 100644
--- a/Doc/library/math.rst
+++ b/Doc/library/math.rst
@@ -191,6 +191,11 @@
.. versionadded:: 3.3
+ .. seealso::
+
+ :meth:`int.bit_length` returns the number of bits necessary to represent
+ an integer in binary, excluding the sign and leading zeros.
+
.. function:: log10(x)
diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst
index 6fdc5aa..717e4e4 100644
--- a/Doc/whatsnew/3.3.rst
+++ b/Doc/whatsnew/3.3.rst
@@ -68,6 +68,15 @@
* Stub
+array
+-----
+
+The :mod:`array` module supports the :c:type:`long long` type using ``q`` and
+``Q`` type codes.
+
+(Contributed by Oren Tirosh and Hirokazu Yamamoto in :issue:`1172711`)
+
+
codecs
------
@@ -94,17 +103,21 @@
crypt
-----
-Addition of salf and modular crypt format to the :mod:`crypt` module.
+Addition of salt and modular crypt format and the :func:`~crypt.mksalt`
+function to the :mod:`crypt` module.
(:issue:`10924`)
curses
------
-The :class:`curses.window` class has a new :meth:`~curses.window.get_wch` method
-to get a wide character. Patch by Iñigo Serna.
+ * The :class:`curses.window` class has a new :meth:`~curses.window.get_wch`
+ method to get a wide character
+ * The :mod:`curses` module has a new :meth:`~curses.unget_wch` function to
+ push a wide character so the next :meth:`~curses.window.get_wch` will return
+ it
-(:issue:`6755`)
+(Contributed by Iñigo Serna in :issue:`6755`)
faulthandler
------------
@@ -115,6 +128,17 @@
* :option:`-X` ``faulthandler``
+ftplib
+------
+
+The :class:`~ftplib.FTP_TLS` class now provides a new
+:func:`~ftplib.FTP_TLS.ccc` function to revert control channel back to
+plaintex. This can be useful to take advantage of firewalls that know how to
+handle NAT with non-secure FTP without opening fixed ports.
+
+(Contributed by Giampaolo Rodolà in :issue:`12139`)
+
+
math
----
@@ -242,17 +266,6 @@
* :func:`~ssl.RAND_pseudo_bytes`: generate pseudo-random bytes.
-ftplib
-------
-
-The :class:`~ftplib.FTP_TLS` class now provides a new
-:func:`~ftplib.FTP_TLS.ccc` function to revert control channel back to
-plaintex. This can be useful to take advantage of firewalls that know how to
-handle NAT with non-secure FTP without opening fixed ports.
-
-(Contributed by Giampaolo Rodolà in :issue:`12139`)
-
-
shutil
------
diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py
index ae2d135..13d3d77 100644
--- a/Lib/multiprocessing/connection.py
+++ b/Lib/multiprocessing/connection.py
@@ -422,7 +422,7 @@
def _send_bytes(self, buf):
# For wire compatibility with 3.2 and lower
n = len(buf)
- self._send(struct.pack("=i", len(buf)))
+ self._send(struct.pack("!i", n))
# The condition is necessary to avoid "broken pipe" errors
# when sending a 0-length buffer if the other end closed the pipe.
if n > 0:
@@ -430,7 +430,7 @@
def _recv_bytes(self, maxsize=None, sentinels=()):
buf = self._recv(4, sentinels)
- size, = struct.unpack("=i", buf.getvalue())
+ size, = struct.unpack("!i", buf.getvalue())
if maxsize is not None and size > maxsize:
return None
return self._recv(size, sentinels)
diff --git a/Lib/test/test_array.py b/Lib/test/test_array.py
index 5190c35..604dcdf 100755
--- a/Lib/test/test_array.py
+++ b/Lib/test/test_array.py
@@ -16,6 +16,13 @@
import array
from array import _array_reconstructor as array_reconstructor
+try:
+ # Try to determine availability of long long independently
+ # of the array module under test
+ struct.calcsize('@q')
+ have_long_long = True
+except struct.error:
+ have_long_long = False
class ArraySubclass(array.array):
pass
@@ -26,6 +33,8 @@
tests = [] # list to accumulate all tests
typecodes = "ubBhHiIlLfd"
+if have_long_long:
+ typecodes += 'qQ'
class BadConstructorTest(unittest.TestCase):
@@ -1205,6 +1214,18 @@
minitemsize = 4
tests.append(UnsignedLongTest)
+@unittest.skipIf(not have_long_long, 'need long long support')
+class LongLongTest(SignedNumberTest):
+ typecode = 'q'
+ minitemsize = 8
+tests.append(LongLongTest)
+
+@unittest.skipIf(not have_long_long, 'need long long support')
+class UnsignedLongLongTest(UnsignedNumberTest):
+ typecode = 'Q'
+ minitemsize = 8
+tests.append(UnsignedLongLongTest)
+
class FPTest(NumberTest):
example = [-42.0, 0, 42, 1e5, -1e10]
smallerexample = [-42.0, 0, 42, 1e5, -2e10]
diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py
index 066a6b7..1729743 100644
--- a/Lib/test/test_httplib.py
+++ b/Lib/test/test_httplib.py
@@ -588,6 +588,7 @@
self.assertEqual(b'body\xc1', f.read())
def test_file_body(self):
+ self.addCleanup(support.unlink, support.TESTFN)
with open(support.TESTFN, "w") as f:
f.write("body")
with open(support.TESTFN) as f:
@@ -599,6 +600,7 @@
self.assertEqual(b'body', f.read())
def test_binary_file_body(self):
+ self.addCleanup(support.unlink, support.TESTFN)
with open(support.TESTFN, "wb") as f:
f.write(b"body\xc1")
with open(support.TESTFN, "rb") as f:
diff --git a/Lib/test/test_multiprocessing.py b/Lib/test/test_multiprocessing.py
index 3f8a9a2..b211861 100644
--- a/Lib/test/test_multiprocessing.py
+++ b/Lib/test/test_multiprocessing.py
@@ -35,7 +35,13 @@
import multiprocessing.heap
import multiprocessing.pool
-from multiprocessing import util, reduction
+from multiprocessing import util
+
+try:
+ from multiprocessing import reduction
+ HAS_REDUCTION = True
+except ImportError:
+ HAS_REDUCTION = False
try:
from multiprocessing.sharedctypes import Value, copy
@@ -1631,6 +1637,7 @@
os.write(fd, data)
os.close(fd)
+ @unittest.skipUnless(HAS_REDUCTION, "test needs multiprocessing.reduction")
def test_fd_transfer(self):
if self.TYPE != 'processes':
self.skipTest("only makes sense with processes")
@@ -1639,6 +1646,7 @@
p = self.Process(target=self._writefd, args=(child_conn, b"foo"))
p.daemon = True
p.start()
+ self.addCleanup(test.support.unlink, test.support.TESTFN)
with open(test.support.TESTFN, "wb") as f:
fd = f.fileno()
if msvcrt:
@@ -1648,6 +1656,7 @@
with open(test.support.TESTFN, "rb") as f:
self.assertEqual(f.read(), b"foo")
+ @unittest.skipUnless(HAS_REDUCTION, "test needs multiprocessing.reduction")
@unittest.skipIf(sys.platform == "win32",
"test semantics don't make sense on Windows")
@unittest.skipIf(MAXFD <= 256,
@@ -1663,6 +1672,7 @@
p = self.Process(target=self._writefd, args=(child_conn, b"bar", True))
p.daemon = True
p.start()
+ self.addCleanup(test.support.unlink, test.support.TESTFN)
with open(test.support.TESTFN, "wb") as f:
fd = f.fileno()
for newfd in range(256, MAXFD):
@@ -1679,6 +1689,23 @@
with open(test.support.TESTFN, "rb") as f:
self.assertEqual(f.read(), b"bar")
+ @classmethod
+ def _send_data_without_fd(self, conn):
+ os.write(conn.fileno(), b"\0")
+
+ @unittest.skipIf(sys.platform == "win32", "doesn't make sense on Windows")
+ def test_missing_fd_transfer(self):
+ # Check that exception is raised when received data is not
+ # accompanied by a file descriptor in ancillary data.
+ if self.TYPE != 'processes':
+ self.skipTest("only makes sense with processes")
+ conn, child_conn = self.Pipe(duplex=True)
+
+ p = self.Process(target=self._send_data_without_fd, args=(child_conn,))
+ p.daemon = True
+ p.start()
+ self.assertRaises(RuntimeError, reduction.recv_handle, conn)
+ p.join()
class _TestListenerClient(BaseTestCase):
@@ -1987,10 +2014,12 @@
'multiprocessing', 'multiprocessing.connection',
'multiprocessing.heap', 'multiprocessing.managers',
'multiprocessing.pool', 'multiprocessing.process',
- 'multiprocessing.reduction',
'multiprocessing.synchronize', 'multiprocessing.util'
]
+ if HAS_REDUCTION:
+ modules.append('multiprocessing.reduction')
+
if c_int is not None:
# This module requires _ctypes
modules.append('multiprocessing.sharedctypes')
diff --git a/Misc/NEWS b/Misc/NEWS
index c786d24..571db4b 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -10,6 +10,13 @@
Core and Builtins
-----------------
+- Issue #13021: Missing decref on an error path. Thanks to Suman Saha for
+ finding the bug and providing a patch.
+
+- Issue #12973: Fix overflow check that relied on undefined behaviour in
+ list_repeat. This bug caused test_list to fail with recent versions
+ of Clang.
+
- Issue #12904: os.utime, os.futimes, os.lutimes, and os.futimesat now write
atime and mtime with nanosecond precision on modern POSIX platforms.
@@ -1280,6 +1287,12 @@
Extension Modules
-----------------
+- Issue #13022: Fix: _multiprocessing.recvfd() doesn't check that
+ file descriptor was actually received.
+
+- Issue #1172711: Add 'long long' support to the array module.
+ Initial patch by Oren Tirosh and Hirokazu Yamamoto.
+
- Issue #12483: ctypes: Fix a crash when the destruction of a callback
object triggers the garbage collector.
diff --git a/Modules/_multiprocessing/multiprocessing.c b/Modules/_multiprocessing/multiprocessing.c
index f6c2259..a1e6ed5 100644
--- a/Modules/_multiprocessing/multiprocessing.c
+++ b/Modules/_multiprocessing/multiprocessing.c
@@ -166,6 +166,17 @@
if (res < 0)
return PyErr_SetFromErrno(PyExc_OSError);
+ if (msg.msg_controllen < CMSG_LEN(sizeof(int)) ||
+ (cmsg = CMSG_FIRSTHDR(&msg)) == NULL ||
+ cmsg->cmsg_level != SOL_SOCKET ||
+ cmsg->cmsg_type != SCM_RIGHTS ||
+ cmsg->cmsg_len < CMSG_LEN(sizeof(int))) {
+ /* If at least one control message is present, there should be
+ no room for any further data in the buffer. */
+ PyErr_SetString(PyExc_RuntimeError, "No file descriptor received");
+ return NULL;
+ }
+
fd = * (int *) CMSG_DATA(cmsg);
return Py_BuildValue("i", fd);
}
diff --git a/Modules/arraymodule.c b/Modules/arraymodule.c
index 81c9c36..5748a3c 100644
--- a/Modules/arraymodule.c
+++ b/Modules/arraymodule.c
@@ -356,6 +356,59 @@
return 0;
}
+#ifdef HAVE_LONG_LONG
+
+static PyObject *
+q_getitem(arrayobject *ap, Py_ssize_t i)
+{
+ return PyLong_FromLongLong(((PY_LONG_LONG *)ap->ob_item)[i]);
+}
+
+static int
+q_setitem(arrayobject *ap, Py_ssize_t i, PyObject *v)
+{
+ PY_LONG_LONG x;
+ if (!PyArg_Parse(v, "L;array item must be integer", &x))
+ return -1;
+ if (i >= 0)
+ ((PY_LONG_LONG *)ap->ob_item)[i] = x;
+ return 0;
+}
+
+static PyObject *
+QQ_getitem(arrayobject *ap, Py_ssize_t i)
+{
+ return PyLong_FromUnsignedLongLong(
+ ((unsigned PY_LONG_LONG *)ap->ob_item)[i]);
+}
+
+static int
+QQ_setitem(arrayobject *ap, Py_ssize_t i, PyObject *v)
+{
+ unsigned PY_LONG_LONG x;
+ if (PyLong_Check(v)) {
+ x = PyLong_AsUnsignedLongLong(v);
+ if (x == (unsigned PY_LONG_LONG) -1 && PyErr_Occurred())
+ return -1;
+ }
+ else {
+ PY_LONG_LONG y;
+ if (!PyArg_Parse(v, "L;array item must be integer", &y))
+ return -1;
+ if (y < 0) {
+ PyErr_SetString(PyExc_OverflowError,
+ "unsigned long long is less than minimum");
+ return -1;
+ }
+ x = (unsigned PY_LONG_LONG)y;
+ }
+
+ if (i >= 0)
+ ((unsigned PY_LONG_LONG *)ap->ob_item)[i] = x;
+ return 0;
+}
+#endif
+
static PyObject *
f_getitem(arrayobject *ap, Py_ssize_t i)
{
@@ -406,6 +459,10 @@
{'I', sizeof(int), II_getitem, II_setitem, "I", 1, 0},
{'l', sizeof(long), l_getitem, l_setitem, "l", 1, 1},
{'L', sizeof(long), LL_getitem, LL_setitem, "L", 1, 0},
+#ifdef HAVE_LONG_LONG
+ {'q', sizeof(PY_LONG_LONG), q_getitem, q_setitem, "q", 1, 1},
+ {'Q', sizeof(PY_LONG_LONG), QQ_getitem, QQ_setitem, "Q", 1, 0},
+#endif
{'f', sizeof(float), f_getitem, f_setitem, "f", 0, 0},
{'d', sizeof(double), d_getitem, d_setitem, "d", 0, 0},
{'\0', 0, 0, 0, 0, 0, 0} /* Sentinel */
@@ -1655,6 +1712,16 @@
intsize = sizeof(long);
is_signed = 0;
break;
+#if HAVE_LONG_LONG
+ case 'q':
+ intsize = sizeof(PY_LONG_LONG);
+ is_signed = 1;
+ break;
+ case 'Q':
+ intsize = sizeof(PY_LONG_LONG);
+ is_signed = 0;
+ break;
+#endif
default:
return UNKNOWN_FORMAT;
}
@@ -2501,7 +2568,11 @@
}
}
PyErr_SetString(PyExc_ValueError,
+#ifdef HAVE_LONG_LONG
+ "bad typecode (must be b, B, u, h, H, i, I, l, L, q, Q, f or d)");
+#else
"bad typecode (must be b, B, u, h, H, i, I, l, L, f or d)");
+#endif
return NULL;
}
@@ -2524,12 +2595,18 @@
'I' unsigned integer 2 \n\
'l' signed integer 4 \n\
'L' unsigned integer 4 \n\
+ 'q' signed integer 8 (see note) \n\
+ 'Q' unsigned integer 8 (see note) \n\
'f' floating point 4 \n\
'd' floating point 8 \n\
\n\
-NOTE: The 'u' typecode corresponds to Python's unicode character. On \n\
+NOTE: The 'u' type code corresponds to Python's unicode character. On \n\
narrow builds this is 2-bytes on wide builds this is 4-bytes.\n\
\n\
+NOTE: The 'q' and 'Q' type codes are only available if the platform \n\
+C compiler used to build Python supports 'long long', or, on Windows, \n\
+'__int64'.\n\
+\n\
The constructor is:\n\
\n\
array(typecode [, initializer]) -- create a new array\n\
diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c
index 2fc509d..25e7f0d 100644
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -8071,16 +8071,20 @@
static PyObject *
posix_statvfs(PyObject *self, PyObject *args)
{
- char *path;
+ PyObject *path;
int res;
struct statvfs st;
- if (!PyArg_ParseTuple(args, "s:statvfs", &path))
+ if (!PyArg_ParseTuple(args, "O&:statvfs", PyUnicode_FSConverter, &path))
return NULL;
Py_BEGIN_ALLOW_THREADS
- res = statvfs(path, &st);
+ res = statvfs(PyBytes_AS_STRING(path), &st);
Py_END_ALLOW_THREADS
- if (res != 0)
- return posix_error_with_filename(path);
+ if (res != 0) {
+ posix_error_with_filename(PyBytes_AS_STRING(path));
+ Py_DECREF(path);
+ return NULL;
+ }
+ Py_DECREF(path);
return _pystatvfs_fromstructstatvfs(st);
}
diff --git a/Objects/listobject.c b/Objects/listobject.c
index 2cd8642..cada708 100644
--- a/Objects/listobject.c
+++ b/Objects/listobject.c
@@ -58,7 +58,7 @@
if (newsize == 0)
new_allocated = 0;
items = self->ob_item;
- if (new_allocated <= ((~(size_t)0) / sizeof(PyObject *)))
+ if (new_allocated <= (PY_SIZE_MAX / sizeof(PyObject *)))
PyMem_RESIZE(items, PyObject *, new_allocated);
else
items = NULL;
@@ -510,9 +510,9 @@
PyObject *elem;
if (n < 0)
n = 0;
- size = Py_SIZE(a) * n;
- if (n && size/n != Py_SIZE(a))
+ if (n > 0 && Py_SIZE(a) > PY_SSIZE_T_MAX / n)
return PyErr_NoMemory();
+ size = Py_SIZE(a) * n;
if (size == 0)
return PyList_New(0);
np = (PyListObject *) PyList_New(size);
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
index 5649e86..1888fba 100644
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -1251,8 +1251,10 @@
Py_DECREF(f);
return -1;
}
- if (PyDict_SetItemString(d, "__cached__", Py_None) < 0)
+ if (PyDict_SetItemString(d, "__cached__", Py_None) < 0) {
+ Py_DECREF(f);
return -1;
+ }
set_file_name = 1;
Py_DECREF(f);
}