Fixes issue #6766: Updated multiprocessing Proxy Objects to support nesting
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
index f886ecb..1813eeb 100644
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -1682,7 +1682,9 @@
of processes. Objects of this type are returned by
:func:`multiprocessing.Manager`.
- It also supports creation of shared lists and dictionaries.
+ Its methods create and return :ref:`multiprocessing-proxy_objects` for a
+ number of commonly used data types to be synchronized across processes.
+ This notably includes shared lists and dictionaries.
.. method:: Barrier(parties[, action[, timeout]])
@@ -1745,31 +1747,17 @@
dict(mapping)
dict(sequence)
- Create a shared ``dict`` object and return a proxy for it.
+ Create a shared :class:`dict` object and return a proxy for it.
.. method:: list()
list(sequence)
- Create a shared ``list`` object and return a proxy for it.
+ Create a shared :class:`list` object and return a proxy for it.
- .. note::
-
- Modifications to mutable values or items in dict and list proxies will not
- be propagated through the manager, because the proxy has no way of knowing
- when its values or items are modified. To modify such an item, you can
- re-assign the modified object to the container proxy::
-
- # create a list proxy and append a mutable object (a dictionary)
- lproxy = manager.list()
- lproxy.append({})
- # now mutate the dictionary
- d = lproxy[0]
- d['a'] = 1
- d['b'] = 2
- # at this point, the changes to d are not yet synced, but by
- # reassigning the dictionary, the proxy is notified of the change
- lproxy[0] = d
-
+ .. versionchanged:: 3.6
+ Shared objects are capable of being nested. For example, a shared
+ container object such as a shared list can contain other shared objects
+ which will all be managed and synchronized by the :class:`SyncManager`.
.. class:: Namespace
@@ -1881,6 +1869,8 @@
>>> s = m.get_server()
>>> s.serve_forever()
+.. _multiprocessing-proxy_objects:
+
Proxy Objects
~~~~~~~~~~~~~
@@ -1890,8 +1880,7 @@
A proxy object has methods which invoke corresponding methods of its referent
(although not every method of the referent will necessarily be available through
-the proxy). A proxy can usually be used in most of the same ways that its
-referent can:
+the proxy). In this way, a proxy can be used just like its referent can:
.. doctest::
@@ -1912,9 +1901,9 @@
the proxy.
An important feature of proxy objects is that they are picklable so they can be
-passed between processes. Note, however, that if a proxy is sent to the
-corresponding manager's process then unpickling it will produce the referent
-itself. This means, for example, that one shared object can contain a second:
+passed between processes. As such, a referent can contain
+:ref:`multiprocessing-proxy_objects`. This permits nesting of these managed
+lists, dicts, and other :ref:`multiprocessing-proxy_objects`:
.. doctest::
@@ -1922,10 +1911,46 @@
>>> b = manager.list()
>>> a.append(b) # referent of a now contains referent of b
>>> print(a, b)
- [[]] []
+ [<ListProxy object, typeid 'list' at ...>] []
>>> b.append('hello')
- >>> print(a, b)
- [['hello']] ['hello']
+ >>> print(a[0], b)
+ ['hello'] ['hello']
+
+Similarly, dict and list proxies may be nested inside one another::
+
+ >>> l_outer = manager.list([ manager.dict() for i in range(2) ])
+ >>> d_first_inner = l_outer[0]
+ >>> d_first_inner['a'] = 1
+ >>> d_first_inner['b'] = 2
+ >>> l_outer[1]['c'] = 3
+ >>> l_outer[1]['z'] = 26
+ >>> print(l_outer[0])
+ {'a': 1, 'b': 2}
+ >>> print(l_outer[1])
+ {'c': 3, 'z': 26}
+
+If standard (non-proxy) :class:`list` or :class:`dict` objects are contained
+in a referent, modifications to those mutable values will not be propagated
+through the manager because the proxy has no way of knowing when the values
+contained within are modified. However, storing a value in a container proxy
+(which triggers a ``__setitem__`` on the proxy object) does propagate through
+the manager and so to effectively modify such an item, one could re-assign the
+modified value to the container proxy::
+
+ # create a list proxy and append a mutable object (a dictionary)
+ lproxy = manager.list()
+ lproxy.append({})
+ # now mutate the dictionary
+ d = lproxy[0]
+ d['a'] = 1
+ d['b'] = 2
+ # at this point, the changes to d are not yet synced, but by
+ # updating the dictionary, the proxy is notified of the change
+ lproxy[0] = d
+
+This approach is perhaps less convenient than employing nested
+:ref:`multiprocessing-proxy_objects` for most use cases but also
+demonstrates a level of control over the synchronization.
.. note::