blob: aac454c8170692077fbd9c7fe199c51e8384898e [file] [log] [blame]
Eric Snow96c6af92015-05-29 22:21:39 -06001/* Ordered Dictionary object implementation.
2
3This implementation is necessarily explicitly equivalent to the pure Python
4OrderedDict class in Lib/collections/__init__.py. The strategy there
5involves using a doubly-linked-list to capture the order. We keep to that
6strategy, using a lower-level linked-list.
7
8About the Linked-List
9=====================
10
11For the linked list we use a basic doubly-linked-list. Using a circularly-
12linked-list does have some benefits, but they don't apply so much here
13since OrderedDict is focused on the ends of the list (for the most part).
14Furthermore, there are some features of generic linked-lists that we simply
15don't need for OrderedDict. Thus a simple custom implementation meets our
16needs. Alternatives to our simple approach include the QCIRCLE_*
17macros from BSD's queue.h, and the linux's list.h.
18
19Getting O(1) Node Lookup
20------------------------
21
22One invariant of Python's OrderedDict is that it preserves time complexity
23of dict's methods, particularly the O(1) operations. Simply adding a
24linked-list on top of dict is not sufficient here; operations for nodes in
25the middle of the linked-list implicitly require finding the node first.
26With a simple linked-list like we're using, that is an O(n) operation.
27Consequently, methods like __delitem__() would change from O(1) to O(n),
28which is unacceptable.
29
30In order to preserve O(1) performance for node removal (finding nodes), we
31must do better than just looping through the linked-list. Here are options
32we've considered:
33
341. use a second dict to map keys to nodes (a la the pure Python version).
352. keep a simple hash table mirroring the order of dict's, mapping each key
36 to the corresponding node in the linked-list.
373. use a version of shared keys (split dict) that allows non-unicode keys.
384. have the value stored for each key be a (value, node) pair, and adjust
39 __getitem__(), get(), etc. accordingly.
40
41The approach with the least performance impact (time and space) is #2,
Benjamin Petersonee178e62016-09-08 11:08:30 -070042mirroring the key order of dict's dk_entries with an array of node pointers.
Eric Snow96c6af92015-05-29 22:21:39 -060043While lookdict() and friends (dk_lookup) don't give us the index into the
44array, we make use of pointer arithmetic to get that index. An alternative
45would be to refactor lookdict() to provide the index, explicitly exposing
46the implementation detail. We could even just use a custom lookup function
47for OrderedDict that facilitates our need. However, both approaches are
48significantly more complicated than just using pointer arithmetic.
49
50The catch with mirroring the hash table ordering is that we have to keep
51the ordering in sync through any dict resizes. However, that order only
52matters during node lookup. We can simply defer any potential resizing
53until we need to do a lookup.
54
55Linked-List Nodes
56-----------------
57
58The current implementation stores a pointer to the associated key only.
59One alternative would be to store a pointer to the PyDictKeyEntry instead.
60This would save one pointer de-reference per item, which is nice during
61calls to values() and items(). However, it adds unnecessary overhead
62otherwise, so we stick with just the key.
63
64Linked-List API
65---------------
66
67As noted, the linked-list implemented here does not have all the bells and
68whistles. However, we recognize that the implementation may need to
69change to accommodate performance improvements or extra functionality. To
70that end, We use a simple API to interact with the linked-list. Here's a
71summary of the methods/macros:
72
73Node info:
74
75* _odictnode_KEY(node)
76* _odictnode_VALUE(od, node)
77* _odictnode_PREV(node)
78* _odictnode_NEXT(node)
79
80Linked-List info:
81
82* _odict_FIRST(od)
83* _odict_LAST(od)
84* _odict_EMPTY(od)
85* _odict_FOREACH(od, node) - used in place of `for (node=...)`
86
87For adding nodes:
88
89* _odict_add_head(od, node)
90* _odict_add_tail(od, node)
Serhiy Storchaka19a70e72015-11-13 14:48:36 +020091* _odict_add_new_node(od, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -060092
93For removing nodes:
94
Serhiy Storchaka19a70e72015-11-13 14:48:36 +020095* _odict_clear_node(od, node, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -060096* _odict_clear_nodes(od, clear_each)
97
98Others:
99
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200100* _odict_find_node_hash(od, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600101* _odict_find_node(od, key)
102* _odict_keys_equal(od1, od2)
103
104Used, but specific to the linked-list implementation:
105
106* _odict_free_fast_nodes(od)
107
108And here's a look at how the linked-list relates to the OrderedDict API:
109
110============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
111method key val prev next mem 1st last empty iter find add rmv clr keq
112============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
113__del__ ~ X
114__delitem__ free ~ node
115__eq__ ~ X
116__iter__ X X
117__new__ X X
118__reduce__ X ~ X
119__repr__ X X X
120__reversed__ X X
121__setitem__ key
122__sizeof__ size X
123clear ~ ~ X
124copy X X X
125items X X X
126keys X X
127move_to_end X X X ~ h/t key
128pop free key
129popitem X X free X X node
130setdefault ~ ? ~
131values X X
132============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
133
134__delitem__ is the only method that directly relies on finding an arbitrary
135node in the linked-list. Everything else is iteration or relates to the
136ends of the linked-list.
137
138Situation that Endangers Consistency
139------------------------------------
140Using a raw linked-list for OrderedDict exposes a key situation that can
141cause problems. If a node is stored in a variable, there is a chance that
142the node may have been deallocated before the variable gets used, thus
143potentially leading to a segmentation fault. A key place where this shows
144up is during iteration through the linked list (via _odict_FOREACH or
145otherwise).
146
147A number of solutions are available to resolve this situation:
148
149* defer looking up the node until as late as possible and certainly after
150 any code that could possibly result in a deletion;
151* if the node is needed both before and after a point where the node might
152 be removed, do a check before using the node at the "after" location to
153 see if the node is still valid;
154* like the last one, but simply pull the node again to ensure it's right;
155* keep the key in the variable instead of the node and then look up the
156 node using the key at the point where the node is needed (this is what
157 we do for the iterators).
158
159Another related problem, preserving consistent ordering during iteration,
160is described below. That one is not exclusive to using linked-lists.
161
162
163Challenges from Subclassing dict
164================================
165
166OrderedDict subclasses dict, which is an unusual relationship between two
167builtin types (other than the base object type). Doing so results in
168some complication and deserves further explanation. There are two things
169to consider here. First, in what circumstances or with what adjustments
170can OrderedDict be used as a drop-in replacement for dict (at the C level)?
171Second, how can the OrderedDict implementation leverage the dict
172implementation effectively without introducing unnecessary coupling or
173inefficiencies?
174
175This second point is reflected here and in the implementation, so the
176further focus is on the first point. It is worth noting that for
177overridden methods, the dict implementation is deferred to as much as
178possible. Furthermore, coupling is limited to as little as is reasonable.
179
180Concrete API Compatibility
181--------------------------
182
183Use of the concrete C-API for dict (PyDict_*) with OrderedDict is
184problematic. (See http://bugs.python.org/issue10977.) The concrete API
185has a number of hard-coded assumptions tied to the dict implementation.
186This is, in part, due to performance reasons, which is understandable
187given the part dict plays in Python.
188
189Any attempt to replace dict with OrderedDict for any role in the
190interpreter (e.g. **kwds) faces a challenge. Such any effort must
191recognize that the instances in affected locations currently interact with
192the concrete API.
193
194Here are some ways to address this challenge:
195
1961. Change the relevant usage of the concrete API in CPython and add
Martin Panter69332c12016-08-04 13:07:31 +0000197 PyDict_CheckExact() calls to each of the concrete API functions.
Eric Snow96c6af92015-05-29 22:21:39 -06001982. Adjust the relevant concrete API functions to explicitly accommodate
199 OrderedDict.
2003. As with #1, add the checks, but improve the abstract API with smart fast
201 paths for dict and OrderedDict, and refactor CPython to use the abstract
202 API. Improvements to the abstract API would be valuable regardless.
203
204Adding the checks to the concrete API would help make any interpreter
205switch to OrderedDict less painful for extension modules. However, this
206won't work. The equivalent C API call to `dict.__setitem__(obj, k, v)`
207is 'PyDict_SetItem(obj, k, v)`. This illustrates how subclasses in C call
208the base class's methods, since there is no equivalent of super() in the
209C API. Calling into Python for parent class API would work, but some
210extension modules already rely on this feature of the concrete API.
211
212For reference, here is a breakdown of some of the dict concrete API:
213
214========================== ============= =======================
215concrete API uses abstract API
216========================== ============= =======================
217PyDict_Check PyMapping_Check
218(PyDict_CheckExact) -
219(PyDict_New) -
220(PyDictProxy_New) -
221PyDict_Clear -
222PyDict_Contains PySequence_Contains
223PyDict_Copy -
224PyDict_SetItem PyObject_SetItem
225PyDict_SetItemString PyMapping_SetItemString
226PyDict_DelItem PyMapping_DelItem
227PyDict_DelItemString PyMapping_DelItemString
228PyDict_GetItem -
229PyDict_GetItemWithError PyObject_GetItem
230_PyDict_GetItemIdWithError -
231PyDict_GetItemString PyMapping_GetItemString
232PyDict_Items PyMapping_Items
233PyDict_Keys PyMapping_Keys
234PyDict_Values PyMapping_Values
235PyDict_Size PyMapping_Size
236 PyMapping_Length
237PyDict_Next PyIter_Next
238_PyDict_Next -
239PyDict_Merge -
240PyDict_Update -
241PyDict_MergeFromSeq2 -
242PyDict_ClearFreeList -
243- PyMapping_HasKeyString
244- PyMapping_HasKey
245========================== ============= =======================
246
247
248The dict Interface Relative to OrderedDict
249==========================================
250
251Since OrderedDict subclasses dict, understanding the various methods and
252attributes of dict is important for implementing OrderedDict.
253
254Relevant Type Slots
255-------------------
256
257================= ================ =================== ================
258slot attribute object dict
259================= ================ =================== ================
260tp_dealloc - object_dealloc dict_dealloc
261tp_repr __repr__ object_repr dict_repr
262sq_contains __contains__ - dict_contains
263mp_length __len__ - dict_length
264mp_subscript __getitem__ - dict_subscript
265mp_ass_subscript __setitem__ - dict_ass_sub
266 __delitem__
267tp_hash __hash__ _Py_HashPointer ..._HashNotImpl
268tp_str __str__ object_str -
269tp_getattro __getattribute__ ..._GenericGetAttr (repeated)
270 __getattr__
271tp_setattro __setattr__ ..._GenericSetAttr (disabled)
272tp_doc __doc__ (literal) dictionary_doc
273tp_traverse - - dict_traverse
274tp_clear - - dict_tp_clear
275tp_richcompare __eq__ object_richcompare dict_richcompare
276 __ne__
277tp_weaklistoffset (__weakref__) - -
278tp_iter __iter__ - dict_iter
279tp_dictoffset (__dict__) - -
280tp_init __init__ object_init dict_init
281tp_alloc - PyType_GenericAlloc (repeated)
282tp_new __new__ object_new dict_new
283tp_free - PyObject_Del PyObject_GC_Del
284================= ================ =================== ================
285
286Relevant Methods
287----------------
288
289================ =================== ===============
290method object dict
291================ =================== ===============
292__reduce__ object_reduce -
293__sizeof__ object_sizeof dict_sizeof
294clear - dict_clear
295copy - dict_copy
296fromkeys - dict_fromkeys
297get - dict_get
298items - dictitems_new
299keys - dictkeys_new
300pop - dict_pop
301popitem - dict_popitem
302setdefault - dict_setdefault
303update - dict_update
304values - dictvalues_new
305================ =================== ===============
306
307
308Pure Python OrderedDict
309=======================
310
311As already noted, compatibility with the pure Python OrderedDict
312implementation is a key goal of this C implementation. To further that
313goal, here's a summary of how OrderedDict-specific methods are implemented
314in collections/__init__.py. Also provided is an indication of which
315methods directly mutate or iterate the object, as well as any relationship
316with the underlying linked-list.
317
318============= ============== == ================ === === ====
319method impl used ll uses inq mut iter
320============= ============== == ================ === === ====
321__contains__ dict - - X
322__delitem__ OrderedDict Y dict.__delitem__ X
323__eq__ OrderedDict N OrderedDict ~
324 dict.__eq__
325 __iter__
326__getitem__ dict - - X
327__iter__ OrderedDict Y - X
328__init__ OrderedDict N update
329__len__ dict - - X
330__ne__ MutableMapping - __eq__ ~
331__reduce__ OrderedDict N OrderedDict ~
332 __iter__
333 __getitem__
334__repr__ OrderedDict N __class__ ~
335 items
336__reversed__ OrderedDict Y - X
337__setitem__ OrderedDict Y __contains__ X
338 dict.__setitem__
339__sizeof__ OrderedDict Y __len__ ~
340 __dict__
341clear OrderedDict Y dict.clear X
342copy OrderedDict N __class__
343 __init__
344fromkeys OrderedDict N __setitem__
345get dict - - ~
346items MutableMapping - ItemsView X
347keys MutableMapping - KeysView X
348move_to_end OrderedDict Y - X
349pop OrderedDict N __contains__ X
350 __getitem__
351 __delitem__
352popitem OrderedDict Y dict.pop X
353setdefault OrderedDict N __contains__ ~
354 __getitem__
355 __setitem__
356update MutableMapping - __setitem__ ~
357values MutableMapping - ValuesView X
358============= ============== == ================ === === ====
359
360__reversed__ and move_to_end are both exclusive to OrderedDict.
361
362
363C OrderedDict Implementation
364============================
365
366================= ================
367slot impl
368================= ================
369tp_dealloc odict_dealloc
370tp_repr odict_repr
371mp_ass_subscript odict_ass_sub
372tp_doc odict_doc
373tp_traverse odict_traverse
374tp_clear odict_tp_clear
375tp_richcompare odict_richcompare
376tp_weaklistoffset (offset)
377tp_iter odict_iter
378tp_dictoffset (offset)
379tp_init odict_init
380tp_alloc (repeated)
381tp_new odict_new
382================= ================
383
384================= ================
385method impl
386================= ================
387__reduce__ odict_reduce
388__sizeof__ odict_sizeof
389clear odict_clear
390copy odict_copy
391fromkeys odict_fromkeys
392items odictitems_new
393keys odictkeys_new
394pop odict_pop
395popitem odict_popitem
396setdefault odict_setdefault
397update odict_update
398values odictvalues_new
399================= ================
400
401Inherited unchanged from object/dict:
402
403================ ==========================
404method type field
405================ ==========================
406- tp_free
407__contains__ tp_as_sequence.sq_contains
408__getattr__ tp_getattro
409__getattribute__ tp_getattro
410__getitem__ tp_as_mapping.mp_subscript
411__hash__ tp_hash
412__len__ tp_as_mapping.mp_length
413__setattr__ tp_setattro
414__str__ tp_str
415get -
416================ ==========================
417
418
419Other Challenges
420================
421
422Preserving Ordering During Iteration
423------------------------------------
424During iteration through an OrderedDict, it is possible that items could
425get added, removed, or reordered. For a linked-list implementation, as
426with some other implementations, that situation may lead to undefined
427behavior. The documentation for dict mentions this in the `iter()` section
428of http://docs.python.org/3.4/library/stdtypes.html#dictionary-view-objects.
429In this implementation we follow dict's lead (as does the pure Python
430implementation) for __iter__(), keys(), values(), and items().
431
432For internal iteration (using _odict_FOREACH or not), there is still the
433risk that not all nodes that we expect to be seen in the loop actually get
434seen. Thus, we are careful in each of those places to ensure that they
435are. This comes, of course, at a small price at each location. The
436solutions are much the same as those detailed in the `Situation that
437Endangers Consistency` section above.
438
439
440Potential Optimizations
441=======================
442
443* Allocate the nodes as a block via od_fast_nodes instead of individually.
444 - Set node->key to NULL to indicate the node is not-in-use.
445 - Add _odict_EXISTS()?
446 - How to maintain consistency across resizes? Existing node pointers
447 would be invalidate after a resize, which is particularly problematic
448 for the iterators.
449* Use a more stream-lined implementation of update() and, likely indirectly,
450 __init__().
451
452*/
453
454/* TODO
455
456sooner:
457- reentrancy (make sure everything is at a thread-safe state when calling
458 into Python). I've already checked this multiple times, but want to
459 make one more pass.
460- add unit tests for reentrancy?
461
462later:
463- make the dict views support the full set API (the pure Python impl does)
464- implement a fuller MutableMapping API in C?
465- move the MutableMapping implementation to abstract.c?
466- optimize mutablemapping_update
467- use PyObject_MALLOC (small object allocator) for odict nodes?
468- support subclasses better (e.g. in odict_richcompare)
469
470*/
471
472#include "Python.h"
473#include "structmember.h"
474#include "dict-common.h"
475#include <stddef.h>
476
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100477#include "clinic/odictobject.c.h"
478
479/*[clinic input]
480class OrderedDict "PyODictObject *" "&PyODict_Type"
481[clinic start generated code]*/
482/*[clinic end generated code: output=da39a3ee5e6b4b0d input=ca0641cf6143d4af]*/
483
Eric Snow96c6af92015-05-29 22:21:39 -0600484
485typedef struct _odictnode _ODictNode;
486
487/* PyODictObject */
488struct _odictobject {
Eric Snow4fabf022015-06-04 00:09:56 -0600489 PyDictObject od_dict; /* the underlying dict */
490 _ODictNode *od_first; /* first node in the linked list, if any */
491 _ODictNode *od_last; /* last node in the linked list, if any */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200492 /* od_fast_nodes, od_fast_nodes_size and od_resize_sentinel are managed
493 * by _odict_resize().
Eric Snow8c7f9552015-08-07 17:45:12 -0600494 * Note that we rely on implementation details of dict for both. */
495 _ODictNode **od_fast_nodes; /* hash table that mirrors the dict table */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200496 Py_ssize_t od_fast_nodes_size;
497 void *od_resize_sentinel; /* changes if odict should be resized */
Eric Snow8c7f9552015-08-07 17:45:12 -0600498
Eric Snow4fabf022015-06-04 00:09:56 -0600499 size_t od_state; /* incremented whenever the LL changes */
500 PyObject *od_inst_dict; /* OrderedDict().__dict__ */
501 PyObject *od_weakreflist; /* holds weakrefs to the odict */
Eric Snow96c6af92015-05-29 22:21:39 -0600502};
503
504
505/* ----------------------------------------------
506 * odict keys (a simple doubly-linked list)
507 */
508
509struct _odictnode {
510 PyObject *key;
Eric Snow4c729182015-06-03 10:50:37 -0600511 Py_hash_t hash;
Eric Snow96c6af92015-05-29 22:21:39 -0600512 _ODictNode *next;
513 _ODictNode *prev;
514};
515
516#define _odictnode_KEY(node) \
517 (node->key)
Eric Snow4c729182015-06-03 10:50:37 -0600518#define _odictnode_HASH(node) \
519 (node->hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600520/* borrowed reference */
521#define _odictnode_VALUE(node, od) \
Eric Snowa762af72015-06-01 22:59:08 -0600522 PyODict_GetItemWithError((PyObject *)od, _odictnode_KEY(node))
Eric Snow96c6af92015-05-29 22:21:39 -0600523#define _odictnode_PREV(node) (node->prev)
524#define _odictnode_NEXT(node) (node->next)
525
526#define _odict_FIRST(od) (((PyODictObject *)od)->od_first)
527#define _odict_LAST(od) (((PyODictObject *)od)->od_last)
528#define _odict_EMPTY(od) (_odict_FIRST(od) == NULL)
529#define _odict_FOREACH(od, node) \
530 for (node = _odict_FIRST(od); node != NULL; node = _odictnode_NEXT(node))
531
Eric Snow8c7f9552015-08-07 17:45:12 -0600532#define _odict_FAST_SIZE(od) ((PyDictObject *)od)->ma_keys->dk_size
Eric Snow96c6af92015-05-29 22:21:39 -0600533
Eric Snow96c6af92015-05-29 22:21:39 -0600534static void
535_odict_free_fast_nodes(PyODictObject *od) {
536 if (od->od_fast_nodes) {
537 PyMem_FREE(od->od_fast_nodes);
538 }
539}
540
Eric Snow4c729182015-06-03 10:50:37 -0600541/* Return the index into the hash table, regardless of a valid node. */
542static Py_ssize_t
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200543_odict_get_index_raw(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow4c729182015-06-03 10:50:37 -0600544{
INADA Naokiba609772016-12-07 20:41:42 +0900545 PyObject *value = NULL;
Eric Snow4c729182015-06-03 10:50:37 -0600546 PyDictKeysObject *keys = ((PyDictObject *)od)->ma_keys;
Victor Stinner742da042016-09-07 17:40:12 -0700547 Py_ssize_t ix;
Eric Snow4c729182015-06-03 10:50:37 -0600548
INADA Naokiba609772016-12-07 20:41:42 +0900549 ix = (keys->dk_lookup)((PyDictObject *)od, key, hash, &value, NULL);
Victor Stinner742da042016-09-07 17:40:12 -0700550 if (ix == DKIX_EMPTY) {
551 return keys->dk_nentries; /* index of new entry */
552 }
553 if (ix < 0)
Eric Snow4c729182015-06-03 10:50:37 -0600554 return -1;
555 /* We use pointer arithmetic to get the entry's index into the table. */
Victor Stinner742da042016-09-07 17:40:12 -0700556 return ix;
Eric Snow4c729182015-06-03 10:50:37 -0600557}
558
559/* Replace od->od_fast_nodes with a new table matching the size of dict's. */
Eric Snow96c6af92015-05-29 22:21:39 -0600560static int
561_odict_resize(PyODictObject *od) {
Eric Snow4c729182015-06-03 10:50:37 -0600562 Py_ssize_t size, i;
Eric Snow96c6af92015-05-29 22:21:39 -0600563 _ODictNode **fast_nodes, *node;
564
565 /* Initialize a new "fast nodes" table. */
566 size = ((PyDictObject *)od)->ma_keys->dk_size;
567 fast_nodes = PyMem_NEW(_ODictNode *, size);
568 if (fast_nodes == NULL) {
569 PyErr_NoMemory();
570 return -1;
571 }
572 for (i = 0; i < size; i++)
573 fast_nodes[i] = NULL;
574
575 /* Copy the current nodes into the table. */
Eric Snow96c6af92015-05-29 22:21:39 -0600576 _odict_FOREACH(od, node) {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200577 i = _odict_get_index_raw(od, _odictnode_KEY(node),
Victor Stinner742da042016-09-07 17:40:12 -0700578 _odictnode_HASH(node));
Eric Snow96c6af92015-05-29 22:21:39 -0600579 if (i < 0) {
Eric Snow96c6af92015-05-29 22:21:39 -0600580 PyMem_FREE(fast_nodes);
581 return -1;
582 }
583 fast_nodes[i] = node;
584 }
Eric Snow96c6af92015-05-29 22:21:39 -0600585
586 /* Replace the old fast nodes table. */
587 _odict_free_fast_nodes(od);
588 od->od_fast_nodes = fast_nodes;
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200589 od->od_fast_nodes_size = size;
590 od->od_resize_sentinel = ((PyDictObject *)od)->ma_keys;
Eric Snow96c6af92015-05-29 22:21:39 -0600591 return 0;
592}
593
594/* Return the index into the hash table, regardless of a valid node. */
595static Py_ssize_t
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200596_odict_get_index(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600597{
Eric Snow4c729182015-06-03 10:50:37 -0600598 PyDictKeysObject *keys;
Eric Snow96c6af92015-05-29 22:21:39 -0600599
600 assert(key != NULL);
Eric Snow4c729182015-06-03 10:50:37 -0600601 keys = ((PyDictObject *)od)->ma_keys;
602
603 /* Ensure od_fast_nodes and dk_entries are in sync. */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200604 if (od->od_resize_sentinel != keys ||
605 od->od_fast_nodes_size != keys->dk_size) {
Eric Snow4c729182015-06-03 10:50:37 -0600606 int resize_res = _odict_resize(od);
607 if (resize_res < 0)
608 return -1;
609 }
610
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200611 return _odict_get_index_raw(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600612}
613
Eric Snow96c6af92015-05-29 22:21:39 -0600614/* Returns NULL if there was some error or the key was not found. */
615static _ODictNode *
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200616_odict_find_node_hash(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600617{
618 Py_ssize_t index;
619
620 if (_odict_EMPTY(od))
621 return NULL;
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200622 index = _odict_get_index(od, key, hash);
623 if (index < 0)
624 return NULL;
625 return od->od_fast_nodes[index];
626}
627
628static _ODictNode *
629_odict_find_node(PyODictObject *od, PyObject *key)
630{
631 Py_ssize_t index;
632 Py_hash_t hash;
633
634 if (_odict_EMPTY(od))
635 return NULL;
636 hash = PyObject_Hash(key);
637 if (hash == -1)
638 return NULL;
639 index = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600640 if (index < 0)
641 return NULL;
642 return od->od_fast_nodes[index];
643}
644
645static void
646_odict_add_head(PyODictObject *od, _ODictNode *node)
647{
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300648 _odictnode_PREV(node) = NULL;
649 _odictnode_NEXT(node) = _odict_FIRST(od);
650 if (_odict_FIRST(od) == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -0600651 _odict_LAST(od) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300652 else
Eric Snow96c6af92015-05-29 22:21:39 -0600653 _odictnode_PREV(_odict_FIRST(od)) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300654 _odict_FIRST(od) = node;
Eric Snow4fabf022015-06-04 00:09:56 -0600655 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600656}
657
658static void
659_odict_add_tail(PyODictObject *od, _ODictNode *node)
660{
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300661 _odictnode_PREV(node) = _odict_LAST(od);
662 _odictnode_NEXT(node) = NULL;
663 if (_odict_LAST(od) == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -0600664 _odict_FIRST(od) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300665 else
Eric Snow96c6af92015-05-29 22:21:39 -0600666 _odictnode_NEXT(_odict_LAST(od)) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300667 _odict_LAST(od) = node;
Eric Snow4fabf022015-06-04 00:09:56 -0600668 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600669}
670
671/* adds the node to the end of the list */
672static int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200673_odict_add_new_node(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600674{
675 Py_ssize_t i;
676 _ODictNode *node;
677
Serhiy Storchakad17427b2015-10-20 18:21:48 +0300678 Py_INCREF(key);
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200679 i = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600680 if (i < 0) {
681 if (!PyErr_Occurred())
682 PyErr_SetObject(PyExc_KeyError, key);
683 Py_DECREF(key);
684 return -1;
685 }
686 else if (od->od_fast_nodes[i] != NULL) {
687 /* We already have a node for the key so there's no need to add one. */
688 Py_DECREF(key);
689 return 0;
690 }
691
692 /* must not be added yet */
693 node = (_ODictNode *)PyMem_MALLOC(sizeof(_ODictNode));
694 if (node == NULL) {
695 Py_DECREF(key);
696 PyErr_NoMemory();
697 return -1;
698 }
699
700 _odictnode_KEY(node) = key;
Eric Snow4c729182015-06-03 10:50:37 -0600701 _odictnode_HASH(node) = hash;
Eric Snow96c6af92015-05-29 22:21:39 -0600702 _odict_add_tail(od, node);
703 od->od_fast_nodes[i] = node;
704 return 0;
705}
706
707/* Putting the decref after the free causes problems. */
708#define _odictnode_DEALLOC(node) \
709 do { \
710 Py_DECREF(_odictnode_KEY(node)); \
711 PyMem_FREE((void *)node); \
712 } while (0)
713
714/* Repeated calls on the same node are no-ops. */
715static void
716_odict_remove_node(PyODictObject *od, _ODictNode *node)
717{
718 if (_odict_FIRST(od) == node)
719 _odict_FIRST(od) = _odictnode_NEXT(node);
720 else if (_odictnode_PREV(node) != NULL)
721 _odictnode_NEXT(_odictnode_PREV(node)) = _odictnode_NEXT(node);
722
723 if (_odict_LAST(od) == node)
724 _odict_LAST(od) = _odictnode_PREV(node);
725 else if (_odictnode_NEXT(node) != NULL)
726 _odictnode_PREV(_odictnode_NEXT(node)) = _odictnode_PREV(node);
727
728 _odictnode_PREV(node) = NULL;
729 _odictnode_NEXT(node) = NULL;
Eric Snow4fabf022015-06-04 00:09:56 -0600730 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600731}
732
Eric Snow96c6af92015-05-29 22:21:39 -0600733/* If someone calls PyDict_DelItem() directly on an OrderedDict, we'll
734 get all sorts of problems here. In PyODict_DelItem we make sure to
735 call _odict_clear_node first.
Victor Stinnerca30b022015-09-03 17:50:04 +0200736
Eric Snow96c6af92015-05-29 22:21:39 -0600737 This matters in the case of colliding keys. Suppose we add 3 keys:
738 [A, B, C], where the hash of C collides with A and the next possible
739 index in the hash table is occupied by B. If we remove B then for C
740 the dict's looknode func will give us the old index of B instead of
741 the index we got before deleting B. However, the node for C in
742 od_fast_nodes is still at the old dict index of C. Thus to be sure
743 things don't get out of sync, we clear the node in od_fast_nodes
744 *before* calling PyDict_DelItem.
745
746 The same must be done for any other OrderedDict operations where
747 we modify od_fast_nodes.
748*/
749static int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200750_odict_clear_node(PyODictObject *od, _ODictNode *node, PyObject *key,
751 Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600752{
753 Py_ssize_t i;
754
755 assert(key != NULL);
756 if (_odict_EMPTY(od)) {
757 /* Let later code decide if this is a KeyError. */
758 return 0;
759 }
760
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200761 i = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600762 if (i < 0)
763 return PyErr_Occurred() ? -1 : 0;
764
765 if (node == NULL)
766 node = od->od_fast_nodes[i];
767 assert(node == od->od_fast_nodes[i]);
768 if (node == NULL) {
769 /* Let later code decide if this is a KeyError. */
770 return 0;
771 }
772
773 // Now clear the node.
774 od->od_fast_nodes[i] = NULL;
775 _odict_remove_node(od, node);
776 _odictnode_DEALLOC(node);
777 return 0;
778}
779
780static void
781_odict_clear_nodes(PyODictObject *od)
782{
783 _ODictNode *node, *next;
784
Eric Snow96c6af92015-05-29 22:21:39 -0600785 _odict_free_fast_nodes(od);
786 od->od_fast_nodes = NULL;
Serhiy Storchakad205d012016-01-19 14:46:25 +0200787
788 node = _odict_FIRST(od);
789 _odict_FIRST(od) = NULL;
790 _odict_LAST(od) = NULL;
791 while (node != NULL) {
792 next = _odictnode_NEXT(node);
793 _odictnode_DEALLOC(node);
794 node = next;
795 }
Eric Snow96c6af92015-05-29 22:21:39 -0600796}
797
798/* There isn't any memory management of nodes past this point. */
799#undef _odictnode_DEALLOC
800
801static int
802_odict_keys_equal(PyODictObject *a, PyODictObject *b)
803{
804 _ODictNode *node_a, *node_b;
805
806 node_a = _odict_FIRST(a);
807 node_b = _odict_FIRST(b);
808 while (1) {
809 if (node_a == NULL && node_b == NULL)
810 /* success: hit the end of each at the same time */
811 return 1;
812 else if (node_a == NULL || node_b == NULL)
813 /* unequal length */
814 return 0;
815 else {
816 int res = PyObject_RichCompareBool(
817 (PyObject *)_odictnode_KEY(node_a),
818 (PyObject *)_odictnode_KEY(node_b),
819 Py_EQ);
820 if (res < 0)
821 return res;
822 else if (res == 0)
823 return 0;
824
825 /* otherwise it must match, so move on to the next one */
826 node_a = _odictnode_NEXT(node_a);
827 node_b = _odictnode_NEXT(node_b);
828 }
829 }
830}
831
832
833/* ----------------------------------------------
834 * OrderedDict mapping methods
835 */
836
837/* mp_ass_subscript: __setitem__() and __delitem__() */
838
839static int
840odict_mp_ass_sub(PyODictObject *od, PyObject *v, PyObject *w)
841{
842 if (w == NULL)
843 return PyODict_DelItem((PyObject *)od, v);
844 else
845 return PyODict_SetItem((PyObject *)od, v, w);
846}
847
848/* tp_as_mapping */
849
850static PyMappingMethods odict_as_mapping = {
851 0, /*mp_length*/
852 0, /*mp_subscript*/
853 (objobjargproc)odict_mp_ass_sub, /*mp_ass_subscript*/
854};
855
856
857/* ----------------------------------------------
858 * OrderedDict methods
859 */
860
861/* __delitem__() */
862
863PyDoc_STRVAR(odict_delitem__doc__, "od.__delitem__(y) <==> del od[y]");
864
865/* __eq__() */
866
867PyDoc_STRVAR(odict_eq__doc__,
868"od.__eq__(y) <==> od==y. Comparison to another OD is order-sensitive \n\
869 while comparison to a regular mapping is order-insensitive.\n\
870 ");
871
872/* forward */
873static PyObject * odict_richcompare(PyObject *v, PyObject *w, int op);
874
875static PyObject *
876odict_eq(PyObject *a, PyObject *b)
877{
878 return odict_richcompare(a, b, Py_EQ);
879}
880
881/* __init__() */
882
883PyDoc_STRVAR(odict_init__doc__,
884"Initialize an ordered dictionary. The signature is the same as\n\
885 regular dictionaries, but keyword arguments are not recommended because\n\
886 their insertion order is arbitrary.\n\
887\n\
888 ");
889
890/* forward */
891static int odict_init(PyObject *self, PyObject *args, PyObject *kwds);
892
893/* __iter__() */
894
895PyDoc_STRVAR(odict_iter__doc__, "od.__iter__() <==> iter(od)");
896
897static PyObject * odict_iter(PyODictObject *self); /* forward */
898
899/* __ne__() */
900
901/* Mapping.__ne__() does not have a docstring. */
902PyDoc_STRVAR(odict_ne__doc__, "");
903
904static PyObject *
905odict_ne(PyObject *a, PyObject *b)
906{
907 return odict_richcompare(a, b, Py_NE);
908}
909
910/* __repr__() */
911
912PyDoc_STRVAR(odict_repr__doc__, "od.__repr__() <==> repr(od)");
913
914static PyObject * odict_repr(PyODictObject *self); /* forward */
915
916/* __setitem__() */
917
918PyDoc_STRVAR(odict_setitem__doc__, "od.__setitem__(i, y) <==> od[i]=y");
919
920/* fromkeys() */
921
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100922/*[clinic input]
923@classmethod
924OrderedDict.fromkeys
925
926 iterable as seq: object
927 value: object = None
928
929New ordered dictionary with keys from S.
930
931If not specified, the value defaults to None.
932[clinic start generated code]*/
Eric Snow96c6af92015-05-29 22:21:39 -0600933
934static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100935OrderedDict_fromkeys_impl(PyTypeObject *type, PyObject *seq, PyObject *value)
936/*[clinic end generated code: output=c10390d452d78d6d input=33eefc496d5eee7b]*/
Eric Snow96c6af92015-05-29 22:21:39 -0600937{
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100938 return _PyDict_FromKeys((PyObject *)type, seq, value);
Eric Snow96c6af92015-05-29 22:21:39 -0600939}
940
941/* __sizeof__() */
942
943/* OrderedDict.__sizeof__() does not have a docstring. */
944PyDoc_STRVAR(odict_sizeof__doc__, "");
945
946static PyObject *
947odict_sizeof(PyODictObject *od)
948{
Serhiy Storchaka0ce7a3a2015-12-22 08:16:18 +0200949 Py_ssize_t res = _PyDict_SizeOf((PyDictObject *)od);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300950 res += sizeof(_ODictNode *) * _odict_FAST_SIZE(od); /* od_fast_nodes */
Eric Snow96c6af92015-05-29 22:21:39 -0600951 if (!_odict_EMPTY(od)) {
952 res += sizeof(_ODictNode) * PyODict_SIZE(od); /* linked-list */
953 }
954 return PyLong_FromSsize_t(res);
955}
956
957/* __reduce__() */
958
959PyDoc_STRVAR(odict_reduce__doc__, "Return state information for pickling");
960
961static PyObject *
962odict_reduce(register PyODictObject *od)
963{
964 _Py_IDENTIFIER(__dict__);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300965 _Py_IDENTIFIER(items);
Serhiy Storchaka4575beb2015-10-22 20:18:24 +0300966 PyObject *dict = NULL, *result = NULL;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300967 PyObject *items_iter, *items, *args = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -0600968
969 /* capture any instance state */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300970 dict = _PyObject_GetAttrId((PyObject *)od, &PyId___dict__);
971 if (dict == NULL)
Eric Snowc5e59602015-05-30 11:28:56 -0600972 goto Done;
973 else {
Eric Snow96c6af92015-05-29 22:21:39 -0600974 /* od.__dict__ isn't necessarily a dict... */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300975 Py_ssize_t dict_len = PyObject_Length(dict);
976 if (dict_len == -1)
Eric Snow96c6af92015-05-29 22:21:39 -0600977 goto Done;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300978 if (!dict_len) {
979 /* nothing to pickle in od.__dict__ */
980 Py_CLEAR(dict);
Eric Snow96c6af92015-05-29 22:21:39 -0600981 }
982 }
983
984 /* build the result */
Eric Snow96c6af92015-05-29 22:21:39 -0600985 args = PyTuple_New(0);
986 if (args == NULL)
987 goto Done;
988
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300989 items = _PyObject_CallMethodIdObjArgs((PyObject *)od, &PyId_items, NULL);
Eric Snow96c6af92015-05-29 22:21:39 -0600990 if (items == NULL)
991 goto Done;
992
993 items_iter = PyObject_GetIter(items);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300994 Py_DECREF(items);
Eric Snow96c6af92015-05-29 22:21:39 -0600995 if (items_iter == NULL)
996 goto Done;
997
Serhiy Storchaka4575beb2015-10-22 20:18:24 +0300998 result = PyTuple_Pack(5, Py_TYPE(od), args, dict ? dict : Py_None, Py_None, items_iter);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300999 Py_DECREF(items_iter);
Eric Snow96c6af92015-05-29 22:21:39 -06001000
1001Done:
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001002 Py_XDECREF(dict);
Eric Snow96c6af92015-05-29 22:21:39 -06001003 Py_XDECREF(args);
Eric Snow96c6af92015-05-29 22:21:39 -06001004
1005 return result;
1006}
1007
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001008/* setdefault(): Skips __missing__() calls. */
Eric Snow96c6af92015-05-29 22:21:39 -06001009
Eric Snow96c6af92015-05-29 22:21:39 -06001010
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001011/*[clinic input]
1012OrderedDict.setdefault
1013
1014 key: object
1015 default as failobj: object = None
1016
1017od.get(k,d), also set od[k]=d if k not in od.
1018[clinic start generated code]*/
1019
Eric Snow96c6af92015-05-29 22:21:39 -06001020static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001021OrderedDict_setdefault_impl(PyODictObject *self, PyObject *key,
1022 PyObject *failobj)
1023/*[clinic end generated code: output=605d0f6f61ccb0a6 input=4ee5006f32f5691b]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001024{
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001025 PyObject *result = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001026
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001027 if (PyODict_CheckExact(self)) {
1028 result = PyODict_GetItemWithError(self, key); /* borrowed */
Eric Snowd1719752015-06-01 23:12:13 -06001029 if (result == NULL) {
1030 if (PyErr_Occurred())
1031 return NULL;
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001032 assert(_odict_find_node(self, key) == NULL);
1033 if (PyODict_SetItem((PyObject *)self, key, failobj) >= 0) {
Eric Snow96c6af92015-05-29 22:21:39 -06001034 result = failobj;
1035 Py_INCREF(failobj);
1036 }
1037 }
1038 else {
Eric Snowd1719752015-06-01 23:12:13 -06001039 Py_INCREF(result);
Eric Snow96c6af92015-05-29 22:21:39 -06001040 }
1041 }
1042 else {
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001043 int exists = PySequence_Contains((PyObject *)self, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001044 if (exists < 0) {
Eric Snowd1719752015-06-01 23:12:13 -06001045 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001046 }
1047 else if (exists) {
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001048 result = PyObject_GetItem((PyObject *)self, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001049 }
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001050 else if (PyObject_SetItem((PyObject *)self, key, failobj) >= 0) {
Eric Snow96c6af92015-05-29 22:21:39 -06001051 result = failobj;
1052 Py_INCREF(failobj);
1053 }
1054 }
1055
Eric Snow96c6af92015-05-29 22:21:39 -06001056 return result;
1057}
1058
1059/* pop() */
1060
1061PyDoc_STRVAR(odict_pop__doc__,
1062"od.pop(k[,d]) -> v, remove specified key and return the corresponding\n\
1063 value. If key is not found, d is returned if given, otherwise KeyError\n\
1064 is raised.\n\
1065\n\
1066 ");
1067
1068/* forward */
1069static PyObject * _odict_popkey(PyObject *, PyObject *, PyObject *);
1070
1071/* Skips __missing__() calls. */
1072static PyObject *
Eric Snowac02ef32015-06-02 20:42:14 -06001073odict_pop(PyObject *od, PyObject *args, PyObject *kwargs)
Eric Snow96c6af92015-05-29 22:21:39 -06001074{
Eric Snowac02ef32015-06-02 20:42:14 -06001075 static char *kwlist[] = {"key", "default", 0};
Eric Snow96c6af92015-05-29 22:21:39 -06001076 PyObject *key, *failobj = NULL;
1077
Eric Snowac02ef32015-06-02 20:42:14 -06001078 /* borrowed */
1079 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:pop", kwlist,
1080 &key, &failobj)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001081 return NULL;
1082 }
1083
1084 return _odict_popkey(od, key, failobj);
1085}
1086
1087static PyObject *
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001088_odict_popkey_hash(PyObject *od, PyObject *key, PyObject *failobj,
1089 Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -06001090{
1091 _ODictNode *node;
1092 PyObject *value = NULL;
1093
Eric Snow96c6af92015-05-29 22:21:39 -06001094 /* Pop the node first to avoid a possible dict resize (due to
1095 eval loop reentrancy) and complications due to hash collision
1096 resolution. */
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001097 node = _odict_find_node_hash((PyODictObject *)od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001098 if (node == NULL) {
1099 if (PyErr_Occurred())
Eric Snowd1719752015-06-01 23:12:13 -06001100 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001101 }
1102 else {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001103 int res = _odict_clear_node((PyODictObject *)od, node, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001104 if (res < 0) {
Eric Snowd1719752015-06-01 23:12:13 -06001105 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001106 }
1107 }
1108
1109 /* Now delete the value from the dict. */
Serhiy Storchaka04386832016-10-30 17:17:24 +02001110 if (PyODict_CheckExact(od)) {
1111 if (node != NULL) {
1112 value = _PyDict_GetItem_KnownHash(od, key, hash); /* borrowed */
1113 if (value != NULL) {
1114 Py_INCREF(value);
1115 if (_PyDict_DelItem_KnownHash(od, key, hash) < 0) {
1116 Py_DECREF(value);
1117 return NULL;
1118 }
1119 }
1120 }
1121 }
1122 else {
1123 int exists = PySequence_Contains(od, key);
1124 if (exists < 0)
1125 return NULL;
1126 if (exists) {
1127 value = PyObject_GetItem(od, key);
1128 if (value != NULL) {
1129 if (PyObject_DelItem(od, key) == -1) {
1130 Py_CLEAR(value);
1131 }
Eric Snow96c6af92015-05-29 22:21:39 -06001132 }
1133 }
1134 }
1135
1136 /* Apply the fallback value, if necessary. */
1137 if (value == NULL && !PyErr_Occurred()) {
1138 if (failobj) {
1139 value = failobj;
1140 Py_INCREF(failobj);
1141 }
1142 else {
1143 PyErr_SetObject(PyExc_KeyError, key);
1144 }
1145 }
1146
Eric Snow96c6af92015-05-29 22:21:39 -06001147 return value;
1148}
1149
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001150static PyObject *
1151_odict_popkey(PyObject *od, PyObject *key, PyObject *failobj)
1152{
1153 Py_hash_t hash = PyObject_Hash(key);
1154 if (hash == -1)
1155 return NULL;
1156
1157 return _odict_popkey_hash(od, key, failobj, hash);
1158}
1159
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001160
Eric Snow96c6af92015-05-29 22:21:39 -06001161/* popitem() */
1162
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001163/*[clinic input]
1164OrderedDict.popitem
1165
1166 last: bool = True
1167
1168Return (k, v) and remove a (key, value) pair.
1169
1170Pairs are returned in LIFO order if last is true or FIFO order if false.
1171[clinic start generated code]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001172
1173static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001174OrderedDict_popitem_impl(PyODictObject *self, int last)
1175/*[clinic end generated code: output=98e7d986690d49eb input=4937da2015939126]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001176{
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001177 PyObject *key, *value, *item = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001178 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001179
1180 /* pull the item */
1181
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001182 if (_odict_EMPTY(self)) {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001183 PyErr_SetString(PyExc_KeyError, "dictionary is empty");
1184 return NULL;
Eric Snowac02ef32015-06-02 20:42:14 -06001185 }
Eric Snow96c6af92015-05-29 22:21:39 -06001186
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001187 node = last ? _odict_LAST(self) : _odict_FIRST(self);
Eric Snow96c6af92015-05-29 22:21:39 -06001188 key = _odictnode_KEY(node);
Eric Snowd1719752015-06-01 23:12:13 -06001189 Py_INCREF(key);
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001190 value = _odict_popkey_hash((PyObject *)self, key, NULL, _odictnode_HASH(node));
Eric Snow96c6af92015-05-29 22:21:39 -06001191 if (value == NULL)
1192 return NULL;
1193 item = PyTuple_Pack(2, key, value);
Eric Snowd1719752015-06-01 23:12:13 -06001194 Py_DECREF(key);
Eric Snow96c6af92015-05-29 22:21:39 -06001195 Py_DECREF(value);
1196 return item;
1197}
1198
1199/* keys() */
1200
1201/* MutableMapping.keys() does not have a docstring. */
1202PyDoc_STRVAR(odict_keys__doc__, "");
1203
1204static PyObject * odictkeys_new(PyObject *od); /* forward */
1205
1206/* values() */
1207
1208/* MutableMapping.values() does not have a docstring. */
1209PyDoc_STRVAR(odict_values__doc__, "");
1210
1211static PyObject * odictvalues_new(PyObject *od); /* forward */
1212
1213/* items() */
1214
1215/* MutableMapping.items() does not have a docstring. */
1216PyDoc_STRVAR(odict_items__doc__, "");
1217
1218static PyObject * odictitems_new(PyObject *od); /* forward */
1219
1220/* update() */
1221
1222/* MutableMapping.update() does not have a docstring. */
1223PyDoc_STRVAR(odict_update__doc__, "");
1224
1225/* forward */
1226static PyObject * mutablemapping_update(PyObject *, PyObject *, PyObject *);
1227
1228#define odict_update mutablemapping_update
1229
1230/* clear() */
1231
1232PyDoc_STRVAR(odict_clear__doc__,
1233 "od.clear() -> None. Remove all items from od.");
1234
1235static PyObject *
1236odict_clear(register PyODictObject *od)
1237{
1238 PyDict_Clear((PyObject *)od);
1239 _odict_clear_nodes(od);
Eric Snow96c6af92015-05-29 22:21:39 -06001240 if (_odict_resize(od) < 0)
1241 return NULL;
1242 Py_RETURN_NONE;
1243}
1244
1245/* copy() */
1246
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001247/* forward */
1248static int _PyODict_SetItem_KnownHash(PyObject *, PyObject *, PyObject *,
1249 Py_hash_t);
1250
Eric Snow96c6af92015-05-29 22:21:39 -06001251PyDoc_STRVAR(odict_copy__doc__, "od.copy() -> a shallow copy of od");
1252
1253static PyObject *
1254odict_copy(register PyODictObject *od)
1255{
1256 _ODictNode *node;
1257 PyObject *od_copy;
1258
1259 if (PyODict_CheckExact(od))
1260 od_copy = PyODict_New();
1261 else
Victor Stinnerf17c3de2016-12-06 18:46:19 +01001262 od_copy = _PyObject_CallNoArg((PyObject *)Py_TYPE(od));
Eric Snow96c6af92015-05-29 22:21:39 -06001263 if (od_copy == NULL)
1264 return NULL;
1265
1266 if (PyODict_CheckExact(od)) {
1267 _odict_FOREACH(od, node) {
Eric Snowa762af72015-06-01 22:59:08 -06001268 PyObject *key = _odictnode_KEY(node);
1269 PyObject *value = _odictnode_VALUE(node, od);
1270 if (value == NULL) {
1271 if (!PyErr_Occurred())
1272 PyErr_SetObject(PyExc_KeyError, key);
1273 goto fail;
1274 }
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001275 if (_PyODict_SetItem_KnownHash((PyObject *)od_copy, key, value,
1276 _odictnode_HASH(node)) != 0)
Eric Snow96c6af92015-05-29 22:21:39 -06001277 goto fail;
1278 }
1279 }
1280 else {
1281 _odict_FOREACH(od, node) {
1282 int res;
1283 PyObject *value = PyObject_GetItem((PyObject *)od,
1284 _odictnode_KEY(node));
1285 if (value == NULL)
1286 goto fail;
1287 res = PyObject_SetItem((PyObject *)od_copy,
1288 _odictnode_KEY(node), value);
1289 Py_DECREF(value);
1290 if (res != 0)
1291 goto fail;
1292 }
1293 }
1294 return od_copy;
1295
1296fail:
1297 Py_DECREF(od_copy);
1298 return NULL;
1299}
1300
1301/* __reversed__() */
1302
1303PyDoc_STRVAR(odict_reversed__doc__, "od.__reversed__() <==> reversed(od)");
1304
1305#define _odict_ITER_REVERSED 1
1306#define _odict_ITER_KEYS 2
1307#define _odict_ITER_VALUES 4
1308
1309/* forward */
1310static PyObject * odictiter_new(PyODictObject *, int);
1311
1312static PyObject *
1313odict_reversed(PyODictObject *od)
1314{
Eric Snow96c6af92015-05-29 22:21:39 -06001315 return odictiter_new(od, _odict_ITER_KEYS|_odict_ITER_REVERSED);
1316}
1317
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001318
Eric Snow96c6af92015-05-29 22:21:39 -06001319/* move_to_end() */
1320
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001321/*[clinic input]
1322OrderedDict.move_to_end
1323
1324 key: object
1325 last: bool = True
1326
1327"Move an existing element to the end (or beginning if last==False).
1328
1329 Raises KeyError if the element does not exist.
1330 When last=True, acts like a fast version of self[key]=self.pop(key).
1331[clinic start generated code]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001332
1333static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001334OrderedDict_move_to_end_impl(PyODictObject *self, PyObject *key, int last)
1335/*[clinic end generated code: output=fafa4c5cc9b92f20 input=3b8283f7d0e15e43]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001336{
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001337 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001338
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001339 if (_odict_EMPTY(self)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001340 PyErr_SetObject(PyExc_KeyError, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001341 return NULL;
1342 }
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001343 node = last ? _odict_LAST(self) : _odict_FIRST(self);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001344 if (key != _odictnode_KEY(node)) {
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001345 node = _odict_find_node(self, key);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001346 if (node == NULL) {
1347 if (!PyErr_Occurred())
1348 PyErr_SetObject(PyExc_KeyError, key);
Eric Snowc5e59602015-05-30 11:28:56 -06001349 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001350 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001351 if (last) {
1352 /* Only move if not already the last one. */
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001353 if (node != _odict_LAST(self)) {
1354 _odict_remove_node(self, node);
1355 _odict_add_tail(self, node);
Eric Snow96c6af92015-05-29 22:21:39 -06001356 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001357 }
1358 else {
1359 /* Only move if not already the first one. */
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001360 if (node != _odict_FIRST(self)) {
1361 _odict_remove_node(self, node);
1362 _odict_add_head(self, node);
Eric Snow96c6af92015-05-29 22:21:39 -06001363 }
1364 }
1365 }
Eric Snow96c6af92015-05-29 22:21:39 -06001366 Py_RETURN_NONE;
1367}
1368
1369
1370/* tp_methods */
1371
1372static PyMethodDef odict_methods[] = {
1373
1374 /* explicitly defined so we can align docstrings with
1375 * collections.OrderedDict */
1376 {"__delitem__", (PyCFunction)odict_mp_ass_sub, METH_NOARGS,
1377 odict_delitem__doc__},
1378 {"__eq__", (PyCFunction)odict_eq, METH_NOARGS,
1379 odict_eq__doc__},
1380 {"__init__", (PyCFunction)odict_init, METH_NOARGS,
1381 odict_init__doc__},
1382 {"__iter__", (PyCFunction)odict_iter, METH_NOARGS,
1383 odict_iter__doc__},
1384 {"__ne__", (PyCFunction)odict_ne, METH_NOARGS,
1385 odict_ne__doc__},
1386 {"__repr__", (PyCFunction)odict_repr, METH_NOARGS,
1387 odict_repr__doc__},
1388 {"__setitem__", (PyCFunction)odict_mp_ass_sub, METH_NOARGS,
1389 odict_setitem__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001390 ORDEREDDICT_FROMKEYS_METHODDEF
Eric Snow96c6af92015-05-29 22:21:39 -06001391
1392 /* overridden dict methods */
1393 {"__sizeof__", (PyCFunction)odict_sizeof, METH_NOARGS,
1394 odict_sizeof__doc__},
1395 {"__reduce__", (PyCFunction)odict_reduce, METH_NOARGS,
1396 odict_reduce__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001397 ORDEREDDICT_SETDEFAULT_METHODDEF
Eric Snowac02ef32015-06-02 20:42:14 -06001398 {"pop", (PyCFunction)odict_pop,
1399 METH_VARARGS | METH_KEYWORDS, odict_pop__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001400 ORDEREDDICT_POPITEM_METHODDEF
Eric Snow96c6af92015-05-29 22:21:39 -06001401 {"keys", (PyCFunction)odictkeys_new, METH_NOARGS,
1402 odict_keys__doc__},
1403 {"values", (PyCFunction)odictvalues_new, METH_NOARGS,
1404 odict_values__doc__},
1405 {"items", (PyCFunction)odictitems_new, METH_NOARGS,
1406 odict_items__doc__},
1407 {"update", (PyCFunction)odict_update, METH_VARARGS | METH_KEYWORDS,
1408 odict_update__doc__},
1409 {"clear", (PyCFunction)odict_clear, METH_NOARGS,
1410 odict_clear__doc__},
1411 {"copy", (PyCFunction)odict_copy, METH_NOARGS,
1412 odict_copy__doc__},
1413
1414 /* new methods */
1415 {"__reversed__", (PyCFunction)odict_reversed, METH_NOARGS,
1416 odict_reversed__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001417 ORDEREDDICT_MOVE_TO_END_METHODDEF
Eric Snow96c6af92015-05-29 22:21:39 -06001418
1419 {NULL, NULL} /* sentinel */
1420};
1421
1422
1423/* ----------------------------------------------
1424 * OrderedDict members
1425 */
1426
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001427/* tp_getset */
Eric Snow96c6af92015-05-29 22:21:39 -06001428
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001429static PyGetSetDef odict_getset[] = {
1430 {"__dict__", PyObject_GenericGetDict, PyObject_GenericSetDict},
1431 {NULL}
Eric Snow96c6af92015-05-29 22:21:39 -06001432};
1433
Eric Snow96c6af92015-05-29 22:21:39 -06001434/* ----------------------------------------------
1435 * OrderedDict type slot methods
1436 */
1437
1438/* tp_dealloc */
1439
1440static void
1441odict_dealloc(PyODictObject *self)
1442{
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001443 PyThreadState *tstate = PyThreadState_GET();
1444
Eric Snow96c6af92015-05-29 22:21:39 -06001445 PyObject_GC_UnTrack(self);
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001446 Py_TRASHCAN_SAFE_BEGIN(self)
1447
Eric Snow96c6af92015-05-29 22:21:39 -06001448 Py_XDECREF(self->od_inst_dict);
1449 if (self->od_weakreflist != NULL)
1450 PyObject_ClearWeakRefs((PyObject *)self);
1451
1452 _odict_clear_nodes(self);
Eric Snow96c6af92015-05-29 22:21:39 -06001453
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001454 /* Call the base tp_dealloc(). Since it too uses the trashcan mechanism,
1455 * temporarily decrement trash_delete_nesting to prevent triggering it
1456 * and putting the partially deallocated object on the trashcan's
1457 * to-be-deleted-later list.
1458 */
1459 --tstate->trash_delete_nesting;
1460 assert(_tstate->trash_delete_nesting < PyTrash_UNWIND_LEVEL);
Eric Snow96c6af92015-05-29 22:21:39 -06001461 PyDict_Type.tp_dealloc((PyObject *)self);
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001462 ++tstate->trash_delete_nesting;
1463
1464 Py_TRASHCAN_SAFE_END(self)
Victor Stinnere1871952016-06-08 10:18:18 +02001465}
Eric Snow96c6af92015-05-29 22:21:39 -06001466
1467/* tp_repr */
1468
1469static PyObject *
1470odict_repr(PyODictObject *self)
1471{
1472 int i;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001473 _Py_IDENTIFIER(items);
Serhiy Storchaka4575beb2015-10-22 20:18:24 +03001474 PyObject *pieces = NULL, *result = NULL;
1475 const char *classname;
1476
1477 classname = strrchr(Py_TYPE(self)->tp_name, '.');
1478 if (classname == NULL)
1479 classname = Py_TYPE(self)->tp_name;
1480 else
1481 classname++;
1482
1483 if (PyODict_SIZE(self) == 0)
1484 return PyUnicode_FromFormat("%s()", classname);
Eric Snow96c6af92015-05-29 22:21:39 -06001485
1486 i = Py_ReprEnter((PyObject *)self);
1487 if (i != 0) {
1488 return i > 0 ? PyUnicode_FromString("...") : NULL;
1489 }
1490
Eric Snow96c6af92015-05-29 22:21:39 -06001491 if (PyODict_CheckExact(self)) {
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001492 Py_ssize_t count = 0;
Eric Snowa762af72015-06-01 22:59:08 -06001493 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001494 pieces = PyList_New(PyODict_SIZE(self));
1495 if (pieces == NULL)
1496 goto Done;
1497
1498 _odict_FOREACH(self, node) {
Eric Snowa762af72015-06-01 22:59:08 -06001499 PyObject *pair;
1500 PyObject *key = _odictnode_KEY(node);
1501 PyObject *value = _odictnode_VALUE(node, self);
1502 if (value == NULL) {
1503 if (!PyErr_Occurred())
1504 PyErr_SetObject(PyExc_KeyError, key);
Eric Snowdb4061c2015-06-03 11:09:48 -06001505 goto Done;
Eric Snowa762af72015-06-01 22:59:08 -06001506 }
1507 pair = PyTuple_Pack(2, key, value);
Eric Snow96c6af92015-05-29 22:21:39 -06001508 if (pair == NULL)
1509 goto Done;
1510
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001511 if (count < PyList_GET_SIZE(pieces))
1512 PyList_SET_ITEM(pieces, count, pair); /* steals reference */
1513 else {
1514 if (PyList_Append(pieces, pair) < 0) {
1515 Py_DECREF(pair);
1516 goto Done;
1517 }
1518 Py_DECREF(pair);
1519 }
1520 count++;
Eric Snow96c6af92015-05-29 22:21:39 -06001521 }
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001522 if (count < PyList_GET_SIZE(pieces))
1523 PyList_GET_SIZE(pieces) = count;
Eric Snow96c6af92015-05-29 22:21:39 -06001524 }
1525 else {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001526 PyObject *items = _PyObject_CallMethodIdObjArgs((PyObject *)self,
1527 &PyId_items, NULL);
Eric Snow96c6af92015-05-29 22:21:39 -06001528 if (items == NULL)
1529 goto Done;
1530 pieces = PySequence_List(items);
1531 Py_DECREF(items);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001532 if (pieces == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -06001533 goto Done;
1534 }
1535
Serhiy Storchaka4575beb2015-10-22 20:18:24 +03001536 result = PyUnicode_FromFormat("%s(%R)", classname, pieces);
Eric Snow96c6af92015-05-29 22:21:39 -06001537
Eric Snow96c6af92015-05-29 22:21:39 -06001538Done:
1539 Py_XDECREF(pieces);
Eric Snow96c6af92015-05-29 22:21:39 -06001540 Py_ReprLeave((PyObject *)self);
1541 return result;
Victor Stinnere1871952016-06-08 10:18:18 +02001542}
Eric Snow96c6af92015-05-29 22:21:39 -06001543
1544/* tp_doc */
1545
1546PyDoc_STRVAR(odict_doc,
1547 "Dictionary that remembers insertion order");
1548
1549/* tp_traverse */
1550
1551static int
1552odict_traverse(PyODictObject *od, visitproc visit, void *arg)
1553{
Serhiy Storchakad205d012016-01-19 14:46:25 +02001554 _ODictNode *node;
1555
Eric Snow96c6af92015-05-29 22:21:39 -06001556 Py_VISIT(od->od_inst_dict);
1557 Py_VISIT(od->od_weakreflist);
Serhiy Storchakad205d012016-01-19 14:46:25 +02001558 _odict_FOREACH(od, node) {
1559 Py_VISIT(_odictnode_KEY(node));
1560 }
Eric Snow96c6af92015-05-29 22:21:39 -06001561 return PyDict_Type.tp_traverse((PyObject *)od, visit, arg);
1562}
1563
1564/* tp_clear */
1565
1566static int
1567odict_tp_clear(PyODictObject *od)
1568{
1569 PyObject *res;
1570 Py_CLEAR(od->od_inst_dict);
1571 Py_CLEAR(od->od_weakreflist);
1572 res = odict_clear(od);
1573 if (res == NULL)
1574 return -1;
1575 Py_DECREF(res);
1576 return 0;
1577}
1578
1579/* tp_richcompare */
1580
1581static PyObject *
1582odict_richcompare(PyObject *v, PyObject *w, int op)
1583{
1584 if (!PyODict_Check(v) || !PyDict_Check(w)) {
1585 Py_RETURN_NOTIMPLEMENTED;
1586 }
1587
1588 if (op == Py_EQ || op == Py_NE) {
1589 PyObject *res, *cmp;
1590 int eq;
1591
1592 cmp = PyDict_Type.tp_richcompare(v, w, op);
1593 if (cmp == NULL)
1594 return NULL;
1595 if (!PyODict_Check(w))
1596 return cmp;
1597 if (op == Py_EQ && cmp == Py_False)
1598 return cmp;
1599 if (op == Py_NE && cmp == Py_True)
1600 return cmp;
1601 Py_DECREF(cmp);
1602
1603 /* Try comparing odict keys. */
1604 eq = _odict_keys_equal((PyODictObject *)v, (PyODictObject *)w);
1605 if (eq < 0)
1606 return NULL;
1607
1608 res = (eq == (op == Py_EQ)) ? Py_True : Py_False;
1609 Py_INCREF(res);
1610 return res;
1611 } else {
1612 Py_RETURN_NOTIMPLEMENTED;
1613 }
Victor Stinnere1871952016-06-08 10:18:18 +02001614}
Eric Snow96c6af92015-05-29 22:21:39 -06001615
1616/* tp_iter */
1617
1618static PyObject *
1619odict_iter(PyODictObject *od)
1620{
1621 return odictiter_new(od, _odict_ITER_KEYS);
Victor Stinnere1871952016-06-08 10:18:18 +02001622}
Eric Snow96c6af92015-05-29 22:21:39 -06001623
1624/* tp_init */
1625
1626static int
1627odict_init(PyObject *self, PyObject *args, PyObject *kwds)
1628{
1629 PyObject *res;
1630 Py_ssize_t len = PyObject_Length(args);
1631
1632 if (len == -1)
1633 return -1;
1634 if (len > 1) {
1635 char *msg = "expected at most 1 arguments, got %d";
1636 PyErr_Format(PyExc_TypeError, msg, len);
1637 return -1;
1638 }
1639
1640 /* __init__() triggering update() is just the way things are! */
1641 res = odict_update(self, args, kwds);
1642 if (res == NULL) {
1643 return -1;
1644 } else {
1645 Py_DECREF(res);
1646 return 0;
1647 }
Victor Stinnere1871952016-06-08 10:18:18 +02001648}
Eric Snow96c6af92015-05-29 22:21:39 -06001649
1650/* tp_new */
1651
1652static PyObject *
1653odict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
1654{
Victor Stinnerca30b022015-09-03 17:50:04 +02001655 PyODictObject *od;
1656
Victor Stinnerca30b022015-09-03 17:50:04 +02001657 od = (PyODictObject *)PyDict_Type.tp_new(type, args, kwds);
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001658 if (od == NULL)
Victor Stinnerca30b022015-09-03 17:50:04 +02001659 return NULL;
Victor Stinnerca30b022015-09-03 17:50:04 +02001660
Victor Stinnerca30b022015-09-03 17:50:04 +02001661 /* type constructor fills the memory with zeros (see
1662 PyType_GenericAlloc()), there is no need to set them to zero again */
1663 if (_odict_resize(od) < 0) {
1664 Py_DECREF(od);
1665 return NULL;
1666 }
1667
1668 return (PyObject*)od;
Eric Snow96c6af92015-05-29 22:21:39 -06001669}
1670
1671/* PyODict_Type */
1672
1673PyTypeObject PyODict_Type = {
1674 PyVarObject_HEAD_INIT(&PyType_Type, 0)
1675 "collections.OrderedDict", /* tp_name */
1676 sizeof(PyODictObject), /* tp_basicsize */
1677 0, /* tp_itemsize */
1678 (destructor)odict_dealloc, /* tp_dealloc */
1679 0, /* tp_print */
1680 0, /* tp_getattr */
1681 0, /* tp_setattr */
1682 0, /* tp_reserved */
1683 (reprfunc)odict_repr, /* tp_repr */
1684 0, /* tp_as_number */
1685 0, /* tp_as_sequence */
1686 &odict_as_mapping, /* tp_as_mapping */
1687 0, /* tp_hash */
1688 0, /* tp_call */
1689 0, /* tp_str */
1690 0, /* tp_getattro */
1691 0, /* tp_setattro */
1692 0, /* tp_as_buffer */
1693 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,/* tp_flags */
1694 odict_doc, /* tp_doc */
1695 (traverseproc)odict_traverse, /* tp_traverse */
1696 (inquiry)odict_tp_clear, /* tp_clear */
1697 (richcmpfunc)odict_richcompare, /* tp_richcompare */
1698 offsetof(PyODictObject, od_weakreflist), /* tp_weaklistoffset */
1699 (getiterfunc)odict_iter, /* tp_iter */
1700 0, /* tp_iternext */
1701 odict_methods, /* tp_methods */
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001702 0, /* tp_members */
1703 odict_getset, /* tp_getset */
Eric Snow96c6af92015-05-29 22:21:39 -06001704 &PyDict_Type, /* tp_base */
1705 0, /* tp_dict */
1706 0, /* tp_descr_get */
1707 0, /* tp_descr_set */
1708 offsetof(PyODictObject, od_inst_dict), /* tp_dictoffset */
1709 (initproc)odict_init, /* tp_init */
1710 PyType_GenericAlloc, /* tp_alloc */
1711 (newfunc)odict_new, /* tp_new */
1712 0, /* tp_free */
1713};
1714
1715
1716/* ----------------------------------------------
1717 * the public OrderedDict API
1718 */
1719
1720PyObject *
1721PyODict_New(void) {
1722 return odict_new(&PyODict_Type, NULL, NULL);
Victor Stinnere1871952016-06-08 10:18:18 +02001723}
Eric Snow96c6af92015-05-29 22:21:39 -06001724
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001725static int
1726_PyODict_SetItem_KnownHash(PyObject *od, PyObject *key, PyObject *value,
1727 Py_hash_t hash)
1728{
1729 int res = _PyDict_SetItem_KnownHash(od, key, value, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001730 if (res == 0) {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001731 res = _odict_add_new_node((PyODictObject *)od, key, hash);
Serhiy Storchakad5f353e2015-11-06 11:07:11 +02001732 if (res < 0) {
1733 /* Revert setting the value on the dict */
1734 PyObject *exc, *val, *tb;
1735 PyErr_Fetch(&exc, &val, &tb);
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001736 (void) _PyDict_DelItem_KnownHash(od, key, hash);
Serhiy Storchakad5f353e2015-11-06 11:07:11 +02001737 _PyErr_ChainExceptions(exc, val, tb);
1738 }
Eric Snow96c6af92015-05-29 22:21:39 -06001739 }
1740 return res;
Victor Stinnere1871952016-06-08 10:18:18 +02001741}
Eric Snow96c6af92015-05-29 22:21:39 -06001742
1743int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001744PyODict_SetItem(PyObject *od, PyObject *key, PyObject *value)
1745{
1746 Py_hash_t hash = PyObject_Hash(key);
1747 if (hash == -1)
1748 return -1;
1749 return _PyODict_SetItem_KnownHash(od, key, value, hash);
Victor Stinnere1871952016-06-08 10:18:18 +02001750}
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001751
1752int
1753PyODict_DelItem(PyObject *od, PyObject *key)
1754{
1755 int res;
1756 Py_hash_t hash = PyObject_Hash(key);
1757 if (hash == -1)
1758 return -1;
1759 res = _odict_clear_node((PyODictObject *)od, NULL, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001760 if (res < 0)
1761 return -1;
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001762 return _PyDict_DelItem_KnownHash(od, key, hash);
Victor Stinnere1871952016-06-08 10:18:18 +02001763}
Eric Snow96c6af92015-05-29 22:21:39 -06001764
1765
1766/* -------------------------------------------
1767 * The OrderedDict views (keys/values/items)
1768 */
1769
1770typedef struct {
1771 PyObject_HEAD
1772 int kind;
1773 PyODictObject *di_odict;
Eric Snowb952ab42015-06-01 23:35:13 -06001774 Py_ssize_t di_size;
Eric Snow4fabf022015-06-04 00:09:56 -06001775 size_t di_state;
Eric Snow96c6af92015-05-29 22:21:39 -06001776 PyObject *di_current;
1777 PyObject *di_result; /* reusable result tuple for iteritems */
1778} odictiterobject;
1779
1780static void
1781odictiter_dealloc(odictiterobject *di)
1782{
1783 _PyObject_GC_UNTRACK(di);
Eric Snowa762af72015-06-01 22:59:08 -06001784 Py_XDECREF(di->di_odict);
Eric Snow96c6af92015-05-29 22:21:39 -06001785 Py_XDECREF(di->di_current);
1786 if (di->kind & (_odict_ITER_KEYS | _odict_ITER_VALUES)) {
1787 Py_DECREF(di->di_result);
1788 }
1789 PyObject_GC_Del(di);
1790}
1791
1792static int
1793odictiter_traverse(odictiterobject *di, visitproc visit, void *arg)
1794{
1795 Py_VISIT(di->di_odict);
1796 Py_VISIT(di->di_current); /* A key could be any type, not just str. */
1797 Py_VISIT(di->di_result);
1798 return 0;
1799}
1800
Eric Snowa762af72015-06-01 22:59:08 -06001801/* In order to protect against modifications during iteration, we track
1802 * the current key instead of the current node. */
Eric Snow96c6af92015-05-29 22:21:39 -06001803static PyObject *
1804odictiter_nextkey(odictiterobject *di)
1805{
Eric Snowa762af72015-06-01 22:59:08 -06001806 PyObject *key = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001807 _ODictNode *node;
1808 int reversed = di->kind & _odict_ITER_REVERSED;
1809
Eric Snowa762af72015-06-01 22:59:08 -06001810 if (di->di_odict == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -06001811 return NULL;
Eric Snowa762af72015-06-01 22:59:08 -06001812 if (di->di_current == NULL)
1813 goto done; /* We're already done. */
1814
Eric Snowb952ab42015-06-01 23:35:13 -06001815 /* Check for unsupported changes. */
Eric Snow4fabf022015-06-04 00:09:56 -06001816 if (di->di_odict->od_state != di->di_state) {
1817 PyErr_SetString(PyExc_RuntimeError,
1818 "OrderedDict mutated during iteration");
1819 goto done;
1820 }
Eric Snowb952ab42015-06-01 23:35:13 -06001821 if (di->di_size != PyODict_SIZE(di->di_odict)) {
1822 PyErr_SetString(PyExc_RuntimeError,
1823 "OrderedDict changed size during iteration");
1824 di->di_size = -1; /* Make this state sticky */
1825 return NULL;
1826 }
1827
Eric Snowa762af72015-06-01 22:59:08 -06001828 /* Get the key. */
Eric Snow96c6af92015-05-29 22:21:39 -06001829 node = _odict_find_node(di->di_odict, di->di_current);
1830 if (node == NULL) {
Serhiy Storchakab45b7b22015-11-04 22:05:38 +02001831 if (!PyErr_Occurred())
1832 PyErr_SetObject(PyExc_KeyError, di->di_current);
Eric Snow96c6af92015-05-29 22:21:39 -06001833 /* Must have been deleted. */
Eric Snowe3dfa9e2015-05-30 12:51:15 -06001834 Py_CLEAR(di->di_current);
Eric Snow96c6af92015-05-29 22:21:39 -06001835 return NULL;
1836 }
1837 key = di->di_current;
1838
1839 /* Advance to the next key. */
1840 node = reversed ? _odictnode_PREV(node) : _odictnode_NEXT(node);
1841 if (node == NULL) {
1842 /* Reached the end. */
1843 di->di_current = NULL;
1844 }
1845 else {
1846 di->di_current = _odictnode_KEY(node);
1847 Py_INCREF(di->di_current);
1848 }
1849
1850 return key;
Eric Snowa762af72015-06-01 22:59:08 -06001851
1852done:
1853 Py_CLEAR(di->di_odict);
1854 return key;
Eric Snow96c6af92015-05-29 22:21:39 -06001855}
1856
1857static PyObject *
1858odictiter_iternext(odictiterobject *di)
1859{
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001860 PyObject *result, *value;
Eric Snow96c6af92015-05-29 22:21:39 -06001861 PyObject *key = odictiter_nextkey(di); /* new reference */
1862
1863 if (key == NULL)
1864 return NULL;
1865
1866 /* Handle the keys case. */
1867 if (! (di->kind & _odict_ITER_VALUES)) {
1868 return key;
1869 }
1870
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001871 value = PyODict_GetItem((PyObject *)di->di_odict, key); /* borrowed */
1872 if (value == NULL) {
1873 if (!PyErr_Occurred())
1874 PyErr_SetObject(PyExc_KeyError, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001875 Py_DECREF(key);
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001876 goto done;
1877 }
1878 Py_INCREF(value);
1879
1880 /* Handle the values case. */
1881 if (!(di->kind & _odict_ITER_KEYS)) {
1882 Py_DECREF(key);
Eric Snow96c6af92015-05-29 22:21:39 -06001883 return value;
1884 }
Eric Snowa762af72015-06-01 22:59:08 -06001885
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001886 /* Handle the items case. */
1887 result = di->di_result;
1888
1889 if (Py_REFCNT(result) == 1) {
1890 /* not in use so we can reuse it
1891 * (the common case during iteration) */
1892 Py_INCREF(result);
1893 Py_DECREF(PyTuple_GET_ITEM(result, 0)); /* borrowed */
1894 Py_DECREF(PyTuple_GET_ITEM(result, 1)); /* borrowed */
1895 }
1896 else {
1897 result = PyTuple_New(2);
1898 if (result == NULL) {
1899 Py_DECREF(key);
1900 Py_DECREF(value);
1901 goto done;
1902 }
1903 }
1904
1905 PyTuple_SET_ITEM(result, 0, key); /* steals reference */
1906 PyTuple_SET_ITEM(result, 1, value); /* steals reference */
1907 return result;
1908
Eric Snowa762af72015-06-01 22:59:08 -06001909done:
1910 Py_CLEAR(di->di_current);
1911 Py_CLEAR(di->di_odict);
1912 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001913}
1914
1915/* No need for tp_clear because odictiterobject is not mutable. */
1916
1917PyDoc_STRVAR(reduce_doc, "Return state information for pickling");
1918
1919static PyObject *
1920odictiter_reduce(odictiterobject *di)
1921{
Eric Snowc5e59602015-05-30 11:28:56 -06001922 PyObject *list, *iter;
Eric Snow96c6af92015-05-29 22:21:39 -06001923
1924 list = PyList_New(0);
1925 if (!list)
1926 return NULL;
1927
1928 /* iterate the temporary into a list */
1929 for(;;) {
1930 PyObject *element = odictiter_iternext(di);
1931 if (element) {
1932 if (PyList_Append(list, element)) {
1933 Py_DECREF(element);
1934 Py_DECREF(list);
1935 return NULL;
1936 }
1937 Py_DECREF(element);
1938 }
1939 else {
1940 /* done iterating? */
1941 break;
1942 }
1943 }
1944 if (PyErr_Occurred()) {
1945 Py_DECREF(list);
1946 return NULL;
1947 }
Eric Snowc5e59602015-05-30 11:28:56 -06001948 iter = _PyObject_GetBuiltin("iter");
1949 if (iter == NULL) {
1950 Py_DECREF(list);
1951 return NULL;
1952 }
1953 return Py_BuildValue("N(N)", iter, list);
Eric Snow96c6af92015-05-29 22:21:39 -06001954}
1955
1956static PyMethodDef odictiter_methods[] = {
1957 {"__reduce__", (PyCFunction)odictiter_reduce, METH_NOARGS, reduce_doc},
1958 {NULL, NULL} /* sentinel */
1959};
1960
1961PyTypeObject PyODictIter_Type = {
1962 PyVarObject_HEAD_INIT(&PyType_Type, 0)
1963 "odict_iterator", /* tp_name */
1964 sizeof(odictiterobject), /* tp_basicsize */
1965 0, /* tp_itemsize */
1966 /* methods */
1967 (destructor)odictiter_dealloc, /* tp_dealloc */
1968 0, /* tp_print */
1969 0, /* tp_getattr */
1970 0, /* tp_setattr */
1971 0, /* tp_reserved */
1972 0, /* tp_repr */
1973 0, /* tp_as_number */
1974 0, /* tp_as_sequence */
1975 0, /* tp_as_mapping */
1976 0, /* tp_hash */
1977 0, /* tp_call */
1978 0, /* tp_str */
1979 PyObject_GenericGetAttr, /* tp_getattro */
1980 0, /* tp_setattro */
1981 0, /* tp_as_buffer */
1982 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */
1983 0, /* tp_doc */
1984 (traverseproc)odictiter_traverse, /* tp_traverse */
1985 0, /* tp_clear */
1986 0, /* tp_richcompare */
1987 0, /* tp_weaklistoffset */
1988 PyObject_SelfIter, /* tp_iter */
1989 (iternextfunc)odictiter_iternext, /* tp_iternext */
1990 odictiter_methods, /* tp_methods */
1991 0,
1992};
1993
1994static PyObject *
1995odictiter_new(PyODictObject *od, int kind)
1996{
1997 odictiterobject *di;
1998 _ODictNode *node;
1999 int reversed = kind & _odict_ITER_REVERSED;
2000
2001 di = PyObject_GC_New(odictiterobject, &PyODictIter_Type);
2002 if (di == NULL)
2003 return NULL;
2004
2005 if (kind & (_odict_ITER_KEYS | _odict_ITER_VALUES)){
2006 di->di_result = PyTuple_Pack(2, Py_None, Py_None);
2007 if (di->di_result == NULL) {
2008 Py_DECREF(di);
2009 return NULL;
2010 }
2011 }
2012 else
2013 di->di_result = NULL;
2014
2015 di->kind = kind;
2016 node = reversed ? _odict_LAST(od) : _odict_FIRST(od);
2017 di->di_current = node ? _odictnode_KEY(node) : NULL;
2018 Py_XINCREF(di->di_current);
Eric Snowb952ab42015-06-01 23:35:13 -06002019 di->di_size = PyODict_SIZE(od);
Eric Snow4fabf022015-06-04 00:09:56 -06002020 di->di_state = od->od_state;
Eric Snow96c6af92015-05-29 22:21:39 -06002021 di->di_odict = od;
2022 Py_INCREF(od);
2023
2024 _PyObject_GC_TRACK(di);
2025 return (PyObject *)di;
2026}
2027
2028/* keys() */
2029
2030static PyObject *
2031odictkeys_iter(_PyDictViewObject *dv)
2032{
2033 if (dv->dv_dict == NULL) {
2034 Py_RETURN_NONE;
2035 }
2036 return odictiter_new((PyODictObject *)dv->dv_dict,
2037 _odict_ITER_KEYS);
2038}
2039
2040static PyObject *
2041odictkeys_reversed(_PyDictViewObject *dv)
2042{
2043 if (dv->dv_dict == NULL) {
2044 Py_RETURN_NONE;
2045 }
2046 return odictiter_new((PyODictObject *)dv->dv_dict,
2047 _odict_ITER_KEYS|_odict_ITER_REVERSED);
2048}
2049
2050static PyMethodDef odictkeys_methods[] = {
2051 {"__reversed__", (PyCFunction)odictkeys_reversed, METH_NOARGS, NULL},
2052 {NULL, NULL} /* sentinel */
2053};
2054
2055PyTypeObject PyODictKeys_Type = {
2056 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2057 "odict_keys", /* tp_name */
2058 0, /* tp_basicsize */
2059 0, /* tp_itemsize */
2060 0, /* tp_dealloc */
2061 0, /* tp_print */
2062 0, /* tp_getattr */
2063 0, /* tp_setattr */
2064 0, /* tp_reserved */
2065 0, /* tp_repr */
2066 0, /* tp_as_number */
2067 0, /* tp_as_sequence */
2068 0, /* tp_as_mapping */
2069 0, /* tp_hash */
2070 0, /* tp_call */
2071 0, /* tp_str */
2072 0, /* tp_getattro */
2073 0, /* tp_setattro */
2074 0, /* tp_as_buffer */
2075 0, /* tp_flags */
2076 0, /* tp_doc */
2077 0, /* tp_traverse */
2078 0, /* tp_clear */
2079 0, /* tp_richcompare */
2080 0, /* tp_weaklistoffset */
2081 (getiterfunc)odictkeys_iter, /* tp_iter */
2082 0, /* tp_iternext */
2083 odictkeys_methods, /* tp_methods */
2084 0, /* tp_members */
2085 0, /* tp_getset */
2086 &PyDictKeys_Type, /* tp_base */
2087};
2088
2089static PyObject *
2090odictkeys_new(PyObject *od)
2091{
2092 return _PyDictView_New(od, &PyODictKeys_Type);
2093}
2094
2095/* items() */
2096
2097static PyObject *
2098odictitems_iter(_PyDictViewObject *dv)
2099{
2100 if (dv->dv_dict == NULL) {
2101 Py_RETURN_NONE;
2102 }
2103 return odictiter_new((PyODictObject *)dv->dv_dict,
2104 _odict_ITER_KEYS|_odict_ITER_VALUES);
2105}
2106
2107static PyObject *
2108odictitems_reversed(_PyDictViewObject *dv)
2109{
2110 if (dv->dv_dict == NULL) {
2111 Py_RETURN_NONE;
2112 }
2113 return odictiter_new((PyODictObject *)dv->dv_dict,
2114 _odict_ITER_KEYS|_odict_ITER_VALUES|_odict_ITER_REVERSED);
2115}
2116
2117static PyMethodDef odictitems_methods[] = {
2118 {"__reversed__", (PyCFunction)odictitems_reversed, METH_NOARGS, NULL},
2119 {NULL, NULL} /* sentinel */
2120};
2121
2122PyTypeObject PyODictItems_Type = {
2123 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2124 "odict_items", /* tp_name */
2125 0, /* tp_basicsize */
2126 0, /* tp_itemsize */
2127 0, /* tp_dealloc */
2128 0, /* tp_print */
2129 0, /* tp_getattr */
2130 0, /* tp_setattr */
2131 0, /* tp_reserved */
2132 0, /* tp_repr */
2133 0, /* tp_as_number */
2134 0, /* tp_as_sequence */
2135 0, /* tp_as_mapping */
2136 0, /* tp_hash */
2137 0, /* tp_call */
2138 0, /* tp_str */
2139 0, /* tp_getattro */
2140 0, /* tp_setattro */
2141 0, /* tp_as_buffer */
2142 0, /* tp_flags */
2143 0, /* tp_doc */
2144 0, /* tp_traverse */
2145 0, /* tp_clear */
2146 0, /* tp_richcompare */
2147 0, /* tp_weaklistoffset */
2148 (getiterfunc)odictitems_iter, /* tp_iter */
2149 0, /* tp_iternext */
2150 odictitems_methods, /* tp_methods */
2151 0, /* tp_members */
2152 0, /* tp_getset */
2153 &PyDictItems_Type, /* tp_base */
2154};
2155
2156static PyObject *
2157odictitems_new(PyObject *od)
2158{
2159 return _PyDictView_New(od, &PyODictItems_Type);
2160}
2161
2162/* values() */
2163
2164static PyObject *
2165odictvalues_iter(_PyDictViewObject *dv)
2166{
2167 if (dv->dv_dict == NULL) {
2168 Py_RETURN_NONE;
2169 }
2170 return odictiter_new((PyODictObject *)dv->dv_dict,
2171 _odict_ITER_VALUES);
2172}
2173
2174static PyObject *
2175odictvalues_reversed(_PyDictViewObject *dv)
2176{
2177 if (dv->dv_dict == NULL) {
2178 Py_RETURN_NONE;
2179 }
2180 return odictiter_new((PyODictObject *)dv->dv_dict,
2181 _odict_ITER_VALUES|_odict_ITER_REVERSED);
2182}
2183
2184static PyMethodDef odictvalues_methods[] = {
2185 {"__reversed__", (PyCFunction)odictvalues_reversed, METH_NOARGS, NULL},
2186 {NULL, NULL} /* sentinel */
2187};
2188
2189PyTypeObject PyODictValues_Type = {
2190 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2191 "odict_values", /* tp_name */
2192 0, /* tp_basicsize */
2193 0, /* tp_itemsize */
2194 0, /* tp_dealloc */
2195 0, /* tp_print */
2196 0, /* tp_getattr */
2197 0, /* tp_setattr */
2198 0, /* tp_reserved */
2199 0, /* tp_repr */
2200 0, /* tp_as_number */
2201 0, /* tp_as_sequence */
2202 0, /* tp_as_mapping */
2203 0, /* tp_hash */
2204 0, /* tp_call */
2205 0, /* tp_str */
2206 0, /* tp_getattro */
2207 0, /* tp_setattro */
2208 0, /* tp_as_buffer */
2209 0, /* tp_flags */
2210 0, /* tp_doc */
2211 0, /* tp_traverse */
2212 0, /* tp_clear */
2213 0, /* tp_richcompare */
2214 0, /* tp_weaklistoffset */
2215 (getiterfunc)odictvalues_iter, /* tp_iter */
2216 0, /* tp_iternext */
2217 odictvalues_methods, /* tp_methods */
2218 0, /* tp_members */
2219 0, /* tp_getset */
2220 &PyDictValues_Type, /* tp_base */
2221};
2222
2223static PyObject *
2224odictvalues_new(PyObject *od)
2225{
2226 return _PyDictView_New(od, &PyODictValues_Type);
2227}
2228
2229
2230/* ----------------------------------------------
Martin Panter536d70e2017-01-14 08:23:08 +00002231 MutableMapping implementations
Eric Snow96c6af92015-05-29 22:21:39 -06002232
2233Mapping:
2234
2235============ ===========
2236method uses
2237============ ===========
2238__contains__ __getitem__
2239__eq__ items
2240__getitem__ +
2241__iter__ +
2242__len__ +
2243__ne__ __eq__
2244get __getitem__
2245items ItemsView
2246keys KeysView
2247values ValuesView
2248============ ===========
2249
2250ItemsView uses __len__, __iter__, and __getitem__.
2251KeysView uses __len__, __iter__, and __contains__.
2252ValuesView uses __len__, __iter__, and __getitem__.
2253
2254MutableMapping:
2255
2256============ ===========
2257method uses
2258============ ===========
2259__delitem__ +
2260__setitem__ +
2261clear popitem
2262pop __getitem__
2263 __delitem__
2264popitem __iter__
2265 _getitem__
2266 __delitem__
2267setdefault __getitem__
2268 __setitem__
2269update __setitem__
2270============ ===========
2271*/
2272
2273static int
2274mutablemapping_add_pairs(PyObject *self, PyObject *pairs)
2275{
2276 PyObject *pair, *iterator, *unexpected;
2277 int res = 0;
2278
2279 iterator = PyObject_GetIter(pairs);
2280 if (iterator == NULL)
2281 return -1;
2282 PyErr_Clear();
2283
2284 while ((pair = PyIter_Next(iterator)) != NULL) {
2285 /* could be more efficient (see UNPACK_SEQUENCE in ceval.c) */
Eric Snowc5e59602015-05-30 11:28:56 -06002286 PyObject *key = NULL, *value = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06002287 PyObject *pair_iterator = PyObject_GetIter(pair);
Eric Snowc5e59602015-05-30 11:28:56 -06002288 if (pair_iterator == NULL)
2289 goto Done;
Eric Snow96c6af92015-05-29 22:21:39 -06002290
2291 key = PyIter_Next(pair_iterator);
2292 if (key == NULL) {
2293 if (!PyErr_Occurred())
2294 PyErr_SetString(PyExc_ValueError,
2295 "need more than 0 values to unpack");
2296 goto Done;
2297 }
2298
2299 value = PyIter_Next(pair_iterator);
2300 if (value == NULL) {
2301 if (!PyErr_Occurred())
2302 PyErr_SetString(PyExc_ValueError,
2303 "need more than 1 value to unpack");
2304 goto Done;
2305 }
2306
2307 unexpected = PyIter_Next(pair_iterator);
2308 if (unexpected != NULL) {
2309 Py_DECREF(unexpected);
2310 PyErr_SetString(PyExc_ValueError,
2311 "too many values to unpack (expected 2)");
2312 goto Done;
2313 }
2314 else if (PyErr_Occurred())
2315 goto Done;
2316
2317 res = PyObject_SetItem(self, key, value);
2318
2319Done:
2320 Py_DECREF(pair);
Eric Snowc5e59602015-05-30 11:28:56 -06002321 Py_XDECREF(pair_iterator);
Eric Snow96c6af92015-05-29 22:21:39 -06002322 Py_XDECREF(key);
2323 Py_XDECREF(value);
2324 if (PyErr_Occurred())
2325 break;
2326 }
2327 Py_DECREF(iterator);
2328
2329 if (res < 0 || PyErr_Occurred() != NULL)
2330 return -1;
2331 else
2332 return 0;
2333}
2334
2335static PyObject *
2336mutablemapping_update(PyObject *self, PyObject *args, PyObject *kwargs)
2337{
2338 int res = 0;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002339 Py_ssize_t len;
2340 _Py_IDENTIFIER(items);
2341 _Py_IDENTIFIER(keys);
Eric Snow96c6af92015-05-29 22:21:39 -06002342
2343 /* first handle args, if any */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002344 assert(args == NULL || PyTuple_Check(args));
2345 len = (args != NULL) ? PyTuple_GET_SIZE(args) : 0;
Benjamin Peterson99e96f22015-06-06 23:20:32 -05002346 if (len > 1) {
Eric Snow96c6af92015-05-29 22:21:39 -06002347 char *msg = "update() takes at most 1 positional argument (%d given)";
2348 PyErr_Format(PyExc_TypeError, msg, len);
2349 return NULL;
2350 }
Eric Snow96c6af92015-05-29 22:21:39 -06002351
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002352 if (len) {
Benjamin Peterson0718de92015-06-07 00:00:42 -05002353 PyObject *other = PyTuple_GET_ITEM(args, 0); /* borrowed reference */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002354 assert(other != NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002355 Py_INCREF(other);
Eric Snow06aed902016-09-09 11:59:08 -07002356 if PyDict_CheckExact(other) {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002357 PyObject *items;
2358 if (PyDict_CheckExact(other))
2359 items = PyDict_Items(other);
2360 else
2361 items = _PyObject_CallMethodId(other, &PyId_items, NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002362 Py_DECREF(other);
2363 if (items == NULL)
2364 return NULL;
2365 res = mutablemapping_add_pairs(self, items);
2366 Py_DECREF(items);
2367 if (res == -1)
2368 return NULL;
2369 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002370 else if (_PyObject_HasAttrId(other, &PyId_keys)) { /* never fails */
Benjamin Peterson0718de92015-06-07 00:00:42 -05002371 PyObject *keys, *iterator, *key;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002372 keys = _PyObject_CallMethodIdObjArgs(other, &PyId_keys, NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002373 if (keys == NULL) {
2374 Py_DECREF(other);
2375 return NULL;
2376 }
2377 iterator = PyObject_GetIter(keys);
2378 Py_DECREF(keys);
2379 if (iterator == NULL) {
2380 Py_DECREF(other);
2381 return NULL;
2382 }
2383 while (res == 0 && (key = PyIter_Next(iterator))) {
2384 PyObject *value = PyObject_GetItem(other, key);
2385 if (value != NULL) {
2386 res = PyObject_SetItem(self, key, value);
2387 Py_DECREF(value);
2388 }
2389 else {
2390 res = -1;
2391 }
2392 Py_DECREF(key);
2393 }
2394 Py_DECREF(other);
2395 Py_DECREF(iterator);
2396 if (res != 0 || PyErr_Occurred())
2397 return NULL;
2398 }
Eric Snow06aed902016-09-09 11:59:08 -07002399 else if (_PyObject_HasAttrId(other, &PyId_items)) { /* never fails */
2400 PyObject *items;
2401 if (PyDict_CheckExact(other))
2402 items = PyDict_Items(other);
2403 else
2404 items = _PyObject_CallMethodId(other, &PyId_items, NULL);
2405 Py_DECREF(other);
2406 if (items == NULL)
2407 return NULL;
2408 res = mutablemapping_add_pairs(self, items);
2409 Py_DECREF(items);
2410 if (res == -1)
2411 return NULL;
2412 }
Benjamin Peterson0718de92015-06-07 00:00:42 -05002413 else {
2414 res = mutablemapping_add_pairs(self, other);
2415 Py_DECREF(other);
2416 if (res != 0)
2417 return NULL;
2418 }
2419 }
2420
2421 /* now handle kwargs */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002422 assert(kwargs == NULL || PyDict_Check(kwargs));
Serhiy Storchaka5ab81d72016-12-16 16:18:57 +02002423 if (kwargs != NULL && PyDict_GET_SIZE(kwargs)) {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002424 PyObject *items = PyDict_Items(kwargs);
Eric Snow96c6af92015-05-29 22:21:39 -06002425 if (items == NULL)
2426 return NULL;
2427 res = mutablemapping_add_pairs(self, items);
2428 Py_DECREF(items);
2429 if (res == -1)
2430 return NULL;
2431 }
Eric Snow96c6af92015-05-29 22:21:39 -06002432
2433 Py_RETURN_NONE;
2434}