blob: b4940e39ec3f5faf09bb096fdc0e7d3b1e322d88 [file] [log] [blame]
Eric Snow96c6af92015-05-29 22:21:39 -06001/* Ordered Dictionary object implementation.
2
3This implementation is necessarily explicitly equivalent to the pure Python
4OrderedDict class in Lib/collections/__init__.py. The strategy there
5involves using a doubly-linked-list to capture the order. We keep to that
6strategy, using a lower-level linked-list.
7
8About the Linked-List
9=====================
10
11For the linked list we use a basic doubly-linked-list. Using a circularly-
12linked-list does have some benefits, but they don't apply so much here
13since OrderedDict is focused on the ends of the list (for the most part).
14Furthermore, there are some features of generic linked-lists that we simply
15don't need for OrderedDict. Thus a simple custom implementation meets our
16needs. Alternatives to our simple approach include the QCIRCLE_*
17macros from BSD's queue.h, and the linux's list.h.
18
19Getting O(1) Node Lookup
20------------------------
21
22One invariant of Python's OrderedDict is that it preserves time complexity
23of dict's methods, particularly the O(1) operations. Simply adding a
24linked-list on top of dict is not sufficient here; operations for nodes in
25the middle of the linked-list implicitly require finding the node first.
26With a simple linked-list like we're using, that is an O(n) operation.
27Consequently, methods like __delitem__() would change from O(1) to O(n),
28which is unacceptable.
29
30In order to preserve O(1) performance for node removal (finding nodes), we
31must do better than just looping through the linked-list. Here are options
32we've considered:
33
341. use a second dict to map keys to nodes (a la the pure Python version).
352. keep a simple hash table mirroring the order of dict's, mapping each key
36 to the corresponding node in the linked-list.
373. use a version of shared keys (split dict) that allows non-unicode keys.
384. have the value stored for each key be a (value, node) pair, and adjust
39 __getitem__(), get(), etc. accordingly.
40
41The approach with the least performance impact (time and space) is #2,
42mirroring the key order of dict's dk_enties with an array of node pointers.
43While lookdict() and friends (dk_lookup) don't give us the index into the
44array, we make use of pointer arithmetic to get that index. An alternative
45would be to refactor lookdict() to provide the index, explicitly exposing
46the implementation detail. We could even just use a custom lookup function
47for OrderedDict that facilitates our need. However, both approaches are
48significantly more complicated than just using pointer arithmetic.
49
50The catch with mirroring the hash table ordering is that we have to keep
51the ordering in sync through any dict resizes. However, that order only
52matters during node lookup. We can simply defer any potential resizing
53until we need to do a lookup.
54
55Linked-List Nodes
56-----------------
57
58The current implementation stores a pointer to the associated key only.
59One alternative would be to store a pointer to the PyDictKeyEntry instead.
60This would save one pointer de-reference per item, which is nice during
61calls to values() and items(). However, it adds unnecessary overhead
62otherwise, so we stick with just the key.
63
64Linked-List API
65---------------
66
67As noted, the linked-list implemented here does not have all the bells and
68whistles. However, we recognize that the implementation may need to
69change to accommodate performance improvements or extra functionality. To
70that end, We use a simple API to interact with the linked-list. Here's a
71summary of the methods/macros:
72
73Node info:
74
75* _odictnode_KEY(node)
76* _odictnode_VALUE(od, node)
77* _odictnode_PREV(node)
78* _odictnode_NEXT(node)
79
80Linked-List info:
81
82* _odict_FIRST(od)
83* _odict_LAST(od)
84* _odict_EMPTY(od)
85* _odict_FOREACH(od, node) - used in place of `for (node=...)`
86
87For adding nodes:
88
89* _odict_add_head(od, node)
90* _odict_add_tail(od, node)
Serhiy Storchaka19a70e72015-11-13 14:48:36 +020091* _odict_add_new_node(od, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -060092
93For removing nodes:
94
Serhiy Storchaka19a70e72015-11-13 14:48:36 +020095* _odict_clear_node(od, node, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -060096* _odict_clear_nodes(od, clear_each)
97
98Others:
99
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200100* _odict_find_node_hash(od, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600101* _odict_find_node(od, key)
102* _odict_keys_equal(od1, od2)
103
104Used, but specific to the linked-list implementation:
105
106* _odict_free_fast_nodes(od)
107
108And here's a look at how the linked-list relates to the OrderedDict API:
109
110============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
111method key val prev next mem 1st last empty iter find add rmv clr keq
112============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
113__del__ ~ X
114__delitem__ free ~ node
115__eq__ ~ X
116__iter__ X X
117__new__ X X
118__reduce__ X ~ X
119__repr__ X X X
120__reversed__ X X
121__setitem__ key
122__sizeof__ size X
123clear ~ ~ X
124copy X X X
125items X X X
126keys X X
127move_to_end X X X ~ h/t key
128pop free key
129popitem X X free X X node
130setdefault ~ ? ~
131values X X
132============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
133
134__delitem__ is the only method that directly relies on finding an arbitrary
135node in the linked-list. Everything else is iteration or relates to the
136ends of the linked-list.
137
138Situation that Endangers Consistency
139------------------------------------
140Using a raw linked-list for OrderedDict exposes a key situation that can
141cause problems. If a node is stored in a variable, there is a chance that
142the node may have been deallocated before the variable gets used, thus
143potentially leading to a segmentation fault. A key place where this shows
144up is during iteration through the linked list (via _odict_FOREACH or
145otherwise).
146
147A number of solutions are available to resolve this situation:
148
149* defer looking up the node until as late as possible and certainly after
150 any code that could possibly result in a deletion;
151* if the node is needed both before and after a point where the node might
152 be removed, do a check before using the node at the "after" location to
153 see if the node is still valid;
154* like the last one, but simply pull the node again to ensure it's right;
155* keep the key in the variable instead of the node and then look up the
156 node using the key at the point where the node is needed (this is what
157 we do for the iterators).
158
159Another related problem, preserving consistent ordering during iteration,
160is described below. That one is not exclusive to using linked-lists.
161
162
163Challenges from Subclassing dict
164================================
165
166OrderedDict subclasses dict, which is an unusual relationship between two
167builtin types (other than the base object type). Doing so results in
168some complication and deserves further explanation. There are two things
169to consider here. First, in what circumstances or with what adjustments
170can OrderedDict be used as a drop-in replacement for dict (at the C level)?
171Second, how can the OrderedDict implementation leverage the dict
172implementation effectively without introducing unnecessary coupling or
173inefficiencies?
174
175This second point is reflected here and in the implementation, so the
176further focus is on the first point. It is worth noting that for
177overridden methods, the dict implementation is deferred to as much as
178possible. Furthermore, coupling is limited to as little as is reasonable.
179
180Concrete API Compatibility
181--------------------------
182
183Use of the concrete C-API for dict (PyDict_*) with OrderedDict is
184problematic. (See http://bugs.python.org/issue10977.) The concrete API
185has a number of hard-coded assumptions tied to the dict implementation.
186This is, in part, due to performance reasons, which is understandable
187given the part dict plays in Python.
188
189Any attempt to replace dict with OrderedDict for any role in the
190interpreter (e.g. **kwds) faces a challenge. Such any effort must
191recognize that the instances in affected locations currently interact with
192the concrete API.
193
194Here are some ways to address this challenge:
195
1961. Change the relevant usage of the concrete API in CPython and add
Martin Panter69332c12016-08-04 13:07:31 +0000197 PyDict_CheckExact() calls to each of the concrete API functions.
Eric Snow96c6af92015-05-29 22:21:39 -06001982. Adjust the relevant concrete API functions to explicitly accommodate
199 OrderedDict.
2003. As with #1, add the checks, but improve the abstract API with smart fast
201 paths for dict and OrderedDict, and refactor CPython to use the abstract
202 API. Improvements to the abstract API would be valuable regardless.
203
204Adding the checks to the concrete API would help make any interpreter
205switch to OrderedDict less painful for extension modules. However, this
206won't work. The equivalent C API call to `dict.__setitem__(obj, k, v)`
207is 'PyDict_SetItem(obj, k, v)`. This illustrates how subclasses in C call
208the base class's methods, since there is no equivalent of super() in the
209C API. Calling into Python for parent class API would work, but some
210extension modules already rely on this feature of the concrete API.
211
212For reference, here is a breakdown of some of the dict concrete API:
213
214========================== ============= =======================
215concrete API uses abstract API
216========================== ============= =======================
217PyDict_Check PyMapping_Check
218(PyDict_CheckExact) -
219(PyDict_New) -
220(PyDictProxy_New) -
221PyDict_Clear -
222PyDict_Contains PySequence_Contains
223PyDict_Copy -
224PyDict_SetItem PyObject_SetItem
225PyDict_SetItemString PyMapping_SetItemString
226PyDict_DelItem PyMapping_DelItem
227PyDict_DelItemString PyMapping_DelItemString
228PyDict_GetItem -
229PyDict_GetItemWithError PyObject_GetItem
230_PyDict_GetItemIdWithError -
231PyDict_GetItemString PyMapping_GetItemString
232PyDict_Items PyMapping_Items
233PyDict_Keys PyMapping_Keys
234PyDict_Values PyMapping_Values
235PyDict_Size PyMapping_Size
236 PyMapping_Length
237PyDict_Next PyIter_Next
238_PyDict_Next -
239PyDict_Merge -
240PyDict_Update -
241PyDict_MergeFromSeq2 -
242PyDict_ClearFreeList -
243- PyMapping_HasKeyString
244- PyMapping_HasKey
245========================== ============= =======================
246
247
248The dict Interface Relative to OrderedDict
249==========================================
250
251Since OrderedDict subclasses dict, understanding the various methods and
252attributes of dict is important for implementing OrderedDict.
253
254Relevant Type Slots
255-------------------
256
257================= ================ =================== ================
258slot attribute object dict
259================= ================ =================== ================
260tp_dealloc - object_dealloc dict_dealloc
261tp_repr __repr__ object_repr dict_repr
262sq_contains __contains__ - dict_contains
263mp_length __len__ - dict_length
264mp_subscript __getitem__ - dict_subscript
265mp_ass_subscript __setitem__ - dict_ass_sub
266 __delitem__
267tp_hash __hash__ _Py_HashPointer ..._HashNotImpl
268tp_str __str__ object_str -
269tp_getattro __getattribute__ ..._GenericGetAttr (repeated)
270 __getattr__
271tp_setattro __setattr__ ..._GenericSetAttr (disabled)
272tp_doc __doc__ (literal) dictionary_doc
273tp_traverse - - dict_traverse
274tp_clear - - dict_tp_clear
275tp_richcompare __eq__ object_richcompare dict_richcompare
276 __ne__
277tp_weaklistoffset (__weakref__) - -
278tp_iter __iter__ - dict_iter
279tp_dictoffset (__dict__) - -
280tp_init __init__ object_init dict_init
281tp_alloc - PyType_GenericAlloc (repeated)
282tp_new __new__ object_new dict_new
283tp_free - PyObject_Del PyObject_GC_Del
284================= ================ =================== ================
285
286Relevant Methods
287----------------
288
289================ =================== ===============
290method object dict
291================ =================== ===============
292__reduce__ object_reduce -
293__sizeof__ object_sizeof dict_sizeof
294clear - dict_clear
295copy - dict_copy
296fromkeys - dict_fromkeys
297get - dict_get
298items - dictitems_new
299keys - dictkeys_new
300pop - dict_pop
301popitem - dict_popitem
302setdefault - dict_setdefault
303update - dict_update
304values - dictvalues_new
305================ =================== ===============
306
307
308Pure Python OrderedDict
309=======================
310
311As already noted, compatibility with the pure Python OrderedDict
312implementation is a key goal of this C implementation. To further that
313goal, here's a summary of how OrderedDict-specific methods are implemented
314in collections/__init__.py. Also provided is an indication of which
315methods directly mutate or iterate the object, as well as any relationship
316with the underlying linked-list.
317
318============= ============== == ================ === === ====
319method impl used ll uses inq mut iter
320============= ============== == ================ === === ====
321__contains__ dict - - X
322__delitem__ OrderedDict Y dict.__delitem__ X
323__eq__ OrderedDict N OrderedDict ~
324 dict.__eq__
325 __iter__
326__getitem__ dict - - X
327__iter__ OrderedDict Y - X
328__init__ OrderedDict N update
329__len__ dict - - X
330__ne__ MutableMapping - __eq__ ~
331__reduce__ OrderedDict N OrderedDict ~
332 __iter__
333 __getitem__
334__repr__ OrderedDict N __class__ ~
335 items
336__reversed__ OrderedDict Y - X
337__setitem__ OrderedDict Y __contains__ X
338 dict.__setitem__
339__sizeof__ OrderedDict Y __len__ ~
340 __dict__
341clear OrderedDict Y dict.clear X
342copy OrderedDict N __class__
343 __init__
344fromkeys OrderedDict N __setitem__
345get dict - - ~
346items MutableMapping - ItemsView X
347keys MutableMapping - KeysView X
348move_to_end OrderedDict Y - X
349pop OrderedDict N __contains__ X
350 __getitem__
351 __delitem__
352popitem OrderedDict Y dict.pop X
353setdefault OrderedDict N __contains__ ~
354 __getitem__
355 __setitem__
356update MutableMapping - __setitem__ ~
357values MutableMapping - ValuesView X
358============= ============== == ================ === === ====
359
360__reversed__ and move_to_end are both exclusive to OrderedDict.
361
362
363C OrderedDict Implementation
364============================
365
366================= ================
367slot impl
368================= ================
369tp_dealloc odict_dealloc
370tp_repr odict_repr
371mp_ass_subscript odict_ass_sub
372tp_doc odict_doc
373tp_traverse odict_traverse
374tp_clear odict_tp_clear
375tp_richcompare odict_richcompare
376tp_weaklistoffset (offset)
377tp_iter odict_iter
378tp_dictoffset (offset)
379tp_init odict_init
380tp_alloc (repeated)
381tp_new odict_new
382================= ================
383
384================= ================
385method impl
386================= ================
387__reduce__ odict_reduce
388__sizeof__ odict_sizeof
389clear odict_clear
390copy odict_copy
391fromkeys odict_fromkeys
392items odictitems_new
393keys odictkeys_new
394pop odict_pop
395popitem odict_popitem
396setdefault odict_setdefault
397update odict_update
398values odictvalues_new
399================= ================
400
401Inherited unchanged from object/dict:
402
403================ ==========================
404method type field
405================ ==========================
406- tp_free
407__contains__ tp_as_sequence.sq_contains
408__getattr__ tp_getattro
409__getattribute__ tp_getattro
410__getitem__ tp_as_mapping.mp_subscript
411__hash__ tp_hash
412__len__ tp_as_mapping.mp_length
413__setattr__ tp_setattro
414__str__ tp_str
415get -
416================ ==========================
417
418
419Other Challenges
420================
421
422Preserving Ordering During Iteration
423------------------------------------
424During iteration through an OrderedDict, it is possible that items could
425get added, removed, or reordered. For a linked-list implementation, as
426with some other implementations, that situation may lead to undefined
427behavior. The documentation for dict mentions this in the `iter()` section
428of http://docs.python.org/3.4/library/stdtypes.html#dictionary-view-objects.
429In this implementation we follow dict's lead (as does the pure Python
430implementation) for __iter__(), keys(), values(), and items().
431
432For internal iteration (using _odict_FOREACH or not), there is still the
433risk that not all nodes that we expect to be seen in the loop actually get
434seen. Thus, we are careful in each of those places to ensure that they
435are. This comes, of course, at a small price at each location. The
436solutions are much the same as those detailed in the `Situation that
437Endangers Consistency` section above.
438
439
440Potential Optimizations
441=======================
442
443* Allocate the nodes as a block via od_fast_nodes instead of individually.
444 - Set node->key to NULL to indicate the node is not-in-use.
445 - Add _odict_EXISTS()?
446 - How to maintain consistency across resizes? Existing node pointers
447 would be invalidate after a resize, which is particularly problematic
448 for the iterators.
449* Use a more stream-lined implementation of update() and, likely indirectly,
450 __init__().
451
452*/
453
454/* TODO
455
456sooner:
457- reentrancy (make sure everything is at a thread-safe state when calling
458 into Python). I've already checked this multiple times, but want to
459 make one more pass.
460- add unit tests for reentrancy?
461
462later:
463- make the dict views support the full set API (the pure Python impl does)
464- implement a fuller MutableMapping API in C?
465- move the MutableMapping implementation to abstract.c?
466- optimize mutablemapping_update
467- use PyObject_MALLOC (small object allocator) for odict nodes?
468- support subclasses better (e.g. in odict_richcompare)
469
470*/
471
472#include "Python.h"
473#include "structmember.h"
474#include "dict-common.h"
475#include <stddef.h>
476
477
478typedef struct _odictnode _ODictNode;
479
480/* PyODictObject */
481struct _odictobject {
Eric Snow4fabf022015-06-04 00:09:56 -0600482 PyDictObject od_dict; /* the underlying dict */
483 _ODictNode *od_first; /* first node in the linked list, if any */
484 _ODictNode *od_last; /* last node in the linked list, if any */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200485 /* od_fast_nodes, od_fast_nodes_size and od_resize_sentinel are managed
486 * by _odict_resize().
Eric Snow8c7f9552015-08-07 17:45:12 -0600487 * Note that we rely on implementation details of dict for both. */
488 _ODictNode **od_fast_nodes; /* hash table that mirrors the dict table */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200489 Py_ssize_t od_fast_nodes_size;
490 void *od_resize_sentinel; /* changes if odict should be resized */
Eric Snow8c7f9552015-08-07 17:45:12 -0600491
Eric Snow4fabf022015-06-04 00:09:56 -0600492 size_t od_state; /* incremented whenever the LL changes */
493 PyObject *od_inst_dict; /* OrderedDict().__dict__ */
494 PyObject *od_weakreflist; /* holds weakrefs to the odict */
Eric Snow96c6af92015-05-29 22:21:39 -0600495};
496
497
498/* ----------------------------------------------
499 * odict keys (a simple doubly-linked list)
500 */
501
502struct _odictnode {
503 PyObject *key;
Eric Snow4c729182015-06-03 10:50:37 -0600504 Py_hash_t hash;
Eric Snow96c6af92015-05-29 22:21:39 -0600505 _ODictNode *next;
506 _ODictNode *prev;
507};
508
509#define _odictnode_KEY(node) \
510 (node->key)
Eric Snow4c729182015-06-03 10:50:37 -0600511#define _odictnode_HASH(node) \
512 (node->hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600513/* borrowed reference */
514#define _odictnode_VALUE(node, od) \
Eric Snowa762af72015-06-01 22:59:08 -0600515 PyODict_GetItemWithError((PyObject *)od, _odictnode_KEY(node))
Eric Snow96c6af92015-05-29 22:21:39 -0600516#define _odictnode_PREV(node) (node->prev)
517#define _odictnode_NEXT(node) (node->next)
518
519#define _odict_FIRST(od) (((PyODictObject *)od)->od_first)
520#define _odict_LAST(od) (((PyODictObject *)od)->od_last)
521#define _odict_EMPTY(od) (_odict_FIRST(od) == NULL)
522#define _odict_FOREACH(od, node) \
523 for (node = _odict_FIRST(od); node != NULL; node = _odictnode_NEXT(node))
524
Eric Snow8c7f9552015-08-07 17:45:12 -0600525#define _odict_FAST_SIZE(od) ((PyDictObject *)od)->ma_keys->dk_size
Eric Snow96c6af92015-05-29 22:21:39 -0600526
Eric Snow96c6af92015-05-29 22:21:39 -0600527static void
528_odict_free_fast_nodes(PyODictObject *od) {
529 if (od->od_fast_nodes) {
530 PyMem_FREE(od->od_fast_nodes);
531 }
532}
533
Eric Snow4c729182015-06-03 10:50:37 -0600534/* Return the index into the hash table, regardless of a valid node. */
535static Py_ssize_t
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200536_odict_get_index_raw(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow4c729182015-06-03 10:50:37 -0600537{
538 PyObject **value_addr = NULL;
539 PyDictKeyEntry *ep;
540 PyDictKeysObject *keys = ((PyDictObject *)od)->ma_keys;
541
542 ep = (keys->dk_lookup)((PyDictObject *)od, key, hash, &value_addr);
543 if (ep == NULL)
544 return -1;
545 /* We use pointer arithmetic to get the entry's index into the table. */
546 return ep - keys->dk_entries;
547}
548
549/* Replace od->od_fast_nodes with a new table matching the size of dict's. */
Eric Snow96c6af92015-05-29 22:21:39 -0600550static int
551_odict_resize(PyODictObject *od) {
Eric Snow4c729182015-06-03 10:50:37 -0600552 Py_ssize_t size, i;
Eric Snow96c6af92015-05-29 22:21:39 -0600553 _ODictNode **fast_nodes, *node;
554
555 /* Initialize a new "fast nodes" table. */
556 size = ((PyDictObject *)od)->ma_keys->dk_size;
557 fast_nodes = PyMem_NEW(_ODictNode *, size);
558 if (fast_nodes == NULL) {
559 PyErr_NoMemory();
560 return -1;
561 }
562 for (i = 0; i < size; i++)
563 fast_nodes[i] = NULL;
564
565 /* Copy the current nodes into the table. */
Eric Snow96c6af92015-05-29 22:21:39 -0600566 _odict_FOREACH(od, node) {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200567 i = _odict_get_index_raw(od, _odictnode_KEY(node),
Eric Snow4c729182015-06-03 10:50:37 -0600568 _odictnode_HASH(node));
Eric Snow96c6af92015-05-29 22:21:39 -0600569 if (i < 0) {
Eric Snow96c6af92015-05-29 22:21:39 -0600570 PyMem_FREE(fast_nodes);
571 return -1;
572 }
573 fast_nodes[i] = node;
574 }
Eric Snow96c6af92015-05-29 22:21:39 -0600575
576 /* Replace the old fast nodes table. */
577 _odict_free_fast_nodes(od);
578 od->od_fast_nodes = fast_nodes;
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200579 od->od_fast_nodes_size = size;
580 od->od_resize_sentinel = ((PyDictObject *)od)->ma_keys;
Eric Snow96c6af92015-05-29 22:21:39 -0600581 return 0;
582}
583
584/* Return the index into the hash table, regardless of a valid node. */
585static Py_ssize_t
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200586_odict_get_index(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600587{
Eric Snow4c729182015-06-03 10:50:37 -0600588 PyDictKeysObject *keys;
Eric Snow96c6af92015-05-29 22:21:39 -0600589
590 assert(key != NULL);
Eric Snow4c729182015-06-03 10:50:37 -0600591 keys = ((PyDictObject *)od)->ma_keys;
592
593 /* Ensure od_fast_nodes and dk_entries are in sync. */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200594 if (od->od_resize_sentinel != keys ||
595 od->od_fast_nodes_size != keys->dk_size) {
Eric Snow4c729182015-06-03 10:50:37 -0600596 int resize_res = _odict_resize(od);
597 if (resize_res < 0)
598 return -1;
599 }
600
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200601 return _odict_get_index_raw(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600602}
603
Eric Snow96c6af92015-05-29 22:21:39 -0600604/* Returns NULL if there was some error or the key was not found. */
605static _ODictNode *
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200606_odict_find_node_hash(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600607{
608 Py_ssize_t index;
609
610 if (_odict_EMPTY(od))
611 return NULL;
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200612 index = _odict_get_index(od, key, hash);
613 if (index < 0)
614 return NULL;
615 return od->od_fast_nodes[index];
616}
617
618static _ODictNode *
619_odict_find_node(PyODictObject *od, PyObject *key)
620{
621 Py_ssize_t index;
622 Py_hash_t hash;
623
624 if (_odict_EMPTY(od))
625 return NULL;
626 hash = PyObject_Hash(key);
627 if (hash == -1)
628 return NULL;
629 index = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600630 if (index < 0)
631 return NULL;
632 return od->od_fast_nodes[index];
633}
634
635static void
636_odict_add_head(PyODictObject *od, _ODictNode *node)
637{
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300638 _odictnode_PREV(node) = NULL;
639 _odictnode_NEXT(node) = _odict_FIRST(od);
640 if (_odict_FIRST(od) == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -0600641 _odict_LAST(od) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300642 else
Eric Snow96c6af92015-05-29 22:21:39 -0600643 _odictnode_PREV(_odict_FIRST(od)) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300644 _odict_FIRST(od) = node;
Eric Snow4fabf022015-06-04 00:09:56 -0600645 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600646}
647
648static void
649_odict_add_tail(PyODictObject *od, _ODictNode *node)
650{
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300651 _odictnode_PREV(node) = _odict_LAST(od);
652 _odictnode_NEXT(node) = NULL;
653 if (_odict_LAST(od) == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -0600654 _odict_FIRST(od) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300655 else
Eric Snow96c6af92015-05-29 22:21:39 -0600656 _odictnode_NEXT(_odict_LAST(od)) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300657 _odict_LAST(od) = node;
Eric Snow4fabf022015-06-04 00:09:56 -0600658 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600659}
660
661/* adds the node to the end of the list */
662static int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200663_odict_add_new_node(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600664{
665 Py_ssize_t i;
666 _ODictNode *node;
667
Serhiy Storchakad17427b2015-10-20 18:21:48 +0300668 Py_INCREF(key);
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200669 i = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600670 if (i < 0) {
671 if (!PyErr_Occurred())
672 PyErr_SetObject(PyExc_KeyError, key);
673 Py_DECREF(key);
674 return -1;
675 }
676 else if (od->od_fast_nodes[i] != NULL) {
677 /* We already have a node for the key so there's no need to add one. */
678 Py_DECREF(key);
679 return 0;
680 }
681
682 /* must not be added yet */
683 node = (_ODictNode *)PyMem_MALLOC(sizeof(_ODictNode));
684 if (node == NULL) {
685 Py_DECREF(key);
686 PyErr_NoMemory();
687 return -1;
688 }
689
690 _odictnode_KEY(node) = key;
Eric Snow4c729182015-06-03 10:50:37 -0600691 _odictnode_HASH(node) = hash;
Eric Snow96c6af92015-05-29 22:21:39 -0600692 _odict_add_tail(od, node);
693 od->od_fast_nodes[i] = node;
694 return 0;
695}
696
697/* Putting the decref after the free causes problems. */
698#define _odictnode_DEALLOC(node) \
699 do { \
700 Py_DECREF(_odictnode_KEY(node)); \
701 PyMem_FREE((void *)node); \
702 } while (0)
703
704/* Repeated calls on the same node are no-ops. */
705static void
706_odict_remove_node(PyODictObject *od, _ODictNode *node)
707{
708 if (_odict_FIRST(od) == node)
709 _odict_FIRST(od) = _odictnode_NEXT(node);
710 else if (_odictnode_PREV(node) != NULL)
711 _odictnode_NEXT(_odictnode_PREV(node)) = _odictnode_NEXT(node);
712
713 if (_odict_LAST(od) == node)
714 _odict_LAST(od) = _odictnode_PREV(node);
715 else if (_odictnode_NEXT(node) != NULL)
716 _odictnode_PREV(_odictnode_NEXT(node)) = _odictnode_PREV(node);
717
718 _odictnode_PREV(node) = NULL;
719 _odictnode_NEXT(node) = NULL;
Eric Snow4fabf022015-06-04 00:09:56 -0600720 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600721}
722
Eric Snow96c6af92015-05-29 22:21:39 -0600723/* If someone calls PyDict_DelItem() directly on an OrderedDict, we'll
724 get all sorts of problems here. In PyODict_DelItem we make sure to
725 call _odict_clear_node first.
Victor Stinnerca30b022015-09-03 17:50:04 +0200726
Eric Snow96c6af92015-05-29 22:21:39 -0600727 This matters in the case of colliding keys. Suppose we add 3 keys:
728 [A, B, C], where the hash of C collides with A and the next possible
729 index in the hash table is occupied by B. If we remove B then for C
730 the dict's looknode func will give us the old index of B instead of
731 the index we got before deleting B. However, the node for C in
732 od_fast_nodes is still at the old dict index of C. Thus to be sure
733 things don't get out of sync, we clear the node in od_fast_nodes
734 *before* calling PyDict_DelItem.
735
736 The same must be done for any other OrderedDict operations where
737 we modify od_fast_nodes.
738*/
739static int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200740_odict_clear_node(PyODictObject *od, _ODictNode *node, PyObject *key,
741 Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600742{
743 Py_ssize_t i;
744
745 assert(key != NULL);
746 if (_odict_EMPTY(od)) {
747 /* Let later code decide if this is a KeyError. */
748 return 0;
749 }
750
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200751 i = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600752 if (i < 0)
753 return PyErr_Occurred() ? -1 : 0;
754
755 if (node == NULL)
756 node = od->od_fast_nodes[i];
757 assert(node == od->od_fast_nodes[i]);
758 if (node == NULL) {
759 /* Let later code decide if this is a KeyError. */
760 return 0;
761 }
762
763 // Now clear the node.
764 od->od_fast_nodes[i] = NULL;
765 _odict_remove_node(od, node);
766 _odictnode_DEALLOC(node);
767 return 0;
768}
769
770static void
771_odict_clear_nodes(PyODictObject *od)
772{
773 _ODictNode *node, *next;
774
Eric Snow96c6af92015-05-29 22:21:39 -0600775 _odict_free_fast_nodes(od);
776 od->od_fast_nodes = NULL;
Serhiy Storchakad205d012016-01-19 14:46:25 +0200777
778 node = _odict_FIRST(od);
779 _odict_FIRST(od) = NULL;
780 _odict_LAST(od) = NULL;
781 while (node != NULL) {
782 next = _odictnode_NEXT(node);
783 _odictnode_DEALLOC(node);
784 node = next;
785 }
Eric Snow96c6af92015-05-29 22:21:39 -0600786}
787
788/* There isn't any memory management of nodes past this point. */
789#undef _odictnode_DEALLOC
790
791static int
792_odict_keys_equal(PyODictObject *a, PyODictObject *b)
793{
794 _ODictNode *node_a, *node_b;
795
796 node_a = _odict_FIRST(a);
797 node_b = _odict_FIRST(b);
798 while (1) {
799 if (node_a == NULL && node_b == NULL)
800 /* success: hit the end of each at the same time */
801 return 1;
802 else if (node_a == NULL || node_b == NULL)
803 /* unequal length */
804 return 0;
805 else {
806 int res = PyObject_RichCompareBool(
807 (PyObject *)_odictnode_KEY(node_a),
808 (PyObject *)_odictnode_KEY(node_b),
809 Py_EQ);
810 if (res < 0)
811 return res;
812 else if (res == 0)
813 return 0;
814
815 /* otherwise it must match, so move on to the next one */
816 node_a = _odictnode_NEXT(node_a);
817 node_b = _odictnode_NEXT(node_b);
818 }
819 }
820}
821
822
823/* ----------------------------------------------
824 * OrderedDict mapping methods
825 */
826
827/* mp_ass_subscript: __setitem__() and __delitem__() */
828
829static int
830odict_mp_ass_sub(PyODictObject *od, PyObject *v, PyObject *w)
831{
832 if (w == NULL)
833 return PyODict_DelItem((PyObject *)od, v);
834 else
835 return PyODict_SetItem((PyObject *)od, v, w);
836}
837
838/* tp_as_mapping */
839
840static PyMappingMethods odict_as_mapping = {
841 0, /*mp_length*/
842 0, /*mp_subscript*/
843 (objobjargproc)odict_mp_ass_sub, /*mp_ass_subscript*/
844};
845
846
847/* ----------------------------------------------
848 * OrderedDict methods
849 */
850
851/* __delitem__() */
852
853PyDoc_STRVAR(odict_delitem__doc__, "od.__delitem__(y) <==> del od[y]");
854
855/* __eq__() */
856
857PyDoc_STRVAR(odict_eq__doc__,
858"od.__eq__(y) <==> od==y. Comparison to another OD is order-sensitive \n\
859 while comparison to a regular mapping is order-insensitive.\n\
860 ");
861
862/* forward */
863static PyObject * odict_richcompare(PyObject *v, PyObject *w, int op);
864
865static PyObject *
866odict_eq(PyObject *a, PyObject *b)
867{
868 return odict_richcompare(a, b, Py_EQ);
869}
870
871/* __init__() */
872
873PyDoc_STRVAR(odict_init__doc__,
874"Initialize an ordered dictionary. The signature is the same as\n\
875 regular dictionaries, but keyword arguments are not recommended because\n\
876 their insertion order is arbitrary.\n\
877\n\
878 ");
879
880/* forward */
881static int odict_init(PyObject *self, PyObject *args, PyObject *kwds);
882
883/* __iter__() */
884
885PyDoc_STRVAR(odict_iter__doc__, "od.__iter__() <==> iter(od)");
886
887static PyObject * odict_iter(PyODictObject *self); /* forward */
888
889/* __ne__() */
890
891/* Mapping.__ne__() does not have a docstring. */
892PyDoc_STRVAR(odict_ne__doc__, "");
893
894static PyObject *
895odict_ne(PyObject *a, PyObject *b)
896{
897 return odict_richcompare(a, b, Py_NE);
898}
899
900/* __repr__() */
901
902PyDoc_STRVAR(odict_repr__doc__, "od.__repr__() <==> repr(od)");
903
904static PyObject * odict_repr(PyODictObject *self); /* forward */
905
906/* __setitem__() */
907
908PyDoc_STRVAR(odict_setitem__doc__, "od.__setitem__(i, y) <==> od[i]=y");
909
910/* fromkeys() */
911
912PyDoc_STRVAR(odict_fromkeys__doc__,
913"OD.fromkeys(S[, v]) -> New ordered dictionary with keys from S.\n\
914 If not specified, the value defaults to None.\n\
915\n\
916 ");
917
918static PyObject *
Eric Snowac02ef32015-06-02 20:42:14 -0600919odict_fromkeys(PyObject *cls, PyObject *args, PyObject *kwargs)
Eric Snow96c6af92015-05-29 22:21:39 -0600920{
Eric Snowac02ef32015-06-02 20:42:14 -0600921 static char *kwlist[] = {"iterable", "value", 0};
Eric Snow96c6af92015-05-29 22:21:39 -0600922 PyObject *seq;
923 PyObject *value = Py_None;
Eric Snowac02ef32015-06-02 20:42:14 -0600924
925 /* both borrowed */
926 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:fromkeys", kwlist,
927 &seq, &value)) {
Eric Snow96c6af92015-05-29 22:21:39 -0600928 return NULL;
Eric Snowac02ef32015-06-02 20:42:14 -0600929 }
Eric Snow96c6af92015-05-29 22:21:39 -0600930 return _PyDict_FromKeys(cls, seq, value);
931}
932
933/* __sizeof__() */
934
935/* OrderedDict.__sizeof__() does not have a docstring. */
936PyDoc_STRVAR(odict_sizeof__doc__, "");
937
938static PyObject *
939odict_sizeof(PyODictObject *od)
940{
Serhiy Storchaka0ce7a3a2015-12-22 08:16:18 +0200941 Py_ssize_t res = _PyDict_SizeOf((PyDictObject *)od);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300942 res += sizeof(_ODictNode *) * _odict_FAST_SIZE(od); /* od_fast_nodes */
Eric Snow96c6af92015-05-29 22:21:39 -0600943 if (!_odict_EMPTY(od)) {
944 res += sizeof(_ODictNode) * PyODict_SIZE(od); /* linked-list */
945 }
946 return PyLong_FromSsize_t(res);
947}
948
949/* __reduce__() */
950
951PyDoc_STRVAR(odict_reduce__doc__, "Return state information for pickling");
952
953static PyObject *
954odict_reduce(register PyODictObject *od)
955{
956 _Py_IDENTIFIER(__dict__);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300957 _Py_IDENTIFIER(items);
Serhiy Storchaka4575beb2015-10-22 20:18:24 +0300958 PyObject *dict = NULL, *result = NULL;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300959 PyObject *items_iter, *items, *args = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -0600960
961 /* capture any instance state */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300962 dict = _PyObject_GetAttrId((PyObject *)od, &PyId___dict__);
963 if (dict == NULL)
Eric Snowc5e59602015-05-30 11:28:56 -0600964 goto Done;
965 else {
Eric Snow96c6af92015-05-29 22:21:39 -0600966 /* od.__dict__ isn't necessarily a dict... */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300967 Py_ssize_t dict_len = PyObject_Length(dict);
968 if (dict_len == -1)
Eric Snow96c6af92015-05-29 22:21:39 -0600969 goto Done;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300970 if (!dict_len) {
971 /* nothing to pickle in od.__dict__ */
972 Py_CLEAR(dict);
Eric Snow96c6af92015-05-29 22:21:39 -0600973 }
974 }
975
976 /* build the result */
Eric Snow96c6af92015-05-29 22:21:39 -0600977 args = PyTuple_New(0);
978 if (args == NULL)
979 goto Done;
980
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300981 items = _PyObject_CallMethodIdObjArgs((PyObject *)od, &PyId_items, NULL);
Eric Snow96c6af92015-05-29 22:21:39 -0600982 if (items == NULL)
983 goto Done;
984
985 items_iter = PyObject_GetIter(items);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300986 Py_DECREF(items);
Eric Snow96c6af92015-05-29 22:21:39 -0600987 if (items_iter == NULL)
988 goto Done;
989
Serhiy Storchaka4575beb2015-10-22 20:18:24 +0300990 result = PyTuple_Pack(5, Py_TYPE(od), args, dict ? dict : Py_None, Py_None, items_iter);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300991 Py_DECREF(items_iter);
Eric Snow96c6af92015-05-29 22:21:39 -0600992
993Done:
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300994 Py_XDECREF(dict);
Eric Snow96c6af92015-05-29 22:21:39 -0600995 Py_XDECREF(args);
Eric Snow96c6af92015-05-29 22:21:39 -0600996
997 return result;
998}
999
1000/* setdefault() */
1001
1002PyDoc_STRVAR(odict_setdefault__doc__,
1003 "od.setdefault(k[,d]) -> od.get(k,d), also set od[k]=d if k not in od");
1004
1005/* Skips __missing__() calls. */
1006static PyObject *
Eric Snowac02ef32015-06-02 20:42:14 -06001007odict_setdefault(register PyODictObject *od, PyObject *args, PyObject *kwargs)
Eric Snow96c6af92015-05-29 22:21:39 -06001008{
Eric Snowac02ef32015-06-02 20:42:14 -06001009 static char *kwlist[] = {"key", "default", 0};
Eric Snow96c6af92015-05-29 22:21:39 -06001010 PyObject *key, *result = NULL;
1011 PyObject *failobj = Py_None;
1012
1013 /* both borrowed */
Eric Snowac02ef32015-06-02 20:42:14 -06001014 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:setdefault", kwlist,
1015 &key, &failobj)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001016 return NULL;
Eric Snowac02ef32015-06-02 20:42:14 -06001017 }
Eric Snow96c6af92015-05-29 22:21:39 -06001018
1019 if (PyODict_CheckExact(od)) {
Eric Snowd1719752015-06-01 23:12:13 -06001020 result = PyODict_GetItemWithError(od, key); /* borrowed */
1021 if (result == NULL) {
1022 if (PyErr_Occurred())
1023 return NULL;
1024 assert(_odict_find_node(od, key) == NULL);
1025 if (PyODict_SetItem((PyObject *)od, key, failobj) >= 0) {
Eric Snow96c6af92015-05-29 22:21:39 -06001026 result = failobj;
1027 Py_INCREF(failobj);
1028 }
1029 }
1030 else {
Eric Snowd1719752015-06-01 23:12:13 -06001031 Py_INCREF(result);
Eric Snow96c6af92015-05-29 22:21:39 -06001032 }
1033 }
1034 else {
1035 int exists = PySequence_Contains((PyObject *)od, key);
1036 if (exists < 0) {
Eric Snowd1719752015-06-01 23:12:13 -06001037 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001038 }
1039 else if (exists) {
1040 result = PyObject_GetItem((PyObject *)od, key);
1041 }
1042 else if (PyObject_SetItem((PyObject *)od, key, failobj) >= 0) {
1043 result = failobj;
1044 Py_INCREF(failobj);
1045 }
1046 }
1047
Eric Snow96c6af92015-05-29 22:21:39 -06001048 return result;
1049}
1050
1051/* pop() */
1052
1053PyDoc_STRVAR(odict_pop__doc__,
1054"od.pop(k[,d]) -> v, remove specified key and return the corresponding\n\
1055 value. If key is not found, d is returned if given, otherwise KeyError\n\
1056 is raised.\n\
1057\n\
1058 ");
1059
1060/* forward */
1061static PyObject * _odict_popkey(PyObject *, PyObject *, PyObject *);
1062
1063/* Skips __missing__() calls. */
1064static PyObject *
Eric Snowac02ef32015-06-02 20:42:14 -06001065odict_pop(PyObject *od, PyObject *args, PyObject *kwargs)
Eric Snow96c6af92015-05-29 22:21:39 -06001066{
Eric Snowac02ef32015-06-02 20:42:14 -06001067 static char *kwlist[] = {"key", "default", 0};
Eric Snow96c6af92015-05-29 22:21:39 -06001068 PyObject *key, *failobj = NULL;
1069
Eric Snowac02ef32015-06-02 20:42:14 -06001070 /* borrowed */
1071 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:pop", kwlist,
1072 &key, &failobj)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001073 return NULL;
1074 }
1075
1076 return _odict_popkey(od, key, failobj);
1077}
1078
1079static PyObject *
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001080_odict_popkey_hash(PyObject *od, PyObject *key, PyObject *failobj,
1081 Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -06001082{
1083 _ODictNode *node;
1084 PyObject *value = NULL;
1085
Eric Snow96c6af92015-05-29 22:21:39 -06001086 /* Pop the node first to avoid a possible dict resize (due to
1087 eval loop reentrancy) and complications due to hash collision
1088 resolution. */
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001089 node = _odict_find_node_hash((PyODictObject *)od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001090 if (node == NULL) {
1091 if (PyErr_Occurred())
Eric Snowd1719752015-06-01 23:12:13 -06001092 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001093 }
1094 else {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001095 int res = _odict_clear_node((PyODictObject *)od, node, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001096 if (res < 0) {
Eric Snowd1719752015-06-01 23:12:13 -06001097 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001098 }
1099 }
1100
1101 /* Now delete the value from the dict. */
Serhiy Storchaka48325802016-10-25 15:33:23 +03001102 if (node != NULL) {
1103 value = _PyDict_GetItem_KnownHash(od, key, hash); /* borrowed */
1104 if (value != NULL) {
1105 Py_INCREF(value);
1106 if (_PyDict_DelItem_KnownHash(od, key, hash) < 0) {
1107 Py_DECREF(value);
1108 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001109 }
1110 }
1111 }
1112
1113 /* Apply the fallback value, if necessary. */
1114 if (value == NULL && !PyErr_Occurred()) {
1115 if (failobj) {
1116 value = failobj;
1117 Py_INCREF(failobj);
1118 }
1119 else {
1120 PyErr_SetObject(PyExc_KeyError, key);
1121 }
1122 }
1123
Eric Snow96c6af92015-05-29 22:21:39 -06001124 return value;
1125}
1126
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001127static PyObject *
1128_odict_popkey(PyObject *od, PyObject *key, PyObject *failobj)
1129{
1130 Py_hash_t hash = PyObject_Hash(key);
1131 if (hash == -1)
1132 return NULL;
1133
1134 return _odict_popkey_hash(od, key, failobj, hash);
1135}
1136
Eric Snow96c6af92015-05-29 22:21:39 -06001137/* popitem() */
1138
1139PyDoc_STRVAR(odict_popitem__doc__,
1140"od.popitem() -> (k, v), return and remove a (key, value) pair.\n\
1141 Pairs are returned in LIFO order if last is true or FIFO order if false.\n\
1142\n\
1143 ");
1144
1145static PyObject *
Eric Snowac02ef32015-06-02 20:42:14 -06001146odict_popitem(PyObject *od, PyObject *args, PyObject *kwargs)
Eric Snow96c6af92015-05-29 22:21:39 -06001147{
Eric Snowac02ef32015-06-02 20:42:14 -06001148 static char *kwlist[] = {"last", 0};
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001149 PyObject *key, *value, *item = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001150 _ODictNode *node;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001151 int last = 1;
Eric Snow96c6af92015-05-29 22:21:39 -06001152
1153 /* pull the item */
1154
Eric Snowac02ef32015-06-02 20:42:14 -06001155 /* borrowed */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001156 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|p:popitem", kwlist,
Eric Snowac02ef32015-06-02 20:42:14 -06001157 &last)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001158 return NULL;
Eric Snowac02ef32015-06-02 20:42:14 -06001159 }
Eric Snow96c6af92015-05-29 22:21:39 -06001160
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001161 if (_odict_EMPTY(od)) {
1162 PyErr_SetString(PyExc_KeyError, "dictionary is empty");
1163 return NULL;
Eric Snowac02ef32015-06-02 20:42:14 -06001164 }
Eric Snow96c6af92015-05-29 22:21:39 -06001165
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001166 node = last ? _odict_LAST(od) : _odict_FIRST(od);
Eric Snow96c6af92015-05-29 22:21:39 -06001167 key = _odictnode_KEY(node);
Eric Snowd1719752015-06-01 23:12:13 -06001168 Py_INCREF(key);
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001169 value = _odict_popkey_hash(od, key, NULL, _odictnode_HASH(node));
Eric Snow96c6af92015-05-29 22:21:39 -06001170 if (value == NULL)
1171 return NULL;
1172 item = PyTuple_Pack(2, key, value);
Eric Snowd1719752015-06-01 23:12:13 -06001173 Py_DECREF(key);
Eric Snow96c6af92015-05-29 22:21:39 -06001174 Py_DECREF(value);
1175 return item;
1176}
1177
1178/* keys() */
1179
1180/* MutableMapping.keys() does not have a docstring. */
1181PyDoc_STRVAR(odict_keys__doc__, "");
1182
1183static PyObject * odictkeys_new(PyObject *od); /* forward */
1184
1185/* values() */
1186
1187/* MutableMapping.values() does not have a docstring. */
1188PyDoc_STRVAR(odict_values__doc__, "");
1189
1190static PyObject * odictvalues_new(PyObject *od); /* forward */
1191
1192/* items() */
1193
1194/* MutableMapping.items() does not have a docstring. */
1195PyDoc_STRVAR(odict_items__doc__, "");
1196
1197static PyObject * odictitems_new(PyObject *od); /* forward */
1198
1199/* update() */
1200
1201/* MutableMapping.update() does not have a docstring. */
1202PyDoc_STRVAR(odict_update__doc__, "");
1203
1204/* forward */
1205static PyObject * mutablemapping_update(PyObject *, PyObject *, PyObject *);
1206
1207#define odict_update mutablemapping_update
1208
1209/* clear() */
1210
1211PyDoc_STRVAR(odict_clear__doc__,
1212 "od.clear() -> None. Remove all items from od.");
1213
1214static PyObject *
1215odict_clear(register PyODictObject *od)
1216{
1217 PyDict_Clear((PyObject *)od);
1218 _odict_clear_nodes(od);
Eric Snow96c6af92015-05-29 22:21:39 -06001219 if (_odict_resize(od) < 0)
1220 return NULL;
1221 Py_RETURN_NONE;
1222}
1223
1224/* copy() */
1225
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001226/* forward */
1227static int _PyODict_SetItem_KnownHash(PyObject *, PyObject *, PyObject *,
1228 Py_hash_t);
1229
Eric Snow96c6af92015-05-29 22:21:39 -06001230PyDoc_STRVAR(odict_copy__doc__, "od.copy() -> a shallow copy of od");
1231
1232static PyObject *
1233odict_copy(register PyODictObject *od)
1234{
1235 _ODictNode *node;
1236 PyObject *od_copy;
1237
1238 if (PyODict_CheckExact(od))
1239 od_copy = PyODict_New();
1240 else
1241 od_copy = PyObject_CallFunctionObjArgs((PyObject *)Py_TYPE(od), NULL);
1242 if (od_copy == NULL)
1243 return NULL;
1244
1245 if (PyODict_CheckExact(od)) {
1246 _odict_FOREACH(od, node) {
Eric Snowa762af72015-06-01 22:59:08 -06001247 PyObject *key = _odictnode_KEY(node);
1248 PyObject *value = _odictnode_VALUE(node, od);
1249 if (value == NULL) {
1250 if (!PyErr_Occurred())
1251 PyErr_SetObject(PyExc_KeyError, key);
1252 goto fail;
1253 }
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001254 if (_PyODict_SetItem_KnownHash((PyObject *)od_copy, key, value,
1255 _odictnode_HASH(node)) != 0)
Eric Snow96c6af92015-05-29 22:21:39 -06001256 goto fail;
1257 }
1258 }
1259 else {
1260 _odict_FOREACH(od, node) {
1261 int res;
1262 PyObject *value = PyObject_GetItem((PyObject *)od,
1263 _odictnode_KEY(node));
1264 if (value == NULL)
1265 goto fail;
1266 res = PyObject_SetItem((PyObject *)od_copy,
1267 _odictnode_KEY(node), value);
1268 Py_DECREF(value);
1269 if (res != 0)
1270 goto fail;
1271 }
1272 }
1273 return od_copy;
1274
1275fail:
1276 Py_DECREF(od_copy);
1277 return NULL;
1278}
1279
1280/* __reversed__() */
1281
1282PyDoc_STRVAR(odict_reversed__doc__, "od.__reversed__() <==> reversed(od)");
1283
1284#define _odict_ITER_REVERSED 1
1285#define _odict_ITER_KEYS 2
1286#define _odict_ITER_VALUES 4
1287
1288/* forward */
1289static PyObject * odictiter_new(PyODictObject *, int);
1290
1291static PyObject *
1292odict_reversed(PyODictObject *od)
1293{
Eric Snow96c6af92015-05-29 22:21:39 -06001294 return odictiter_new(od, _odict_ITER_KEYS|_odict_ITER_REVERSED);
1295}
1296
1297/* move_to_end() */
1298
1299PyDoc_STRVAR(odict_move_to_end__doc__,
1300"Move an existing element to the end (or beginning if last==False).\n\
1301\n\
1302 Raises KeyError if the element does not exist.\n\
1303 When last=True, acts like a fast version of self[key]=self.pop(key).\n\
1304\n\
1305 ");
1306
1307static PyObject *
Eric Snowac02ef32015-06-02 20:42:14 -06001308odict_move_to_end(PyODictObject *od, PyObject *args, PyObject *kwargs)
Eric Snow96c6af92015-05-29 22:21:39 -06001309{
Eric Snowac02ef32015-06-02 20:42:14 -06001310 static char *kwlist[] = {"key", "last", 0};
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001311 PyObject *key;
1312 int last = 1;
1313 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001314
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001315 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|p:move_to_end", kwlist,
Eric Snowac02ef32015-06-02 20:42:14 -06001316 &key, &last)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001317 return NULL;
Eric Snowac02ef32015-06-02 20:42:14 -06001318 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001319
Eric Snow96c6af92015-05-29 22:21:39 -06001320 if (_odict_EMPTY(od)) {
1321 PyErr_SetObject(PyExc_KeyError, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001322 return NULL;
1323 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001324 node = last ? _odict_LAST(od) : _odict_FIRST(od);
1325 if (key != _odictnode_KEY(node)) {
1326 node = _odict_find_node(od, key);
1327 if (node == NULL) {
1328 if (!PyErr_Occurred())
1329 PyErr_SetObject(PyExc_KeyError, key);
Eric Snowc5e59602015-05-30 11:28:56 -06001330 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001331 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001332 if (last) {
1333 /* Only move if not already the last one. */
1334 if (node != _odict_LAST(od)) {
1335 _odict_remove_node(od, node);
Eric Snow96c6af92015-05-29 22:21:39 -06001336 _odict_add_tail(od, node);
1337 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001338 }
1339 else {
1340 /* Only move if not already the first one. */
1341 if (node != _odict_FIRST(od)) {
1342 _odict_remove_node(od, node);
1343 _odict_add_head(od, node);
Eric Snow96c6af92015-05-29 22:21:39 -06001344 }
1345 }
1346 }
Eric Snow96c6af92015-05-29 22:21:39 -06001347 Py_RETURN_NONE;
1348}
1349
1350
1351/* tp_methods */
1352
1353static PyMethodDef odict_methods[] = {
1354
1355 /* explicitly defined so we can align docstrings with
1356 * collections.OrderedDict */
1357 {"__delitem__", (PyCFunction)odict_mp_ass_sub, METH_NOARGS,
1358 odict_delitem__doc__},
1359 {"__eq__", (PyCFunction)odict_eq, METH_NOARGS,
1360 odict_eq__doc__},
1361 {"__init__", (PyCFunction)odict_init, METH_NOARGS,
1362 odict_init__doc__},
1363 {"__iter__", (PyCFunction)odict_iter, METH_NOARGS,
1364 odict_iter__doc__},
1365 {"__ne__", (PyCFunction)odict_ne, METH_NOARGS,
1366 odict_ne__doc__},
1367 {"__repr__", (PyCFunction)odict_repr, METH_NOARGS,
1368 odict_repr__doc__},
1369 {"__setitem__", (PyCFunction)odict_mp_ass_sub, METH_NOARGS,
1370 odict_setitem__doc__},
Eric Snowac02ef32015-06-02 20:42:14 -06001371 {"fromkeys", (PyCFunction)odict_fromkeys,
1372 METH_VARARGS | METH_KEYWORDS | METH_CLASS, odict_fromkeys__doc__},
Eric Snow96c6af92015-05-29 22:21:39 -06001373
1374 /* overridden dict methods */
1375 {"__sizeof__", (PyCFunction)odict_sizeof, METH_NOARGS,
1376 odict_sizeof__doc__},
1377 {"__reduce__", (PyCFunction)odict_reduce, METH_NOARGS,
1378 odict_reduce__doc__},
Eric Snowac02ef32015-06-02 20:42:14 -06001379 {"setdefault", (PyCFunction)odict_setdefault,
1380 METH_VARARGS | METH_KEYWORDS, odict_setdefault__doc__},
1381 {"pop", (PyCFunction)odict_pop,
1382 METH_VARARGS | METH_KEYWORDS, odict_pop__doc__},
1383 {"popitem", (PyCFunction)odict_popitem,
1384 METH_VARARGS | METH_KEYWORDS, odict_popitem__doc__},
Eric Snow96c6af92015-05-29 22:21:39 -06001385 {"keys", (PyCFunction)odictkeys_new, METH_NOARGS,
1386 odict_keys__doc__},
1387 {"values", (PyCFunction)odictvalues_new, METH_NOARGS,
1388 odict_values__doc__},
1389 {"items", (PyCFunction)odictitems_new, METH_NOARGS,
1390 odict_items__doc__},
1391 {"update", (PyCFunction)odict_update, METH_VARARGS | METH_KEYWORDS,
1392 odict_update__doc__},
1393 {"clear", (PyCFunction)odict_clear, METH_NOARGS,
1394 odict_clear__doc__},
1395 {"copy", (PyCFunction)odict_copy, METH_NOARGS,
1396 odict_copy__doc__},
1397
1398 /* new methods */
1399 {"__reversed__", (PyCFunction)odict_reversed, METH_NOARGS,
1400 odict_reversed__doc__},
Eric Snowac02ef32015-06-02 20:42:14 -06001401 {"move_to_end", (PyCFunction)odict_move_to_end,
1402 METH_VARARGS | METH_KEYWORDS, odict_move_to_end__doc__},
Eric Snow96c6af92015-05-29 22:21:39 -06001403
1404 {NULL, NULL} /* sentinel */
1405};
1406
1407
1408/* ----------------------------------------------
1409 * OrderedDict members
1410 */
1411
1412/* tp_members */
1413
1414static PyMemberDef odict_members[] = {
1415 {"__dict__", T_OBJECT, offsetof(PyODictObject, od_inst_dict), READONLY},
1416 {0}
1417};
1418
1419
1420/* ----------------------------------------------
1421 * OrderedDict type slot methods
1422 */
1423
1424/* tp_dealloc */
1425
1426static void
1427odict_dealloc(PyODictObject *self)
1428{
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001429 PyThreadState *tstate = PyThreadState_GET();
1430
Eric Snow96c6af92015-05-29 22:21:39 -06001431 PyObject_GC_UnTrack(self);
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001432 Py_TRASHCAN_SAFE_BEGIN(self)
1433
Eric Snow96c6af92015-05-29 22:21:39 -06001434 Py_XDECREF(self->od_inst_dict);
1435 if (self->od_weakreflist != NULL)
1436 PyObject_ClearWeakRefs((PyObject *)self);
1437
1438 _odict_clear_nodes(self);
Eric Snow96c6af92015-05-29 22:21:39 -06001439
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001440 /* Call the base tp_dealloc(). Since it too uses the trashcan mechanism,
1441 * temporarily decrement trash_delete_nesting to prevent triggering it
1442 * and putting the partially deallocated object on the trashcan's
1443 * to-be-deleted-later list.
1444 */
1445 --tstate->trash_delete_nesting;
1446 assert(_tstate->trash_delete_nesting < PyTrash_UNWIND_LEVEL);
Eric Snow96c6af92015-05-29 22:21:39 -06001447 PyDict_Type.tp_dealloc((PyObject *)self);
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001448 ++tstate->trash_delete_nesting;
1449
1450 Py_TRASHCAN_SAFE_END(self)
Eric Snow96c6af92015-05-29 22:21:39 -06001451};
1452
1453/* tp_repr */
1454
1455static PyObject *
1456odict_repr(PyODictObject *self)
1457{
1458 int i;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001459 _Py_IDENTIFIER(items);
Serhiy Storchaka4575beb2015-10-22 20:18:24 +03001460 PyObject *pieces = NULL, *result = NULL;
1461 const char *classname;
1462
1463 classname = strrchr(Py_TYPE(self)->tp_name, '.');
1464 if (classname == NULL)
1465 classname = Py_TYPE(self)->tp_name;
1466 else
1467 classname++;
1468
1469 if (PyODict_SIZE(self) == 0)
1470 return PyUnicode_FromFormat("%s()", classname);
Eric Snow96c6af92015-05-29 22:21:39 -06001471
1472 i = Py_ReprEnter((PyObject *)self);
1473 if (i != 0) {
1474 return i > 0 ? PyUnicode_FromString("...") : NULL;
1475 }
1476
Eric Snow96c6af92015-05-29 22:21:39 -06001477 if (PyODict_CheckExact(self)) {
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001478 Py_ssize_t count = 0;
Eric Snowa762af72015-06-01 22:59:08 -06001479 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001480 pieces = PyList_New(PyODict_SIZE(self));
1481 if (pieces == NULL)
1482 goto Done;
1483
1484 _odict_FOREACH(self, node) {
Eric Snowa762af72015-06-01 22:59:08 -06001485 PyObject *pair;
1486 PyObject *key = _odictnode_KEY(node);
1487 PyObject *value = _odictnode_VALUE(node, self);
1488 if (value == NULL) {
1489 if (!PyErr_Occurred())
1490 PyErr_SetObject(PyExc_KeyError, key);
Eric Snowdb4061c2015-06-03 11:09:48 -06001491 goto Done;
Eric Snowa762af72015-06-01 22:59:08 -06001492 }
1493 pair = PyTuple_Pack(2, key, value);
Eric Snow96c6af92015-05-29 22:21:39 -06001494 if (pair == NULL)
1495 goto Done;
1496
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001497 if (count < PyList_GET_SIZE(pieces))
1498 PyList_SET_ITEM(pieces, count, pair); /* steals reference */
1499 else {
1500 if (PyList_Append(pieces, pair) < 0) {
1501 Py_DECREF(pair);
1502 goto Done;
1503 }
1504 Py_DECREF(pair);
1505 }
1506 count++;
Eric Snow96c6af92015-05-29 22:21:39 -06001507 }
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001508 if (count < PyList_GET_SIZE(pieces))
1509 PyList_GET_SIZE(pieces) = count;
Eric Snow96c6af92015-05-29 22:21:39 -06001510 }
1511 else {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001512 PyObject *items = _PyObject_CallMethodIdObjArgs((PyObject *)self,
1513 &PyId_items, NULL);
Eric Snow96c6af92015-05-29 22:21:39 -06001514 if (items == NULL)
1515 goto Done;
1516 pieces = PySequence_List(items);
1517 Py_DECREF(items);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001518 if (pieces == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -06001519 goto Done;
1520 }
1521
Serhiy Storchaka4575beb2015-10-22 20:18:24 +03001522 result = PyUnicode_FromFormat("%s(%R)", classname, pieces);
Eric Snow96c6af92015-05-29 22:21:39 -06001523
Eric Snow96c6af92015-05-29 22:21:39 -06001524Done:
1525 Py_XDECREF(pieces);
Eric Snow96c6af92015-05-29 22:21:39 -06001526 Py_ReprLeave((PyObject *)self);
1527 return result;
1528};
1529
1530/* tp_doc */
1531
1532PyDoc_STRVAR(odict_doc,
1533 "Dictionary that remembers insertion order");
1534
1535/* tp_traverse */
1536
1537static int
1538odict_traverse(PyODictObject *od, visitproc visit, void *arg)
1539{
Serhiy Storchakad205d012016-01-19 14:46:25 +02001540 _ODictNode *node;
1541
Eric Snow96c6af92015-05-29 22:21:39 -06001542 Py_VISIT(od->od_inst_dict);
1543 Py_VISIT(od->od_weakreflist);
Serhiy Storchakad205d012016-01-19 14:46:25 +02001544 _odict_FOREACH(od, node) {
1545 Py_VISIT(_odictnode_KEY(node));
1546 }
Eric Snow96c6af92015-05-29 22:21:39 -06001547 return PyDict_Type.tp_traverse((PyObject *)od, visit, arg);
1548}
1549
1550/* tp_clear */
1551
1552static int
1553odict_tp_clear(PyODictObject *od)
1554{
1555 PyObject *res;
1556 Py_CLEAR(od->od_inst_dict);
1557 Py_CLEAR(od->od_weakreflist);
1558 res = odict_clear(od);
1559 if (res == NULL)
1560 return -1;
1561 Py_DECREF(res);
1562 return 0;
1563}
1564
1565/* tp_richcompare */
1566
1567static PyObject *
1568odict_richcompare(PyObject *v, PyObject *w, int op)
1569{
1570 if (!PyODict_Check(v) || !PyDict_Check(w)) {
1571 Py_RETURN_NOTIMPLEMENTED;
1572 }
1573
1574 if (op == Py_EQ || op == Py_NE) {
1575 PyObject *res, *cmp;
1576 int eq;
1577
1578 cmp = PyDict_Type.tp_richcompare(v, w, op);
1579 if (cmp == NULL)
1580 return NULL;
1581 if (!PyODict_Check(w))
1582 return cmp;
1583 if (op == Py_EQ && cmp == Py_False)
1584 return cmp;
1585 if (op == Py_NE && cmp == Py_True)
1586 return cmp;
1587 Py_DECREF(cmp);
1588
1589 /* Try comparing odict keys. */
1590 eq = _odict_keys_equal((PyODictObject *)v, (PyODictObject *)w);
1591 if (eq < 0)
1592 return NULL;
1593
1594 res = (eq == (op == Py_EQ)) ? Py_True : Py_False;
1595 Py_INCREF(res);
1596 return res;
1597 } else {
1598 Py_RETURN_NOTIMPLEMENTED;
1599 }
1600};
1601
1602/* tp_iter */
1603
1604static PyObject *
1605odict_iter(PyODictObject *od)
1606{
1607 return odictiter_new(od, _odict_ITER_KEYS);
1608};
1609
1610/* tp_init */
1611
1612static int
1613odict_init(PyObject *self, PyObject *args, PyObject *kwds)
1614{
1615 PyObject *res;
1616 Py_ssize_t len = PyObject_Length(args);
1617
1618 if (len == -1)
1619 return -1;
1620 if (len > 1) {
1621 char *msg = "expected at most 1 arguments, got %d";
1622 PyErr_Format(PyExc_TypeError, msg, len);
1623 return -1;
1624 }
1625
1626 /* __init__() triggering update() is just the way things are! */
1627 res = odict_update(self, args, kwds);
1628 if (res == NULL) {
1629 return -1;
1630 } else {
1631 Py_DECREF(res);
1632 return 0;
1633 }
1634};
1635
1636/* tp_new */
1637
1638static PyObject *
1639odict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
1640{
Victor Stinnerca30b022015-09-03 17:50:04 +02001641 PyObject *dict;
1642 PyODictObject *od;
1643
1644 dict = PyDict_New();
1645 if (dict == NULL)
1646 return NULL;
1647
1648 od = (PyODictObject *)PyDict_Type.tp_new(type, args, kwds);
1649 if (od == NULL) {
1650 Py_DECREF(dict);
1651 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001652 }
Victor Stinnerca30b022015-09-03 17:50:04 +02001653
1654 od->od_inst_dict = dict;
1655 /* type constructor fills the memory with zeros (see
1656 PyType_GenericAlloc()), there is no need to set them to zero again */
1657 if (_odict_resize(od) < 0) {
1658 Py_DECREF(od);
1659 return NULL;
1660 }
1661
1662 return (PyObject*)od;
Eric Snow96c6af92015-05-29 22:21:39 -06001663}
1664
1665/* PyODict_Type */
1666
1667PyTypeObject PyODict_Type = {
1668 PyVarObject_HEAD_INIT(&PyType_Type, 0)
1669 "collections.OrderedDict", /* tp_name */
1670 sizeof(PyODictObject), /* tp_basicsize */
1671 0, /* tp_itemsize */
1672 (destructor)odict_dealloc, /* tp_dealloc */
1673 0, /* tp_print */
1674 0, /* tp_getattr */
1675 0, /* tp_setattr */
1676 0, /* tp_reserved */
1677 (reprfunc)odict_repr, /* tp_repr */
1678 0, /* tp_as_number */
1679 0, /* tp_as_sequence */
1680 &odict_as_mapping, /* tp_as_mapping */
1681 0, /* tp_hash */
1682 0, /* tp_call */
1683 0, /* tp_str */
1684 0, /* tp_getattro */
1685 0, /* tp_setattro */
1686 0, /* tp_as_buffer */
1687 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,/* tp_flags */
1688 odict_doc, /* tp_doc */
1689 (traverseproc)odict_traverse, /* tp_traverse */
1690 (inquiry)odict_tp_clear, /* tp_clear */
1691 (richcmpfunc)odict_richcompare, /* tp_richcompare */
1692 offsetof(PyODictObject, od_weakreflist), /* tp_weaklistoffset */
1693 (getiterfunc)odict_iter, /* tp_iter */
1694 0, /* tp_iternext */
1695 odict_methods, /* tp_methods */
1696 odict_members, /* tp_members */
1697 0, /* tp_getset */
1698 &PyDict_Type, /* tp_base */
1699 0, /* tp_dict */
1700 0, /* tp_descr_get */
1701 0, /* tp_descr_set */
1702 offsetof(PyODictObject, od_inst_dict), /* tp_dictoffset */
1703 (initproc)odict_init, /* tp_init */
1704 PyType_GenericAlloc, /* tp_alloc */
1705 (newfunc)odict_new, /* tp_new */
1706 0, /* tp_free */
1707};
1708
1709
1710/* ----------------------------------------------
1711 * the public OrderedDict API
1712 */
1713
1714PyObject *
1715PyODict_New(void) {
1716 return odict_new(&PyODict_Type, NULL, NULL);
1717};
1718
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001719static int
1720_PyODict_SetItem_KnownHash(PyObject *od, PyObject *key, PyObject *value,
1721 Py_hash_t hash)
1722{
1723 int res = _PyDict_SetItem_KnownHash(od, key, value, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001724 if (res == 0) {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001725 res = _odict_add_new_node((PyODictObject *)od, key, hash);
Serhiy Storchakad5f353e2015-11-06 11:07:11 +02001726 if (res < 0) {
1727 /* Revert setting the value on the dict */
1728 PyObject *exc, *val, *tb;
1729 PyErr_Fetch(&exc, &val, &tb);
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001730 (void) _PyDict_DelItem_KnownHash(od, key, hash);
Serhiy Storchakad5f353e2015-11-06 11:07:11 +02001731 _PyErr_ChainExceptions(exc, val, tb);
1732 }
Eric Snow96c6af92015-05-29 22:21:39 -06001733 }
1734 return res;
1735};
1736
1737int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001738PyODict_SetItem(PyObject *od, PyObject *key, PyObject *value)
1739{
1740 Py_hash_t hash = PyObject_Hash(key);
1741 if (hash == -1)
1742 return -1;
1743 return _PyODict_SetItem_KnownHash(od, key, value, hash);
1744};
1745
1746int
1747PyODict_DelItem(PyObject *od, PyObject *key)
1748{
1749 int res;
1750 Py_hash_t hash = PyObject_Hash(key);
1751 if (hash == -1)
1752 return -1;
1753 res = _odict_clear_node((PyODictObject *)od, NULL, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001754 if (res < 0)
1755 return -1;
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001756 return _PyDict_DelItem_KnownHash(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001757};
1758
1759
1760/* -------------------------------------------
1761 * The OrderedDict views (keys/values/items)
1762 */
1763
1764typedef struct {
1765 PyObject_HEAD
1766 int kind;
1767 PyODictObject *di_odict;
Eric Snowb952ab42015-06-01 23:35:13 -06001768 Py_ssize_t di_size;
Eric Snow4fabf022015-06-04 00:09:56 -06001769 size_t di_state;
Eric Snow96c6af92015-05-29 22:21:39 -06001770 PyObject *di_current;
1771 PyObject *di_result; /* reusable result tuple for iteritems */
1772} odictiterobject;
1773
1774static void
1775odictiter_dealloc(odictiterobject *di)
1776{
1777 _PyObject_GC_UNTRACK(di);
Eric Snowa762af72015-06-01 22:59:08 -06001778 Py_XDECREF(di->di_odict);
Eric Snow96c6af92015-05-29 22:21:39 -06001779 Py_XDECREF(di->di_current);
1780 if (di->kind & (_odict_ITER_KEYS | _odict_ITER_VALUES)) {
1781 Py_DECREF(di->di_result);
1782 }
1783 PyObject_GC_Del(di);
1784}
1785
1786static int
1787odictiter_traverse(odictiterobject *di, visitproc visit, void *arg)
1788{
1789 Py_VISIT(di->di_odict);
1790 Py_VISIT(di->di_current); /* A key could be any type, not just str. */
1791 Py_VISIT(di->di_result);
1792 return 0;
1793}
1794
Eric Snowa762af72015-06-01 22:59:08 -06001795/* In order to protect against modifications during iteration, we track
1796 * the current key instead of the current node. */
Eric Snow96c6af92015-05-29 22:21:39 -06001797static PyObject *
1798odictiter_nextkey(odictiterobject *di)
1799{
Eric Snowa762af72015-06-01 22:59:08 -06001800 PyObject *key = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001801 _ODictNode *node;
1802 int reversed = di->kind & _odict_ITER_REVERSED;
1803
Eric Snowa762af72015-06-01 22:59:08 -06001804 if (di->di_odict == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -06001805 return NULL;
Eric Snowa762af72015-06-01 22:59:08 -06001806 if (di->di_current == NULL)
1807 goto done; /* We're already done. */
1808
Eric Snowb952ab42015-06-01 23:35:13 -06001809 /* Check for unsupported changes. */
Eric Snow4fabf022015-06-04 00:09:56 -06001810 if (di->di_odict->od_state != di->di_state) {
1811 PyErr_SetString(PyExc_RuntimeError,
1812 "OrderedDict mutated during iteration");
1813 goto done;
1814 }
Eric Snowb952ab42015-06-01 23:35:13 -06001815 if (di->di_size != PyODict_SIZE(di->di_odict)) {
1816 PyErr_SetString(PyExc_RuntimeError,
1817 "OrderedDict changed size during iteration");
1818 di->di_size = -1; /* Make this state sticky */
1819 return NULL;
1820 }
1821
Eric Snowa762af72015-06-01 22:59:08 -06001822 /* Get the key. */
Eric Snow96c6af92015-05-29 22:21:39 -06001823 node = _odict_find_node(di->di_odict, di->di_current);
1824 if (node == NULL) {
Serhiy Storchakab45b7b22015-11-04 22:05:38 +02001825 if (!PyErr_Occurred())
1826 PyErr_SetObject(PyExc_KeyError, di->di_current);
Eric Snow96c6af92015-05-29 22:21:39 -06001827 /* Must have been deleted. */
Eric Snowe3dfa9e2015-05-30 12:51:15 -06001828 Py_CLEAR(di->di_current);
Eric Snow96c6af92015-05-29 22:21:39 -06001829 return NULL;
1830 }
1831 key = di->di_current;
1832
1833 /* Advance to the next key. */
1834 node = reversed ? _odictnode_PREV(node) : _odictnode_NEXT(node);
1835 if (node == NULL) {
1836 /* Reached the end. */
1837 di->di_current = NULL;
1838 }
1839 else {
1840 di->di_current = _odictnode_KEY(node);
1841 Py_INCREF(di->di_current);
1842 }
1843
1844 return key;
Eric Snowa762af72015-06-01 22:59:08 -06001845
1846done:
1847 Py_CLEAR(di->di_odict);
1848 return key;
Eric Snow96c6af92015-05-29 22:21:39 -06001849}
1850
1851static PyObject *
1852odictiter_iternext(odictiterobject *di)
1853{
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001854 PyObject *result, *value;
Eric Snow96c6af92015-05-29 22:21:39 -06001855 PyObject *key = odictiter_nextkey(di); /* new reference */
1856
1857 if (key == NULL)
1858 return NULL;
1859
1860 /* Handle the keys case. */
1861 if (! (di->kind & _odict_ITER_VALUES)) {
1862 return key;
1863 }
1864
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001865 value = PyODict_GetItem((PyObject *)di->di_odict, key); /* borrowed */
1866 if (value == NULL) {
1867 if (!PyErr_Occurred())
1868 PyErr_SetObject(PyExc_KeyError, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001869 Py_DECREF(key);
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001870 goto done;
1871 }
1872 Py_INCREF(value);
1873
1874 /* Handle the values case. */
1875 if (!(di->kind & _odict_ITER_KEYS)) {
1876 Py_DECREF(key);
Eric Snow96c6af92015-05-29 22:21:39 -06001877 return value;
1878 }
Eric Snowa762af72015-06-01 22:59:08 -06001879
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001880 /* Handle the items case. */
1881 result = di->di_result;
1882
1883 if (Py_REFCNT(result) == 1) {
1884 /* not in use so we can reuse it
1885 * (the common case during iteration) */
1886 Py_INCREF(result);
1887 Py_DECREF(PyTuple_GET_ITEM(result, 0)); /* borrowed */
1888 Py_DECREF(PyTuple_GET_ITEM(result, 1)); /* borrowed */
1889 }
1890 else {
1891 result = PyTuple_New(2);
1892 if (result == NULL) {
1893 Py_DECREF(key);
1894 Py_DECREF(value);
1895 goto done;
1896 }
1897 }
1898
1899 PyTuple_SET_ITEM(result, 0, key); /* steals reference */
1900 PyTuple_SET_ITEM(result, 1, value); /* steals reference */
1901 return result;
1902
Eric Snowa762af72015-06-01 22:59:08 -06001903done:
1904 Py_CLEAR(di->di_current);
1905 Py_CLEAR(di->di_odict);
1906 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001907}
1908
1909/* No need for tp_clear because odictiterobject is not mutable. */
1910
1911PyDoc_STRVAR(reduce_doc, "Return state information for pickling");
1912
1913static PyObject *
1914odictiter_reduce(odictiterobject *di)
1915{
Eric Snowc5e59602015-05-30 11:28:56 -06001916 PyObject *list, *iter;
Eric Snow96c6af92015-05-29 22:21:39 -06001917
1918 list = PyList_New(0);
1919 if (!list)
1920 return NULL;
1921
1922 /* iterate the temporary into a list */
1923 for(;;) {
1924 PyObject *element = odictiter_iternext(di);
1925 if (element) {
1926 if (PyList_Append(list, element)) {
1927 Py_DECREF(element);
1928 Py_DECREF(list);
1929 return NULL;
1930 }
1931 Py_DECREF(element);
1932 }
1933 else {
1934 /* done iterating? */
1935 break;
1936 }
1937 }
1938 if (PyErr_Occurred()) {
1939 Py_DECREF(list);
1940 return NULL;
1941 }
Eric Snowc5e59602015-05-30 11:28:56 -06001942 iter = _PyObject_GetBuiltin("iter");
1943 if (iter == NULL) {
1944 Py_DECREF(list);
1945 return NULL;
1946 }
1947 return Py_BuildValue("N(N)", iter, list);
Eric Snow96c6af92015-05-29 22:21:39 -06001948}
1949
1950static PyMethodDef odictiter_methods[] = {
1951 {"__reduce__", (PyCFunction)odictiter_reduce, METH_NOARGS, reduce_doc},
1952 {NULL, NULL} /* sentinel */
1953};
1954
1955PyTypeObject PyODictIter_Type = {
1956 PyVarObject_HEAD_INIT(&PyType_Type, 0)
1957 "odict_iterator", /* tp_name */
1958 sizeof(odictiterobject), /* tp_basicsize */
1959 0, /* tp_itemsize */
1960 /* methods */
1961 (destructor)odictiter_dealloc, /* tp_dealloc */
1962 0, /* tp_print */
1963 0, /* tp_getattr */
1964 0, /* tp_setattr */
1965 0, /* tp_reserved */
1966 0, /* tp_repr */
1967 0, /* tp_as_number */
1968 0, /* tp_as_sequence */
1969 0, /* tp_as_mapping */
1970 0, /* tp_hash */
1971 0, /* tp_call */
1972 0, /* tp_str */
1973 PyObject_GenericGetAttr, /* tp_getattro */
1974 0, /* tp_setattro */
1975 0, /* tp_as_buffer */
1976 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */
1977 0, /* tp_doc */
1978 (traverseproc)odictiter_traverse, /* tp_traverse */
1979 0, /* tp_clear */
1980 0, /* tp_richcompare */
1981 0, /* tp_weaklistoffset */
1982 PyObject_SelfIter, /* tp_iter */
1983 (iternextfunc)odictiter_iternext, /* tp_iternext */
1984 odictiter_methods, /* tp_methods */
1985 0,
1986};
1987
1988static PyObject *
1989odictiter_new(PyODictObject *od, int kind)
1990{
1991 odictiterobject *di;
1992 _ODictNode *node;
1993 int reversed = kind & _odict_ITER_REVERSED;
1994
1995 di = PyObject_GC_New(odictiterobject, &PyODictIter_Type);
1996 if (di == NULL)
1997 return NULL;
1998
1999 if (kind & (_odict_ITER_KEYS | _odict_ITER_VALUES)){
2000 di->di_result = PyTuple_Pack(2, Py_None, Py_None);
2001 if (di->di_result == NULL) {
2002 Py_DECREF(di);
2003 return NULL;
2004 }
2005 }
2006 else
2007 di->di_result = NULL;
2008
2009 di->kind = kind;
2010 node = reversed ? _odict_LAST(od) : _odict_FIRST(od);
2011 di->di_current = node ? _odictnode_KEY(node) : NULL;
2012 Py_XINCREF(di->di_current);
Eric Snowb952ab42015-06-01 23:35:13 -06002013 di->di_size = PyODict_SIZE(od);
Eric Snow4fabf022015-06-04 00:09:56 -06002014 di->di_state = od->od_state;
Eric Snow96c6af92015-05-29 22:21:39 -06002015 di->di_odict = od;
2016 Py_INCREF(od);
2017
2018 _PyObject_GC_TRACK(di);
2019 return (PyObject *)di;
2020}
2021
2022/* keys() */
2023
2024static PyObject *
2025odictkeys_iter(_PyDictViewObject *dv)
2026{
2027 if (dv->dv_dict == NULL) {
2028 Py_RETURN_NONE;
2029 }
2030 return odictiter_new((PyODictObject *)dv->dv_dict,
2031 _odict_ITER_KEYS);
2032}
2033
2034static PyObject *
2035odictkeys_reversed(_PyDictViewObject *dv)
2036{
2037 if (dv->dv_dict == NULL) {
2038 Py_RETURN_NONE;
2039 }
2040 return odictiter_new((PyODictObject *)dv->dv_dict,
2041 _odict_ITER_KEYS|_odict_ITER_REVERSED);
2042}
2043
2044static PyMethodDef odictkeys_methods[] = {
2045 {"__reversed__", (PyCFunction)odictkeys_reversed, METH_NOARGS, NULL},
2046 {NULL, NULL} /* sentinel */
2047};
2048
2049PyTypeObject PyODictKeys_Type = {
2050 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2051 "odict_keys", /* tp_name */
2052 0, /* tp_basicsize */
2053 0, /* tp_itemsize */
2054 0, /* tp_dealloc */
2055 0, /* tp_print */
2056 0, /* tp_getattr */
2057 0, /* tp_setattr */
2058 0, /* tp_reserved */
2059 0, /* tp_repr */
2060 0, /* tp_as_number */
2061 0, /* tp_as_sequence */
2062 0, /* tp_as_mapping */
2063 0, /* tp_hash */
2064 0, /* tp_call */
2065 0, /* tp_str */
2066 0, /* tp_getattro */
2067 0, /* tp_setattro */
2068 0, /* tp_as_buffer */
2069 0, /* tp_flags */
2070 0, /* tp_doc */
2071 0, /* tp_traverse */
2072 0, /* tp_clear */
2073 0, /* tp_richcompare */
2074 0, /* tp_weaklistoffset */
2075 (getiterfunc)odictkeys_iter, /* tp_iter */
2076 0, /* tp_iternext */
2077 odictkeys_methods, /* tp_methods */
2078 0, /* tp_members */
2079 0, /* tp_getset */
2080 &PyDictKeys_Type, /* tp_base */
2081};
2082
2083static PyObject *
2084odictkeys_new(PyObject *od)
2085{
2086 return _PyDictView_New(od, &PyODictKeys_Type);
2087}
2088
2089/* items() */
2090
2091static PyObject *
2092odictitems_iter(_PyDictViewObject *dv)
2093{
2094 if (dv->dv_dict == NULL) {
2095 Py_RETURN_NONE;
2096 }
2097 return odictiter_new((PyODictObject *)dv->dv_dict,
2098 _odict_ITER_KEYS|_odict_ITER_VALUES);
2099}
2100
2101static PyObject *
2102odictitems_reversed(_PyDictViewObject *dv)
2103{
2104 if (dv->dv_dict == NULL) {
2105 Py_RETURN_NONE;
2106 }
2107 return odictiter_new((PyODictObject *)dv->dv_dict,
2108 _odict_ITER_KEYS|_odict_ITER_VALUES|_odict_ITER_REVERSED);
2109}
2110
2111static PyMethodDef odictitems_methods[] = {
2112 {"__reversed__", (PyCFunction)odictitems_reversed, METH_NOARGS, NULL},
2113 {NULL, NULL} /* sentinel */
2114};
2115
2116PyTypeObject PyODictItems_Type = {
2117 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2118 "odict_items", /* tp_name */
2119 0, /* tp_basicsize */
2120 0, /* tp_itemsize */
2121 0, /* tp_dealloc */
2122 0, /* tp_print */
2123 0, /* tp_getattr */
2124 0, /* tp_setattr */
2125 0, /* tp_reserved */
2126 0, /* tp_repr */
2127 0, /* tp_as_number */
2128 0, /* tp_as_sequence */
2129 0, /* tp_as_mapping */
2130 0, /* tp_hash */
2131 0, /* tp_call */
2132 0, /* tp_str */
2133 0, /* tp_getattro */
2134 0, /* tp_setattro */
2135 0, /* tp_as_buffer */
2136 0, /* tp_flags */
2137 0, /* tp_doc */
2138 0, /* tp_traverse */
2139 0, /* tp_clear */
2140 0, /* tp_richcompare */
2141 0, /* tp_weaklistoffset */
2142 (getiterfunc)odictitems_iter, /* tp_iter */
2143 0, /* tp_iternext */
2144 odictitems_methods, /* tp_methods */
2145 0, /* tp_members */
2146 0, /* tp_getset */
2147 &PyDictItems_Type, /* tp_base */
2148};
2149
2150static PyObject *
2151odictitems_new(PyObject *od)
2152{
2153 return _PyDictView_New(od, &PyODictItems_Type);
2154}
2155
2156/* values() */
2157
2158static PyObject *
2159odictvalues_iter(_PyDictViewObject *dv)
2160{
2161 if (dv->dv_dict == NULL) {
2162 Py_RETURN_NONE;
2163 }
2164 return odictiter_new((PyODictObject *)dv->dv_dict,
2165 _odict_ITER_VALUES);
2166}
2167
2168static PyObject *
2169odictvalues_reversed(_PyDictViewObject *dv)
2170{
2171 if (dv->dv_dict == NULL) {
2172 Py_RETURN_NONE;
2173 }
2174 return odictiter_new((PyODictObject *)dv->dv_dict,
2175 _odict_ITER_VALUES|_odict_ITER_REVERSED);
2176}
2177
2178static PyMethodDef odictvalues_methods[] = {
2179 {"__reversed__", (PyCFunction)odictvalues_reversed, METH_NOARGS, NULL},
2180 {NULL, NULL} /* sentinel */
2181};
2182
2183PyTypeObject PyODictValues_Type = {
2184 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2185 "odict_values", /* tp_name */
2186 0, /* tp_basicsize */
2187 0, /* tp_itemsize */
2188 0, /* tp_dealloc */
2189 0, /* tp_print */
2190 0, /* tp_getattr */
2191 0, /* tp_setattr */
2192 0, /* tp_reserved */
2193 0, /* tp_repr */
2194 0, /* tp_as_number */
2195 0, /* tp_as_sequence */
2196 0, /* tp_as_mapping */
2197 0, /* tp_hash */
2198 0, /* tp_call */
2199 0, /* tp_str */
2200 0, /* tp_getattro */
2201 0, /* tp_setattro */
2202 0, /* tp_as_buffer */
2203 0, /* tp_flags */
2204 0, /* tp_doc */
2205 0, /* tp_traverse */
2206 0, /* tp_clear */
2207 0, /* tp_richcompare */
2208 0, /* tp_weaklistoffset */
2209 (getiterfunc)odictvalues_iter, /* tp_iter */
2210 0, /* tp_iternext */
2211 odictvalues_methods, /* tp_methods */
2212 0, /* tp_members */
2213 0, /* tp_getset */
2214 &PyDictValues_Type, /* tp_base */
2215};
2216
2217static PyObject *
2218odictvalues_new(PyObject *od)
2219{
2220 return _PyDictView_New(od, &PyODictValues_Type);
2221}
2222
2223
2224/* ----------------------------------------------
2225 MutableMappping implementations
2226
2227Mapping:
2228
2229============ ===========
2230method uses
2231============ ===========
2232__contains__ __getitem__
2233__eq__ items
2234__getitem__ +
2235__iter__ +
2236__len__ +
2237__ne__ __eq__
2238get __getitem__
2239items ItemsView
2240keys KeysView
2241values ValuesView
2242============ ===========
2243
2244ItemsView uses __len__, __iter__, and __getitem__.
2245KeysView uses __len__, __iter__, and __contains__.
2246ValuesView uses __len__, __iter__, and __getitem__.
2247
2248MutableMapping:
2249
2250============ ===========
2251method uses
2252============ ===========
2253__delitem__ +
2254__setitem__ +
2255clear popitem
2256pop __getitem__
2257 __delitem__
2258popitem __iter__
2259 _getitem__
2260 __delitem__
2261setdefault __getitem__
2262 __setitem__
2263update __setitem__
2264============ ===========
2265*/
2266
2267static int
2268mutablemapping_add_pairs(PyObject *self, PyObject *pairs)
2269{
2270 PyObject *pair, *iterator, *unexpected;
2271 int res = 0;
2272
2273 iterator = PyObject_GetIter(pairs);
2274 if (iterator == NULL)
2275 return -1;
2276 PyErr_Clear();
2277
2278 while ((pair = PyIter_Next(iterator)) != NULL) {
2279 /* could be more efficient (see UNPACK_SEQUENCE in ceval.c) */
Eric Snowc5e59602015-05-30 11:28:56 -06002280 PyObject *key = NULL, *value = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06002281 PyObject *pair_iterator = PyObject_GetIter(pair);
Eric Snowc5e59602015-05-30 11:28:56 -06002282 if (pair_iterator == NULL)
2283 goto Done;
Eric Snow96c6af92015-05-29 22:21:39 -06002284
2285 key = PyIter_Next(pair_iterator);
2286 if (key == NULL) {
2287 if (!PyErr_Occurred())
2288 PyErr_SetString(PyExc_ValueError,
2289 "need more than 0 values to unpack");
2290 goto Done;
2291 }
2292
2293 value = PyIter_Next(pair_iterator);
2294 if (value == NULL) {
2295 if (!PyErr_Occurred())
2296 PyErr_SetString(PyExc_ValueError,
2297 "need more than 1 value to unpack");
2298 goto Done;
2299 }
2300
2301 unexpected = PyIter_Next(pair_iterator);
2302 if (unexpected != NULL) {
2303 Py_DECREF(unexpected);
2304 PyErr_SetString(PyExc_ValueError,
2305 "too many values to unpack (expected 2)");
2306 goto Done;
2307 }
2308 else if (PyErr_Occurred())
2309 goto Done;
2310
2311 res = PyObject_SetItem(self, key, value);
2312
2313Done:
2314 Py_DECREF(pair);
Eric Snowc5e59602015-05-30 11:28:56 -06002315 Py_XDECREF(pair_iterator);
Eric Snow96c6af92015-05-29 22:21:39 -06002316 Py_XDECREF(key);
2317 Py_XDECREF(value);
2318 if (PyErr_Occurred())
2319 break;
2320 }
2321 Py_DECREF(iterator);
2322
2323 if (res < 0 || PyErr_Occurred() != NULL)
2324 return -1;
2325 else
2326 return 0;
2327}
2328
2329static PyObject *
2330mutablemapping_update(PyObject *self, PyObject *args, PyObject *kwargs)
2331{
2332 int res = 0;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002333 Py_ssize_t len;
2334 _Py_IDENTIFIER(items);
2335 _Py_IDENTIFIER(keys);
Eric Snow96c6af92015-05-29 22:21:39 -06002336
2337 /* first handle args, if any */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002338 assert(args == NULL || PyTuple_Check(args));
2339 len = (args != NULL) ? PyTuple_GET_SIZE(args) : 0;
Benjamin Peterson99e96f22015-06-06 23:20:32 -05002340 if (len > 1) {
Eric Snow96c6af92015-05-29 22:21:39 -06002341 char *msg = "update() takes at most 1 positional argument (%d given)";
2342 PyErr_Format(PyExc_TypeError, msg, len);
2343 return NULL;
2344 }
Eric Snow96c6af92015-05-29 22:21:39 -06002345
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002346 if (len) {
Benjamin Peterson0718de92015-06-07 00:00:42 -05002347 PyObject *other = PyTuple_GET_ITEM(args, 0); /* borrowed reference */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002348 assert(other != NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002349 Py_INCREF(other);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002350 if (PyDict_CheckExact(other) ||
2351 _PyObject_HasAttrId(other, &PyId_items)) { /* never fails */
2352 PyObject *items;
2353 if (PyDict_CheckExact(other))
2354 items = PyDict_Items(other);
2355 else
2356 items = _PyObject_CallMethodId(other, &PyId_items, NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002357 Py_DECREF(other);
2358 if (items == NULL)
2359 return NULL;
2360 res = mutablemapping_add_pairs(self, items);
2361 Py_DECREF(items);
2362 if (res == -1)
2363 return NULL;
2364 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002365 else if (_PyObject_HasAttrId(other, &PyId_keys)) { /* never fails */
Benjamin Peterson0718de92015-06-07 00:00:42 -05002366 PyObject *keys, *iterator, *key;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002367 keys = _PyObject_CallMethodIdObjArgs(other, &PyId_keys, NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002368 if (keys == NULL) {
2369 Py_DECREF(other);
2370 return NULL;
2371 }
2372 iterator = PyObject_GetIter(keys);
2373 Py_DECREF(keys);
2374 if (iterator == NULL) {
2375 Py_DECREF(other);
2376 return NULL;
2377 }
2378 while (res == 0 && (key = PyIter_Next(iterator))) {
2379 PyObject *value = PyObject_GetItem(other, key);
2380 if (value != NULL) {
2381 res = PyObject_SetItem(self, key, value);
2382 Py_DECREF(value);
2383 }
2384 else {
2385 res = -1;
2386 }
2387 Py_DECREF(key);
2388 }
2389 Py_DECREF(other);
2390 Py_DECREF(iterator);
2391 if (res != 0 || PyErr_Occurred())
2392 return NULL;
2393 }
2394 else {
2395 res = mutablemapping_add_pairs(self, other);
2396 Py_DECREF(other);
2397 if (res != 0)
2398 return NULL;
2399 }
2400 }
2401
2402 /* now handle kwargs */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002403 assert(kwargs == NULL || PyDict_Check(kwargs));
2404 len = (kwargs != NULL) ? PyDict_Size(kwargs) : 0;
Benjamin Peterson0718de92015-06-07 00:00:42 -05002405 if (len > 0) {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002406 PyObject *items = PyDict_Items(kwargs);
Eric Snow96c6af92015-05-29 22:21:39 -06002407 if (items == NULL)
2408 return NULL;
2409 res = mutablemapping_add_pairs(self, items);
2410 Py_DECREF(items);
2411 if (res == -1)
2412 return NULL;
2413 }
Eric Snow96c6af92015-05-29 22:21:39 -06002414
2415 Py_RETURN_NONE;
2416}