blob: c2cef21b49774856447d66611961c009188d1f12 [file] [log] [blame]
Eric Snow96c6af92015-05-29 22:21:39 -06001/* Ordered Dictionary object implementation.
2
3This implementation is necessarily explicitly equivalent to the pure Python
4OrderedDict class in Lib/collections/__init__.py. The strategy there
5involves using a doubly-linked-list to capture the order. We keep to that
6strategy, using a lower-level linked-list.
7
8About the Linked-List
9=====================
10
11For the linked list we use a basic doubly-linked-list. Using a circularly-
12linked-list does have some benefits, but they don't apply so much here
13since OrderedDict is focused on the ends of the list (for the most part).
14Furthermore, there are some features of generic linked-lists that we simply
15don't need for OrderedDict. Thus a simple custom implementation meets our
16needs. Alternatives to our simple approach include the QCIRCLE_*
17macros from BSD's queue.h, and the linux's list.h.
18
19Getting O(1) Node Lookup
20------------------------
21
22One invariant of Python's OrderedDict is that it preserves time complexity
23of dict's methods, particularly the O(1) operations. Simply adding a
24linked-list on top of dict is not sufficient here; operations for nodes in
25the middle of the linked-list implicitly require finding the node first.
26With a simple linked-list like we're using, that is an O(n) operation.
27Consequently, methods like __delitem__() would change from O(1) to O(n),
28which is unacceptable.
29
30In order to preserve O(1) performance for node removal (finding nodes), we
31must do better than just looping through the linked-list. Here are options
32we've considered:
33
341. use a second dict to map keys to nodes (a la the pure Python version).
352. keep a simple hash table mirroring the order of dict's, mapping each key
36 to the corresponding node in the linked-list.
373. use a version of shared keys (split dict) that allows non-unicode keys.
384. have the value stored for each key be a (value, node) pair, and adjust
39 __getitem__(), get(), etc. accordingly.
40
41The approach with the least performance impact (time and space) is #2,
Benjamin Petersonee178e62016-09-08 11:08:30 -070042mirroring the key order of dict's dk_entries with an array of node pointers.
Eric Snow96c6af92015-05-29 22:21:39 -060043While lookdict() and friends (dk_lookup) don't give us the index into the
44array, we make use of pointer arithmetic to get that index. An alternative
45would be to refactor lookdict() to provide the index, explicitly exposing
46the implementation detail. We could even just use a custom lookup function
47for OrderedDict that facilitates our need. However, both approaches are
48significantly more complicated than just using pointer arithmetic.
49
50The catch with mirroring the hash table ordering is that we have to keep
51the ordering in sync through any dict resizes. However, that order only
52matters during node lookup. We can simply defer any potential resizing
53until we need to do a lookup.
54
55Linked-List Nodes
56-----------------
57
58The current implementation stores a pointer to the associated key only.
59One alternative would be to store a pointer to the PyDictKeyEntry instead.
60This would save one pointer de-reference per item, which is nice during
61calls to values() and items(). However, it adds unnecessary overhead
62otherwise, so we stick with just the key.
63
64Linked-List API
65---------------
66
67As noted, the linked-list implemented here does not have all the bells and
68whistles. However, we recognize that the implementation may need to
69change to accommodate performance improvements or extra functionality. To
70that end, We use a simple API to interact with the linked-list. Here's a
71summary of the methods/macros:
72
73Node info:
74
75* _odictnode_KEY(node)
76* _odictnode_VALUE(od, node)
77* _odictnode_PREV(node)
78* _odictnode_NEXT(node)
79
80Linked-List info:
81
82* _odict_FIRST(od)
83* _odict_LAST(od)
84* _odict_EMPTY(od)
85* _odict_FOREACH(od, node) - used in place of `for (node=...)`
86
87For adding nodes:
88
89* _odict_add_head(od, node)
90* _odict_add_tail(od, node)
Serhiy Storchaka19a70e72015-11-13 14:48:36 +020091* _odict_add_new_node(od, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -060092
93For removing nodes:
94
Serhiy Storchaka19a70e72015-11-13 14:48:36 +020095* _odict_clear_node(od, node, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -060096* _odict_clear_nodes(od, clear_each)
97
98Others:
99
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200100* _odict_find_node_hash(od, key, hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600101* _odict_find_node(od, key)
102* _odict_keys_equal(od1, od2)
103
104Used, but specific to the linked-list implementation:
105
106* _odict_free_fast_nodes(od)
107
108And here's a look at how the linked-list relates to the OrderedDict API:
109
110============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
111method key val prev next mem 1st last empty iter find add rmv clr keq
112============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
113__del__ ~ X
114__delitem__ free ~ node
115__eq__ ~ X
116__iter__ X X
117__new__ X X
118__reduce__ X ~ X
119__repr__ X X X
120__reversed__ X X
121__setitem__ key
122__sizeof__ size X
123clear ~ ~ X
124copy X X X
125items X X X
126keys X X
127move_to_end X X X ~ h/t key
128pop free key
129popitem X X free X X node
130setdefault ~ ? ~
131values X X
132============ === === ==== ==== ==== === ==== ===== ==== ==== === ==== === ===
133
134__delitem__ is the only method that directly relies on finding an arbitrary
135node in the linked-list. Everything else is iteration or relates to the
136ends of the linked-list.
137
138Situation that Endangers Consistency
139------------------------------------
140Using a raw linked-list for OrderedDict exposes a key situation that can
141cause problems. If a node is stored in a variable, there is a chance that
142the node may have been deallocated before the variable gets used, thus
143potentially leading to a segmentation fault. A key place where this shows
144up is during iteration through the linked list (via _odict_FOREACH or
145otherwise).
146
147A number of solutions are available to resolve this situation:
148
149* defer looking up the node until as late as possible and certainly after
150 any code that could possibly result in a deletion;
151* if the node is needed both before and after a point where the node might
152 be removed, do a check before using the node at the "after" location to
153 see if the node is still valid;
154* like the last one, but simply pull the node again to ensure it's right;
155* keep the key in the variable instead of the node and then look up the
156 node using the key at the point where the node is needed (this is what
157 we do for the iterators).
158
159Another related problem, preserving consistent ordering during iteration,
160is described below. That one is not exclusive to using linked-lists.
161
162
163Challenges from Subclassing dict
164================================
165
166OrderedDict subclasses dict, which is an unusual relationship between two
167builtin types (other than the base object type). Doing so results in
168some complication and deserves further explanation. There are two things
169to consider here. First, in what circumstances or with what adjustments
170can OrderedDict be used as a drop-in replacement for dict (at the C level)?
171Second, how can the OrderedDict implementation leverage the dict
172implementation effectively without introducing unnecessary coupling or
173inefficiencies?
174
175This second point is reflected here and in the implementation, so the
176further focus is on the first point. It is worth noting that for
177overridden methods, the dict implementation is deferred to as much as
178possible. Furthermore, coupling is limited to as little as is reasonable.
179
180Concrete API Compatibility
181--------------------------
182
183Use of the concrete C-API for dict (PyDict_*) with OrderedDict is
184problematic. (See http://bugs.python.org/issue10977.) The concrete API
185has a number of hard-coded assumptions tied to the dict implementation.
186This is, in part, due to performance reasons, which is understandable
187given the part dict plays in Python.
188
189Any attempt to replace dict with OrderedDict for any role in the
190interpreter (e.g. **kwds) faces a challenge. Such any effort must
191recognize that the instances in affected locations currently interact with
192the concrete API.
193
194Here are some ways to address this challenge:
195
1961. Change the relevant usage of the concrete API in CPython and add
Martin Panter69332c12016-08-04 13:07:31 +0000197 PyDict_CheckExact() calls to each of the concrete API functions.
Eric Snow96c6af92015-05-29 22:21:39 -06001982. Adjust the relevant concrete API functions to explicitly accommodate
199 OrderedDict.
2003. As with #1, add the checks, but improve the abstract API with smart fast
201 paths for dict and OrderedDict, and refactor CPython to use the abstract
202 API. Improvements to the abstract API would be valuable regardless.
203
204Adding the checks to the concrete API would help make any interpreter
205switch to OrderedDict less painful for extension modules. However, this
206won't work. The equivalent C API call to `dict.__setitem__(obj, k, v)`
207is 'PyDict_SetItem(obj, k, v)`. This illustrates how subclasses in C call
208the base class's methods, since there is no equivalent of super() in the
209C API. Calling into Python for parent class API would work, but some
210extension modules already rely on this feature of the concrete API.
211
212For reference, here is a breakdown of some of the dict concrete API:
213
214========================== ============= =======================
215concrete API uses abstract API
216========================== ============= =======================
217PyDict_Check PyMapping_Check
218(PyDict_CheckExact) -
219(PyDict_New) -
220(PyDictProxy_New) -
221PyDict_Clear -
222PyDict_Contains PySequence_Contains
223PyDict_Copy -
224PyDict_SetItem PyObject_SetItem
225PyDict_SetItemString PyMapping_SetItemString
226PyDict_DelItem PyMapping_DelItem
227PyDict_DelItemString PyMapping_DelItemString
228PyDict_GetItem -
229PyDict_GetItemWithError PyObject_GetItem
230_PyDict_GetItemIdWithError -
231PyDict_GetItemString PyMapping_GetItemString
232PyDict_Items PyMapping_Items
233PyDict_Keys PyMapping_Keys
234PyDict_Values PyMapping_Values
235PyDict_Size PyMapping_Size
236 PyMapping_Length
237PyDict_Next PyIter_Next
238_PyDict_Next -
239PyDict_Merge -
240PyDict_Update -
241PyDict_MergeFromSeq2 -
242PyDict_ClearFreeList -
243- PyMapping_HasKeyString
244- PyMapping_HasKey
245========================== ============= =======================
246
247
248The dict Interface Relative to OrderedDict
249==========================================
250
251Since OrderedDict subclasses dict, understanding the various methods and
252attributes of dict is important for implementing OrderedDict.
253
254Relevant Type Slots
255-------------------
256
257================= ================ =================== ================
258slot attribute object dict
259================= ================ =================== ================
260tp_dealloc - object_dealloc dict_dealloc
261tp_repr __repr__ object_repr dict_repr
262sq_contains __contains__ - dict_contains
263mp_length __len__ - dict_length
264mp_subscript __getitem__ - dict_subscript
265mp_ass_subscript __setitem__ - dict_ass_sub
266 __delitem__
267tp_hash __hash__ _Py_HashPointer ..._HashNotImpl
268tp_str __str__ object_str -
269tp_getattro __getattribute__ ..._GenericGetAttr (repeated)
270 __getattr__
271tp_setattro __setattr__ ..._GenericSetAttr (disabled)
272tp_doc __doc__ (literal) dictionary_doc
273tp_traverse - - dict_traverse
274tp_clear - - dict_tp_clear
275tp_richcompare __eq__ object_richcompare dict_richcompare
276 __ne__
277tp_weaklistoffset (__weakref__) - -
278tp_iter __iter__ - dict_iter
279tp_dictoffset (__dict__) - -
280tp_init __init__ object_init dict_init
281tp_alloc - PyType_GenericAlloc (repeated)
282tp_new __new__ object_new dict_new
283tp_free - PyObject_Del PyObject_GC_Del
284================= ================ =================== ================
285
286Relevant Methods
287----------------
288
289================ =================== ===============
290method object dict
291================ =================== ===============
292__reduce__ object_reduce -
293__sizeof__ object_sizeof dict_sizeof
294clear - dict_clear
295copy - dict_copy
296fromkeys - dict_fromkeys
297get - dict_get
298items - dictitems_new
299keys - dictkeys_new
300pop - dict_pop
301popitem - dict_popitem
302setdefault - dict_setdefault
303update - dict_update
304values - dictvalues_new
305================ =================== ===============
306
307
308Pure Python OrderedDict
309=======================
310
311As already noted, compatibility with the pure Python OrderedDict
312implementation is a key goal of this C implementation. To further that
313goal, here's a summary of how OrderedDict-specific methods are implemented
314in collections/__init__.py. Also provided is an indication of which
315methods directly mutate or iterate the object, as well as any relationship
316with the underlying linked-list.
317
318============= ============== == ================ === === ====
319method impl used ll uses inq mut iter
320============= ============== == ================ === === ====
321__contains__ dict - - X
322__delitem__ OrderedDict Y dict.__delitem__ X
323__eq__ OrderedDict N OrderedDict ~
324 dict.__eq__
325 __iter__
326__getitem__ dict - - X
327__iter__ OrderedDict Y - X
328__init__ OrderedDict N update
329__len__ dict - - X
330__ne__ MutableMapping - __eq__ ~
331__reduce__ OrderedDict N OrderedDict ~
332 __iter__
333 __getitem__
334__repr__ OrderedDict N __class__ ~
335 items
336__reversed__ OrderedDict Y - X
337__setitem__ OrderedDict Y __contains__ X
338 dict.__setitem__
339__sizeof__ OrderedDict Y __len__ ~
340 __dict__
341clear OrderedDict Y dict.clear X
342copy OrderedDict N __class__
343 __init__
344fromkeys OrderedDict N __setitem__
345get dict - - ~
346items MutableMapping - ItemsView X
347keys MutableMapping - KeysView X
348move_to_end OrderedDict Y - X
349pop OrderedDict N __contains__ X
350 __getitem__
351 __delitem__
352popitem OrderedDict Y dict.pop X
353setdefault OrderedDict N __contains__ ~
354 __getitem__
355 __setitem__
356update MutableMapping - __setitem__ ~
357values MutableMapping - ValuesView X
358============= ============== == ================ === === ====
359
360__reversed__ and move_to_end are both exclusive to OrderedDict.
361
362
363C OrderedDict Implementation
364============================
365
366================= ================
367slot impl
368================= ================
369tp_dealloc odict_dealloc
370tp_repr odict_repr
371mp_ass_subscript odict_ass_sub
372tp_doc odict_doc
373tp_traverse odict_traverse
374tp_clear odict_tp_clear
375tp_richcompare odict_richcompare
376tp_weaklistoffset (offset)
377tp_iter odict_iter
378tp_dictoffset (offset)
379tp_init odict_init
380tp_alloc (repeated)
381tp_new odict_new
382================= ================
383
384================= ================
385method impl
386================= ================
387__reduce__ odict_reduce
388__sizeof__ odict_sizeof
389clear odict_clear
390copy odict_copy
391fromkeys odict_fromkeys
392items odictitems_new
393keys odictkeys_new
394pop odict_pop
395popitem odict_popitem
396setdefault odict_setdefault
397update odict_update
398values odictvalues_new
399================= ================
400
401Inherited unchanged from object/dict:
402
403================ ==========================
404method type field
405================ ==========================
406- tp_free
407__contains__ tp_as_sequence.sq_contains
408__getattr__ tp_getattro
409__getattribute__ tp_getattro
410__getitem__ tp_as_mapping.mp_subscript
411__hash__ tp_hash
412__len__ tp_as_mapping.mp_length
413__setattr__ tp_setattro
414__str__ tp_str
415get -
416================ ==========================
417
418
419Other Challenges
420================
421
422Preserving Ordering During Iteration
423------------------------------------
424During iteration through an OrderedDict, it is possible that items could
425get added, removed, or reordered. For a linked-list implementation, as
426with some other implementations, that situation may lead to undefined
427behavior. The documentation for dict mentions this in the `iter()` section
428of http://docs.python.org/3.4/library/stdtypes.html#dictionary-view-objects.
429In this implementation we follow dict's lead (as does the pure Python
430implementation) for __iter__(), keys(), values(), and items().
431
432For internal iteration (using _odict_FOREACH or not), there is still the
433risk that not all nodes that we expect to be seen in the loop actually get
434seen. Thus, we are careful in each of those places to ensure that they
435are. This comes, of course, at a small price at each location. The
436solutions are much the same as those detailed in the `Situation that
437Endangers Consistency` section above.
438
439
440Potential Optimizations
441=======================
442
443* Allocate the nodes as a block via od_fast_nodes instead of individually.
444 - Set node->key to NULL to indicate the node is not-in-use.
445 - Add _odict_EXISTS()?
446 - How to maintain consistency across resizes? Existing node pointers
447 would be invalidate after a resize, which is particularly problematic
448 for the iterators.
449* Use a more stream-lined implementation of update() and, likely indirectly,
450 __init__().
451
452*/
453
454/* TODO
455
456sooner:
457- reentrancy (make sure everything is at a thread-safe state when calling
458 into Python). I've already checked this multiple times, but want to
459 make one more pass.
460- add unit tests for reentrancy?
461
462later:
463- make the dict views support the full set API (the pure Python impl does)
464- implement a fuller MutableMapping API in C?
465- move the MutableMapping implementation to abstract.c?
466- optimize mutablemapping_update
467- use PyObject_MALLOC (small object allocator) for odict nodes?
468- support subclasses better (e.g. in odict_richcompare)
469
470*/
471
472#include "Python.h"
473#include "structmember.h"
474#include "dict-common.h"
475#include <stddef.h>
476
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100477#include "clinic/odictobject.c.h"
478
479/*[clinic input]
480class OrderedDict "PyODictObject *" "&PyODict_Type"
481[clinic start generated code]*/
482/*[clinic end generated code: output=da39a3ee5e6b4b0d input=ca0641cf6143d4af]*/
483
Eric Snow96c6af92015-05-29 22:21:39 -0600484
485typedef struct _odictnode _ODictNode;
486
487/* PyODictObject */
488struct _odictobject {
Eric Snow4fabf022015-06-04 00:09:56 -0600489 PyDictObject od_dict; /* the underlying dict */
490 _ODictNode *od_first; /* first node in the linked list, if any */
491 _ODictNode *od_last; /* last node in the linked list, if any */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200492 /* od_fast_nodes, od_fast_nodes_size and od_resize_sentinel are managed
493 * by _odict_resize().
Eric Snow8c7f9552015-08-07 17:45:12 -0600494 * Note that we rely on implementation details of dict for both. */
495 _ODictNode **od_fast_nodes; /* hash table that mirrors the dict table */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200496 Py_ssize_t od_fast_nodes_size;
497 void *od_resize_sentinel; /* changes if odict should be resized */
Eric Snow8c7f9552015-08-07 17:45:12 -0600498
Eric Snow4fabf022015-06-04 00:09:56 -0600499 size_t od_state; /* incremented whenever the LL changes */
500 PyObject *od_inst_dict; /* OrderedDict().__dict__ */
501 PyObject *od_weakreflist; /* holds weakrefs to the odict */
Eric Snow96c6af92015-05-29 22:21:39 -0600502};
503
504
505/* ----------------------------------------------
506 * odict keys (a simple doubly-linked list)
507 */
508
509struct _odictnode {
510 PyObject *key;
Eric Snow4c729182015-06-03 10:50:37 -0600511 Py_hash_t hash;
Eric Snow96c6af92015-05-29 22:21:39 -0600512 _ODictNode *next;
513 _ODictNode *prev;
514};
515
516#define _odictnode_KEY(node) \
517 (node->key)
Eric Snow4c729182015-06-03 10:50:37 -0600518#define _odictnode_HASH(node) \
519 (node->hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600520/* borrowed reference */
521#define _odictnode_VALUE(node, od) \
Eric Snowa762af72015-06-01 22:59:08 -0600522 PyODict_GetItemWithError((PyObject *)od, _odictnode_KEY(node))
Eric Snow96c6af92015-05-29 22:21:39 -0600523#define _odictnode_PREV(node) (node->prev)
524#define _odictnode_NEXT(node) (node->next)
525
526#define _odict_FIRST(od) (((PyODictObject *)od)->od_first)
527#define _odict_LAST(od) (((PyODictObject *)od)->od_last)
528#define _odict_EMPTY(od) (_odict_FIRST(od) == NULL)
529#define _odict_FOREACH(od, node) \
530 for (node = _odict_FIRST(od); node != NULL; node = _odictnode_NEXT(node))
531
Eric Snow8c7f9552015-08-07 17:45:12 -0600532#define _odict_FAST_SIZE(od) ((PyDictObject *)od)->ma_keys->dk_size
Eric Snow96c6af92015-05-29 22:21:39 -0600533
Eric Snow96c6af92015-05-29 22:21:39 -0600534static void
535_odict_free_fast_nodes(PyODictObject *od) {
536 if (od->od_fast_nodes) {
537 PyMem_FREE(od->od_fast_nodes);
538 }
539}
540
Eric Snow4c729182015-06-03 10:50:37 -0600541/* Return the index into the hash table, regardless of a valid node. */
542static Py_ssize_t
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200543_odict_get_index_raw(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow4c729182015-06-03 10:50:37 -0600544{
INADA Naokiba609772016-12-07 20:41:42 +0900545 PyObject *value = NULL;
Eric Snow4c729182015-06-03 10:50:37 -0600546 PyDictKeysObject *keys = ((PyDictObject *)od)->ma_keys;
Victor Stinner742da042016-09-07 17:40:12 -0700547 Py_ssize_t ix;
Eric Snow4c729182015-06-03 10:50:37 -0600548
INADA Naokiba609772016-12-07 20:41:42 +0900549 ix = (keys->dk_lookup)((PyDictObject *)od, key, hash, &value, NULL);
Victor Stinner742da042016-09-07 17:40:12 -0700550 if (ix == DKIX_EMPTY) {
551 return keys->dk_nentries; /* index of new entry */
552 }
553 if (ix < 0)
Eric Snow4c729182015-06-03 10:50:37 -0600554 return -1;
555 /* We use pointer arithmetic to get the entry's index into the table. */
Victor Stinner742da042016-09-07 17:40:12 -0700556 return ix;
Eric Snow4c729182015-06-03 10:50:37 -0600557}
558
559/* Replace od->od_fast_nodes with a new table matching the size of dict's. */
Eric Snow96c6af92015-05-29 22:21:39 -0600560static int
561_odict_resize(PyODictObject *od) {
Eric Snow4c729182015-06-03 10:50:37 -0600562 Py_ssize_t size, i;
Eric Snow96c6af92015-05-29 22:21:39 -0600563 _ODictNode **fast_nodes, *node;
564
565 /* Initialize a new "fast nodes" table. */
566 size = ((PyDictObject *)od)->ma_keys->dk_size;
567 fast_nodes = PyMem_NEW(_ODictNode *, size);
568 if (fast_nodes == NULL) {
569 PyErr_NoMemory();
570 return -1;
571 }
572 for (i = 0; i < size; i++)
573 fast_nodes[i] = NULL;
574
575 /* Copy the current nodes into the table. */
Eric Snow96c6af92015-05-29 22:21:39 -0600576 _odict_FOREACH(od, node) {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200577 i = _odict_get_index_raw(od, _odictnode_KEY(node),
Victor Stinner742da042016-09-07 17:40:12 -0700578 _odictnode_HASH(node));
Eric Snow96c6af92015-05-29 22:21:39 -0600579 if (i < 0) {
Eric Snow96c6af92015-05-29 22:21:39 -0600580 PyMem_FREE(fast_nodes);
581 return -1;
582 }
583 fast_nodes[i] = node;
584 }
Eric Snow96c6af92015-05-29 22:21:39 -0600585
586 /* Replace the old fast nodes table. */
587 _odict_free_fast_nodes(od);
588 od->od_fast_nodes = fast_nodes;
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200589 od->od_fast_nodes_size = size;
590 od->od_resize_sentinel = ((PyDictObject *)od)->ma_keys;
Eric Snow96c6af92015-05-29 22:21:39 -0600591 return 0;
592}
593
594/* Return the index into the hash table, regardless of a valid node. */
595static Py_ssize_t
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200596_odict_get_index(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600597{
Eric Snow4c729182015-06-03 10:50:37 -0600598 PyDictKeysObject *keys;
Eric Snow96c6af92015-05-29 22:21:39 -0600599
600 assert(key != NULL);
Eric Snow4c729182015-06-03 10:50:37 -0600601 keys = ((PyDictObject *)od)->ma_keys;
602
603 /* Ensure od_fast_nodes and dk_entries are in sync. */
Serhiy Storchaka97f46db2015-11-06 12:00:03 +0200604 if (od->od_resize_sentinel != keys ||
605 od->od_fast_nodes_size != keys->dk_size) {
Eric Snow4c729182015-06-03 10:50:37 -0600606 int resize_res = _odict_resize(od);
607 if (resize_res < 0)
608 return -1;
609 }
610
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200611 return _odict_get_index_raw(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600612}
613
Eric Snow96c6af92015-05-29 22:21:39 -0600614/* Returns NULL if there was some error or the key was not found. */
615static _ODictNode *
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200616_odict_find_node_hash(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600617{
618 Py_ssize_t index;
619
620 if (_odict_EMPTY(od))
621 return NULL;
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200622 index = _odict_get_index(od, key, hash);
623 if (index < 0)
624 return NULL;
625 return od->od_fast_nodes[index];
626}
627
628static _ODictNode *
629_odict_find_node(PyODictObject *od, PyObject *key)
630{
631 Py_ssize_t index;
632 Py_hash_t hash;
633
634 if (_odict_EMPTY(od))
635 return NULL;
636 hash = PyObject_Hash(key);
637 if (hash == -1)
638 return NULL;
639 index = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600640 if (index < 0)
641 return NULL;
642 return od->od_fast_nodes[index];
643}
644
645static void
646_odict_add_head(PyODictObject *od, _ODictNode *node)
647{
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300648 _odictnode_PREV(node) = NULL;
649 _odictnode_NEXT(node) = _odict_FIRST(od);
650 if (_odict_FIRST(od) == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -0600651 _odict_LAST(od) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300652 else
Eric Snow96c6af92015-05-29 22:21:39 -0600653 _odictnode_PREV(_odict_FIRST(od)) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300654 _odict_FIRST(od) = node;
Eric Snow4fabf022015-06-04 00:09:56 -0600655 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600656}
657
658static void
659_odict_add_tail(PyODictObject *od, _ODictNode *node)
660{
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300661 _odictnode_PREV(node) = _odict_LAST(od);
662 _odictnode_NEXT(node) = NULL;
663 if (_odict_LAST(od) == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -0600664 _odict_FIRST(od) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300665 else
Eric Snow96c6af92015-05-29 22:21:39 -0600666 _odictnode_NEXT(_odict_LAST(od)) = node;
Serhiy Storchaka992ec462015-10-14 19:21:24 +0300667 _odict_LAST(od) = node;
Eric Snow4fabf022015-06-04 00:09:56 -0600668 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600669}
670
671/* adds the node to the end of the list */
672static int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200673_odict_add_new_node(PyODictObject *od, PyObject *key, Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600674{
675 Py_ssize_t i;
676 _ODictNode *node;
677
Serhiy Storchakad17427b2015-10-20 18:21:48 +0300678 Py_INCREF(key);
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200679 i = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600680 if (i < 0) {
681 if (!PyErr_Occurred())
682 PyErr_SetObject(PyExc_KeyError, key);
683 Py_DECREF(key);
684 return -1;
685 }
686 else if (od->od_fast_nodes[i] != NULL) {
687 /* We already have a node for the key so there's no need to add one. */
688 Py_DECREF(key);
689 return 0;
690 }
691
692 /* must not be added yet */
693 node = (_ODictNode *)PyMem_MALLOC(sizeof(_ODictNode));
694 if (node == NULL) {
695 Py_DECREF(key);
696 PyErr_NoMemory();
697 return -1;
698 }
699
700 _odictnode_KEY(node) = key;
Eric Snow4c729182015-06-03 10:50:37 -0600701 _odictnode_HASH(node) = hash;
Eric Snow96c6af92015-05-29 22:21:39 -0600702 _odict_add_tail(od, node);
703 od->od_fast_nodes[i] = node;
704 return 0;
705}
706
707/* Putting the decref after the free causes problems. */
708#define _odictnode_DEALLOC(node) \
709 do { \
710 Py_DECREF(_odictnode_KEY(node)); \
711 PyMem_FREE((void *)node); \
712 } while (0)
713
714/* Repeated calls on the same node are no-ops. */
715static void
716_odict_remove_node(PyODictObject *od, _ODictNode *node)
717{
718 if (_odict_FIRST(od) == node)
719 _odict_FIRST(od) = _odictnode_NEXT(node);
720 else if (_odictnode_PREV(node) != NULL)
721 _odictnode_NEXT(_odictnode_PREV(node)) = _odictnode_NEXT(node);
722
723 if (_odict_LAST(od) == node)
724 _odict_LAST(od) = _odictnode_PREV(node);
725 else if (_odictnode_NEXT(node) != NULL)
726 _odictnode_PREV(_odictnode_NEXT(node)) = _odictnode_PREV(node);
727
728 _odictnode_PREV(node) = NULL;
729 _odictnode_NEXT(node) = NULL;
Eric Snow4fabf022015-06-04 00:09:56 -0600730 od->od_state++;
Eric Snow96c6af92015-05-29 22:21:39 -0600731}
732
Eric Snow96c6af92015-05-29 22:21:39 -0600733/* If someone calls PyDict_DelItem() directly on an OrderedDict, we'll
734 get all sorts of problems here. In PyODict_DelItem we make sure to
735 call _odict_clear_node first.
Victor Stinnerca30b022015-09-03 17:50:04 +0200736
Eric Snow96c6af92015-05-29 22:21:39 -0600737 This matters in the case of colliding keys. Suppose we add 3 keys:
738 [A, B, C], where the hash of C collides with A and the next possible
739 index in the hash table is occupied by B. If we remove B then for C
740 the dict's looknode func will give us the old index of B instead of
741 the index we got before deleting B. However, the node for C in
742 od_fast_nodes is still at the old dict index of C. Thus to be sure
743 things don't get out of sync, we clear the node in od_fast_nodes
744 *before* calling PyDict_DelItem.
745
746 The same must be done for any other OrderedDict operations where
747 we modify od_fast_nodes.
748*/
749static int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200750_odict_clear_node(PyODictObject *od, _ODictNode *node, PyObject *key,
751 Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -0600752{
753 Py_ssize_t i;
754
755 assert(key != NULL);
756 if (_odict_EMPTY(od)) {
757 /* Let later code decide if this is a KeyError. */
758 return 0;
759 }
760
Serhiy Storchaka19a70e72015-11-13 14:48:36 +0200761 i = _odict_get_index(od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -0600762 if (i < 0)
763 return PyErr_Occurred() ? -1 : 0;
764
765 if (node == NULL)
766 node = od->od_fast_nodes[i];
767 assert(node == od->od_fast_nodes[i]);
768 if (node == NULL) {
769 /* Let later code decide if this is a KeyError. */
770 return 0;
771 }
772
773 // Now clear the node.
774 od->od_fast_nodes[i] = NULL;
775 _odict_remove_node(od, node);
776 _odictnode_DEALLOC(node);
777 return 0;
778}
779
780static void
781_odict_clear_nodes(PyODictObject *od)
782{
783 _ODictNode *node, *next;
784
Eric Snow96c6af92015-05-29 22:21:39 -0600785 _odict_free_fast_nodes(od);
786 od->od_fast_nodes = NULL;
Serhiy Storchakad205d012016-01-19 14:46:25 +0200787
788 node = _odict_FIRST(od);
789 _odict_FIRST(od) = NULL;
790 _odict_LAST(od) = NULL;
791 while (node != NULL) {
792 next = _odictnode_NEXT(node);
793 _odictnode_DEALLOC(node);
794 node = next;
795 }
Eric Snow96c6af92015-05-29 22:21:39 -0600796}
797
798/* There isn't any memory management of nodes past this point. */
799#undef _odictnode_DEALLOC
800
801static int
802_odict_keys_equal(PyODictObject *a, PyODictObject *b)
803{
804 _ODictNode *node_a, *node_b;
805
806 node_a = _odict_FIRST(a);
807 node_b = _odict_FIRST(b);
808 while (1) {
809 if (node_a == NULL && node_b == NULL)
810 /* success: hit the end of each at the same time */
811 return 1;
812 else if (node_a == NULL || node_b == NULL)
813 /* unequal length */
814 return 0;
815 else {
816 int res = PyObject_RichCompareBool(
817 (PyObject *)_odictnode_KEY(node_a),
818 (PyObject *)_odictnode_KEY(node_b),
819 Py_EQ);
820 if (res < 0)
821 return res;
822 else if (res == 0)
823 return 0;
824
825 /* otherwise it must match, so move on to the next one */
826 node_a = _odictnode_NEXT(node_a);
827 node_b = _odictnode_NEXT(node_b);
828 }
829 }
830}
831
832
833/* ----------------------------------------------
834 * OrderedDict mapping methods
835 */
836
837/* mp_ass_subscript: __setitem__() and __delitem__() */
838
839static int
840odict_mp_ass_sub(PyODictObject *od, PyObject *v, PyObject *w)
841{
842 if (w == NULL)
843 return PyODict_DelItem((PyObject *)od, v);
844 else
845 return PyODict_SetItem((PyObject *)od, v, w);
846}
847
848/* tp_as_mapping */
849
850static PyMappingMethods odict_as_mapping = {
851 0, /*mp_length*/
852 0, /*mp_subscript*/
853 (objobjargproc)odict_mp_ass_sub, /*mp_ass_subscript*/
854};
855
856
857/* ----------------------------------------------
858 * OrderedDict methods
859 */
860
861/* __delitem__() */
862
863PyDoc_STRVAR(odict_delitem__doc__, "od.__delitem__(y) <==> del od[y]");
864
865/* __eq__() */
866
867PyDoc_STRVAR(odict_eq__doc__,
868"od.__eq__(y) <==> od==y. Comparison to another OD is order-sensitive \n\
869 while comparison to a regular mapping is order-insensitive.\n\
870 ");
871
872/* forward */
873static PyObject * odict_richcompare(PyObject *v, PyObject *w, int op);
874
875static PyObject *
876odict_eq(PyObject *a, PyObject *b)
877{
878 return odict_richcompare(a, b, Py_EQ);
879}
880
881/* __init__() */
882
883PyDoc_STRVAR(odict_init__doc__,
884"Initialize an ordered dictionary. The signature is the same as\n\
885 regular dictionaries, but keyword arguments are not recommended because\n\
886 their insertion order is arbitrary.\n\
887\n\
888 ");
889
890/* forward */
891static int odict_init(PyObject *self, PyObject *args, PyObject *kwds);
892
893/* __iter__() */
894
895PyDoc_STRVAR(odict_iter__doc__, "od.__iter__() <==> iter(od)");
896
897static PyObject * odict_iter(PyODictObject *self); /* forward */
898
899/* __ne__() */
900
901/* Mapping.__ne__() does not have a docstring. */
902PyDoc_STRVAR(odict_ne__doc__, "");
903
904static PyObject *
905odict_ne(PyObject *a, PyObject *b)
906{
907 return odict_richcompare(a, b, Py_NE);
908}
909
910/* __repr__() */
911
912PyDoc_STRVAR(odict_repr__doc__, "od.__repr__() <==> repr(od)");
913
914static PyObject * odict_repr(PyODictObject *self); /* forward */
915
916/* __setitem__() */
917
918PyDoc_STRVAR(odict_setitem__doc__, "od.__setitem__(i, y) <==> od[i]=y");
919
920/* fromkeys() */
921
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100922/*[clinic input]
923@classmethod
924OrderedDict.fromkeys
925
926 iterable as seq: object
927 value: object = None
928
Serhiy Storchaka78d9e582017-01-25 00:30:04 +0200929Create a new ordered dictionary with keys from iterable and values set to value.
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100930[clinic start generated code]*/
Eric Snow96c6af92015-05-29 22:21:39 -0600931
932static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100933OrderedDict_fromkeys_impl(PyTypeObject *type, PyObject *seq, PyObject *value)
Serhiy Storchaka78d9e582017-01-25 00:30:04 +0200934/*[clinic end generated code: output=c10390d452d78d6d input=1a0476c229c597b3]*/
Eric Snow96c6af92015-05-29 22:21:39 -0600935{
Victor Stinnerb05cbac2017-01-17 03:46:13 +0100936 return _PyDict_FromKeys((PyObject *)type, seq, value);
Eric Snow96c6af92015-05-29 22:21:39 -0600937}
938
939/* __sizeof__() */
940
941/* OrderedDict.__sizeof__() does not have a docstring. */
942PyDoc_STRVAR(odict_sizeof__doc__, "");
943
944static PyObject *
945odict_sizeof(PyODictObject *od)
946{
Serhiy Storchaka0ce7a3a2015-12-22 08:16:18 +0200947 Py_ssize_t res = _PyDict_SizeOf((PyDictObject *)od);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300948 res += sizeof(_ODictNode *) * _odict_FAST_SIZE(od); /* od_fast_nodes */
Eric Snow96c6af92015-05-29 22:21:39 -0600949 if (!_odict_EMPTY(od)) {
950 res += sizeof(_ODictNode) * PyODict_SIZE(od); /* linked-list */
951 }
952 return PyLong_FromSsize_t(res);
953}
954
955/* __reduce__() */
956
957PyDoc_STRVAR(odict_reduce__doc__, "Return state information for pickling");
958
959static PyObject *
960odict_reduce(register PyODictObject *od)
961{
962 _Py_IDENTIFIER(__dict__);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300963 _Py_IDENTIFIER(items);
Serhiy Storchaka4575beb2015-10-22 20:18:24 +0300964 PyObject *dict = NULL, *result = NULL;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300965 PyObject *items_iter, *items, *args = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -0600966
967 /* capture any instance state */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300968 dict = _PyObject_GetAttrId((PyObject *)od, &PyId___dict__);
969 if (dict == NULL)
Eric Snowc5e59602015-05-30 11:28:56 -0600970 goto Done;
971 else {
Eric Snow96c6af92015-05-29 22:21:39 -0600972 /* od.__dict__ isn't necessarily a dict... */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300973 Py_ssize_t dict_len = PyObject_Length(dict);
974 if (dict_len == -1)
Eric Snow96c6af92015-05-29 22:21:39 -0600975 goto Done;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300976 if (!dict_len) {
977 /* nothing to pickle in od.__dict__ */
978 Py_CLEAR(dict);
Eric Snow96c6af92015-05-29 22:21:39 -0600979 }
980 }
981
982 /* build the result */
Eric Snow96c6af92015-05-29 22:21:39 -0600983 args = PyTuple_New(0);
984 if (args == NULL)
985 goto Done;
986
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300987 items = _PyObject_CallMethodIdObjArgs((PyObject *)od, &PyId_items, NULL);
Eric Snow96c6af92015-05-29 22:21:39 -0600988 if (items == NULL)
989 goto Done;
990
991 items_iter = PyObject_GetIter(items);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300992 Py_DECREF(items);
Eric Snow96c6af92015-05-29 22:21:39 -0600993 if (items_iter == NULL)
994 goto Done;
995
Serhiy Storchaka4575beb2015-10-22 20:18:24 +0300996 result = PyTuple_Pack(5, Py_TYPE(od), args, dict ? dict : Py_None, Py_None, items_iter);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +0300997 Py_DECREF(items_iter);
Eric Snow96c6af92015-05-29 22:21:39 -0600998
999Done:
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001000 Py_XDECREF(dict);
Eric Snow96c6af92015-05-29 22:21:39 -06001001 Py_XDECREF(args);
Eric Snow96c6af92015-05-29 22:21:39 -06001002
1003 return result;
1004}
1005
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001006/* setdefault(): Skips __missing__() calls. */
Eric Snow96c6af92015-05-29 22:21:39 -06001007
Eric Snow96c6af92015-05-29 22:21:39 -06001008
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001009/*[clinic input]
1010OrderedDict.setdefault
1011
1012 key: object
Serhiy Storchakaa70eaf22017-01-19 19:38:13 +02001013 default: object = None
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001014
Serhiy Storchaka78d9e582017-01-25 00:30:04 +02001015Insert key with a value of default if key is not in the dictionary.
1016
1017Return the value for key if key is in the dictionary, else default.
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001018[clinic start generated code]*/
1019
Eric Snow96c6af92015-05-29 22:21:39 -06001020static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001021OrderedDict_setdefault_impl(PyODictObject *self, PyObject *key,
Serhiy Storchakaa70eaf22017-01-19 19:38:13 +02001022 PyObject *default_value)
Serhiy Storchaka78d9e582017-01-25 00:30:04 +02001023/*[clinic end generated code: output=97537cb7c28464b6 input=38e098381c1efbc6]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001024{
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001025 PyObject *result = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001026
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001027 if (PyODict_CheckExact(self)) {
1028 result = PyODict_GetItemWithError(self, key); /* borrowed */
Eric Snowd1719752015-06-01 23:12:13 -06001029 if (result == NULL) {
1030 if (PyErr_Occurred())
1031 return NULL;
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001032 assert(_odict_find_node(self, key) == NULL);
Serhiy Storchakaa70eaf22017-01-19 19:38:13 +02001033 if (PyODict_SetItem((PyObject *)self, key, default_value) >= 0) {
1034 result = default_value;
1035 Py_INCREF(result);
Eric Snow96c6af92015-05-29 22:21:39 -06001036 }
1037 }
1038 else {
Eric Snowd1719752015-06-01 23:12:13 -06001039 Py_INCREF(result);
Eric Snow96c6af92015-05-29 22:21:39 -06001040 }
1041 }
1042 else {
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001043 int exists = PySequence_Contains((PyObject *)self, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001044 if (exists < 0) {
Eric Snowd1719752015-06-01 23:12:13 -06001045 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001046 }
1047 else if (exists) {
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001048 result = PyObject_GetItem((PyObject *)self, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001049 }
Serhiy Storchakaa70eaf22017-01-19 19:38:13 +02001050 else if (PyObject_SetItem((PyObject *)self, key, default_value) >= 0) {
1051 result = default_value;
1052 Py_INCREF(result);
Eric Snow96c6af92015-05-29 22:21:39 -06001053 }
1054 }
1055
Eric Snow96c6af92015-05-29 22:21:39 -06001056 return result;
1057}
1058
1059/* pop() */
1060
1061PyDoc_STRVAR(odict_pop__doc__,
1062"od.pop(k[,d]) -> v, remove specified key and return the corresponding\n\
1063 value. If key is not found, d is returned if given, otherwise KeyError\n\
1064 is raised.\n\
1065\n\
1066 ");
1067
1068/* forward */
1069static PyObject * _odict_popkey(PyObject *, PyObject *, PyObject *);
1070
1071/* Skips __missing__() calls. */
1072static PyObject *
Eric Snowac02ef32015-06-02 20:42:14 -06001073odict_pop(PyObject *od, PyObject *args, PyObject *kwargs)
Eric Snow96c6af92015-05-29 22:21:39 -06001074{
Eric Snowac02ef32015-06-02 20:42:14 -06001075 static char *kwlist[] = {"key", "default", 0};
Eric Snow96c6af92015-05-29 22:21:39 -06001076 PyObject *key, *failobj = NULL;
1077
Eric Snowac02ef32015-06-02 20:42:14 -06001078 /* borrowed */
1079 if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O|O:pop", kwlist,
1080 &key, &failobj)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001081 return NULL;
1082 }
1083
1084 return _odict_popkey(od, key, failobj);
1085}
1086
1087static PyObject *
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001088_odict_popkey_hash(PyObject *od, PyObject *key, PyObject *failobj,
1089 Py_hash_t hash)
Eric Snow96c6af92015-05-29 22:21:39 -06001090{
1091 _ODictNode *node;
1092 PyObject *value = NULL;
1093
Eric Snow96c6af92015-05-29 22:21:39 -06001094 /* Pop the node first to avoid a possible dict resize (due to
1095 eval loop reentrancy) and complications due to hash collision
1096 resolution. */
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001097 node = _odict_find_node_hash((PyODictObject *)od, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001098 if (node == NULL) {
1099 if (PyErr_Occurred())
Eric Snowd1719752015-06-01 23:12:13 -06001100 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001101 }
1102 else {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001103 int res = _odict_clear_node((PyODictObject *)od, node, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001104 if (res < 0) {
Eric Snowd1719752015-06-01 23:12:13 -06001105 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001106 }
1107 }
1108
1109 /* Now delete the value from the dict. */
Serhiy Storchaka04386832016-10-30 17:17:24 +02001110 if (PyODict_CheckExact(od)) {
1111 if (node != NULL) {
1112 value = _PyDict_GetItem_KnownHash(od, key, hash); /* borrowed */
1113 if (value != NULL) {
1114 Py_INCREF(value);
1115 if (_PyDict_DelItem_KnownHash(od, key, hash) < 0) {
1116 Py_DECREF(value);
1117 return NULL;
1118 }
1119 }
1120 }
1121 }
1122 else {
1123 int exists = PySequence_Contains(od, key);
1124 if (exists < 0)
1125 return NULL;
1126 if (exists) {
1127 value = PyObject_GetItem(od, key);
1128 if (value != NULL) {
1129 if (PyObject_DelItem(od, key) == -1) {
1130 Py_CLEAR(value);
1131 }
Eric Snow96c6af92015-05-29 22:21:39 -06001132 }
1133 }
1134 }
1135
1136 /* Apply the fallback value, if necessary. */
1137 if (value == NULL && !PyErr_Occurred()) {
1138 if (failobj) {
1139 value = failobj;
1140 Py_INCREF(failobj);
1141 }
1142 else {
1143 PyErr_SetObject(PyExc_KeyError, key);
1144 }
1145 }
1146
Eric Snow96c6af92015-05-29 22:21:39 -06001147 return value;
1148}
1149
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001150static PyObject *
1151_odict_popkey(PyObject *od, PyObject *key, PyObject *failobj)
1152{
1153 Py_hash_t hash = PyObject_Hash(key);
1154 if (hash == -1)
1155 return NULL;
1156
1157 return _odict_popkey_hash(od, key, failobj, hash);
1158}
1159
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001160
Eric Snow96c6af92015-05-29 22:21:39 -06001161/* popitem() */
1162
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001163/*[clinic input]
1164OrderedDict.popitem
1165
1166 last: bool = True
1167
Serhiy Storchaka78d9e582017-01-25 00:30:04 +02001168Remove and return a (key, value) pair from the dictionary.
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001169
1170Pairs are returned in LIFO order if last is true or FIFO order if false.
1171[clinic start generated code]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001172
1173static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001174OrderedDict_popitem_impl(PyODictObject *self, int last)
Serhiy Storchaka78d9e582017-01-25 00:30:04 +02001175/*[clinic end generated code: output=98e7d986690d49eb input=d992ac5ee8305e1a]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001176{
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001177 PyObject *key, *value, *item = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001178 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001179
1180 /* pull the item */
1181
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001182 if (_odict_EMPTY(self)) {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001183 PyErr_SetString(PyExc_KeyError, "dictionary is empty");
1184 return NULL;
Eric Snowac02ef32015-06-02 20:42:14 -06001185 }
Eric Snow96c6af92015-05-29 22:21:39 -06001186
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001187 node = last ? _odict_LAST(self) : _odict_FIRST(self);
Eric Snow96c6af92015-05-29 22:21:39 -06001188 key = _odictnode_KEY(node);
Eric Snowd1719752015-06-01 23:12:13 -06001189 Py_INCREF(key);
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001190 value = _odict_popkey_hash((PyObject *)self, key, NULL, _odictnode_HASH(node));
Eric Snow96c6af92015-05-29 22:21:39 -06001191 if (value == NULL)
1192 return NULL;
1193 item = PyTuple_Pack(2, key, value);
Eric Snowd1719752015-06-01 23:12:13 -06001194 Py_DECREF(key);
Eric Snow96c6af92015-05-29 22:21:39 -06001195 Py_DECREF(value);
1196 return item;
1197}
1198
1199/* keys() */
1200
1201/* MutableMapping.keys() does not have a docstring. */
1202PyDoc_STRVAR(odict_keys__doc__, "");
1203
1204static PyObject * odictkeys_new(PyObject *od); /* forward */
1205
1206/* values() */
1207
1208/* MutableMapping.values() does not have a docstring. */
1209PyDoc_STRVAR(odict_values__doc__, "");
1210
1211static PyObject * odictvalues_new(PyObject *od); /* forward */
1212
1213/* items() */
1214
1215/* MutableMapping.items() does not have a docstring. */
1216PyDoc_STRVAR(odict_items__doc__, "");
1217
1218static PyObject * odictitems_new(PyObject *od); /* forward */
1219
1220/* update() */
1221
1222/* MutableMapping.update() does not have a docstring. */
1223PyDoc_STRVAR(odict_update__doc__, "");
1224
1225/* forward */
1226static PyObject * mutablemapping_update(PyObject *, PyObject *, PyObject *);
1227
1228#define odict_update mutablemapping_update
1229
1230/* clear() */
1231
1232PyDoc_STRVAR(odict_clear__doc__,
1233 "od.clear() -> None. Remove all items from od.");
1234
1235static PyObject *
1236odict_clear(register PyODictObject *od)
1237{
1238 PyDict_Clear((PyObject *)od);
1239 _odict_clear_nodes(od);
Eric Snow96c6af92015-05-29 22:21:39 -06001240 if (_odict_resize(od) < 0)
1241 return NULL;
1242 Py_RETURN_NONE;
1243}
1244
1245/* copy() */
1246
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001247/* forward */
1248static int _PyODict_SetItem_KnownHash(PyObject *, PyObject *, PyObject *,
1249 Py_hash_t);
1250
Eric Snow96c6af92015-05-29 22:21:39 -06001251PyDoc_STRVAR(odict_copy__doc__, "od.copy() -> a shallow copy of od");
1252
1253static PyObject *
1254odict_copy(register PyODictObject *od)
1255{
1256 _ODictNode *node;
1257 PyObject *od_copy;
1258
1259 if (PyODict_CheckExact(od))
1260 od_copy = PyODict_New();
1261 else
Victor Stinnerf17c3de2016-12-06 18:46:19 +01001262 od_copy = _PyObject_CallNoArg((PyObject *)Py_TYPE(od));
Eric Snow96c6af92015-05-29 22:21:39 -06001263 if (od_copy == NULL)
1264 return NULL;
1265
1266 if (PyODict_CheckExact(od)) {
1267 _odict_FOREACH(od, node) {
Eric Snowa762af72015-06-01 22:59:08 -06001268 PyObject *key = _odictnode_KEY(node);
1269 PyObject *value = _odictnode_VALUE(node, od);
1270 if (value == NULL) {
1271 if (!PyErr_Occurred())
1272 PyErr_SetObject(PyExc_KeyError, key);
1273 goto fail;
1274 }
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001275 if (_PyODict_SetItem_KnownHash((PyObject *)od_copy, key, value,
1276 _odictnode_HASH(node)) != 0)
Eric Snow96c6af92015-05-29 22:21:39 -06001277 goto fail;
1278 }
1279 }
1280 else {
1281 _odict_FOREACH(od, node) {
1282 int res;
1283 PyObject *value = PyObject_GetItem((PyObject *)od,
1284 _odictnode_KEY(node));
1285 if (value == NULL)
1286 goto fail;
1287 res = PyObject_SetItem((PyObject *)od_copy,
1288 _odictnode_KEY(node), value);
1289 Py_DECREF(value);
1290 if (res != 0)
1291 goto fail;
1292 }
1293 }
1294 return od_copy;
1295
1296fail:
1297 Py_DECREF(od_copy);
1298 return NULL;
1299}
1300
1301/* __reversed__() */
1302
1303PyDoc_STRVAR(odict_reversed__doc__, "od.__reversed__() <==> reversed(od)");
1304
1305#define _odict_ITER_REVERSED 1
1306#define _odict_ITER_KEYS 2
1307#define _odict_ITER_VALUES 4
1308
1309/* forward */
1310static PyObject * odictiter_new(PyODictObject *, int);
1311
1312static PyObject *
1313odict_reversed(PyODictObject *od)
1314{
Eric Snow96c6af92015-05-29 22:21:39 -06001315 return odictiter_new(od, _odict_ITER_KEYS|_odict_ITER_REVERSED);
1316}
1317
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001318
Eric Snow96c6af92015-05-29 22:21:39 -06001319/* move_to_end() */
1320
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001321/*[clinic input]
1322OrderedDict.move_to_end
1323
1324 key: object
1325 last: bool = True
1326
Serhiy Storchaka78d9e582017-01-25 00:30:04 +02001327Move an existing element to the end (or beginning if last is false).
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001328
Serhiy Storchaka78d9e582017-01-25 00:30:04 +02001329Raise KeyError if the element does not exist.
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001330[clinic start generated code]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001331
1332static PyObject *
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001333OrderedDict_move_to_end_impl(PyODictObject *self, PyObject *key, int last)
Serhiy Storchaka78d9e582017-01-25 00:30:04 +02001334/*[clinic end generated code: output=fafa4c5cc9b92f20 input=d6ceff7132a2fcd7]*/
Eric Snow96c6af92015-05-29 22:21:39 -06001335{
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001336 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001337
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001338 if (_odict_EMPTY(self)) {
Eric Snow96c6af92015-05-29 22:21:39 -06001339 PyErr_SetObject(PyExc_KeyError, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001340 return NULL;
1341 }
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001342 node = last ? _odict_LAST(self) : _odict_FIRST(self);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001343 if (key != _odictnode_KEY(node)) {
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001344 node = _odict_find_node(self, key);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001345 if (node == NULL) {
1346 if (!PyErr_Occurred())
1347 PyErr_SetObject(PyExc_KeyError, key);
Eric Snowc5e59602015-05-30 11:28:56 -06001348 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001349 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001350 if (last) {
1351 /* Only move if not already the last one. */
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001352 if (node != _odict_LAST(self)) {
1353 _odict_remove_node(self, node);
1354 _odict_add_tail(self, node);
Eric Snow96c6af92015-05-29 22:21:39 -06001355 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001356 }
1357 else {
1358 /* Only move if not already the first one. */
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001359 if (node != _odict_FIRST(self)) {
1360 _odict_remove_node(self, node);
1361 _odict_add_head(self, node);
Eric Snow96c6af92015-05-29 22:21:39 -06001362 }
1363 }
1364 }
Eric Snow96c6af92015-05-29 22:21:39 -06001365 Py_RETURN_NONE;
1366}
1367
1368
1369/* tp_methods */
1370
1371static PyMethodDef odict_methods[] = {
1372
1373 /* explicitly defined so we can align docstrings with
1374 * collections.OrderedDict */
1375 {"__delitem__", (PyCFunction)odict_mp_ass_sub, METH_NOARGS,
1376 odict_delitem__doc__},
1377 {"__eq__", (PyCFunction)odict_eq, METH_NOARGS,
1378 odict_eq__doc__},
1379 {"__init__", (PyCFunction)odict_init, METH_NOARGS,
1380 odict_init__doc__},
1381 {"__iter__", (PyCFunction)odict_iter, METH_NOARGS,
1382 odict_iter__doc__},
1383 {"__ne__", (PyCFunction)odict_ne, METH_NOARGS,
1384 odict_ne__doc__},
1385 {"__repr__", (PyCFunction)odict_repr, METH_NOARGS,
1386 odict_repr__doc__},
1387 {"__setitem__", (PyCFunction)odict_mp_ass_sub, METH_NOARGS,
1388 odict_setitem__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001389 ORDEREDDICT_FROMKEYS_METHODDEF
Eric Snow96c6af92015-05-29 22:21:39 -06001390
1391 /* overridden dict methods */
1392 {"__sizeof__", (PyCFunction)odict_sizeof, METH_NOARGS,
1393 odict_sizeof__doc__},
1394 {"__reduce__", (PyCFunction)odict_reduce, METH_NOARGS,
1395 odict_reduce__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001396 ORDEREDDICT_SETDEFAULT_METHODDEF
Eric Snowac02ef32015-06-02 20:42:14 -06001397 {"pop", (PyCFunction)odict_pop,
1398 METH_VARARGS | METH_KEYWORDS, odict_pop__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001399 ORDEREDDICT_POPITEM_METHODDEF
Eric Snow96c6af92015-05-29 22:21:39 -06001400 {"keys", (PyCFunction)odictkeys_new, METH_NOARGS,
1401 odict_keys__doc__},
1402 {"values", (PyCFunction)odictvalues_new, METH_NOARGS,
1403 odict_values__doc__},
1404 {"items", (PyCFunction)odictitems_new, METH_NOARGS,
1405 odict_items__doc__},
1406 {"update", (PyCFunction)odict_update, METH_VARARGS | METH_KEYWORDS,
1407 odict_update__doc__},
1408 {"clear", (PyCFunction)odict_clear, METH_NOARGS,
1409 odict_clear__doc__},
1410 {"copy", (PyCFunction)odict_copy, METH_NOARGS,
1411 odict_copy__doc__},
1412
1413 /* new methods */
1414 {"__reversed__", (PyCFunction)odict_reversed, METH_NOARGS,
1415 odict_reversed__doc__},
Victor Stinnerb05cbac2017-01-17 03:46:13 +01001416 ORDEREDDICT_MOVE_TO_END_METHODDEF
Eric Snow96c6af92015-05-29 22:21:39 -06001417
1418 {NULL, NULL} /* sentinel */
1419};
1420
1421
1422/* ----------------------------------------------
1423 * OrderedDict members
1424 */
1425
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001426/* tp_getset */
Eric Snow96c6af92015-05-29 22:21:39 -06001427
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001428static PyGetSetDef odict_getset[] = {
1429 {"__dict__", PyObject_GenericGetDict, PyObject_GenericSetDict},
1430 {NULL}
Eric Snow96c6af92015-05-29 22:21:39 -06001431};
1432
Eric Snow96c6af92015-05-29 22:21:39 -06001433/* ----------------------------------------------
1434 * OrderedDict type slot methods
1435 */
1436
1437/* tp_dealloc */
1438
1439static void
1440odict_dealloc(PyODictObject *self)
1441{
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001442 PyThreadState *tstate = PyThreadState_GET();
1443
Eric Snow96c6af92015-05-29 22:21:39 -06001444 PyObject_GC_UnTrack(self);
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001445 Py_TRASHCAN_SAFE_BEGIN(self)
1446
Eric Snow96c6af92015-05-29 22:21:39 -06001447 Py_XDECREF(self->od_inst_dict);
1448 if (self->od_weakreflist != NULL)
1449 PyObject_ClearWeakRefs((PyObject *)self);
1450
1451 _odict_clear_nodes(self);
Eric Snow96c6af92015-05-29 22:21:39 -06001452
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001453 /* Call the base tp_dealloc(). Since it too uses the trashcan mechanism,
1454 * temporarily decrement trash_delete_nesting to prevent triggering it
1455 * and putting the partially deallocated object on the trashcan's
1456 * to-be-deleted-later list.
1457 */
1458 --tstate->trash_delete_nesting;
1459 assert(_tstate->trash_delete_nesting < PyTrash_UNWIND_LEVEL);
Eric Snow96c6af92015-05-29 22:21:39 -06001460 PyDict_Type.tp_dealloc((PyObject *)self);
Serhiy Storchaka14eefe32015-11-01 16:12:34 +02001461 ++tstate->trash_delete_nesting;
1462
1463 Py_TRASHCAN_SAFE_END(self)
Victor Stinnere1871952016-06-08 10:18:18 +02001464}
Eric Snow96c6af92015-05-29 22:21:39 -06001465
1466/* tp_repr */
1467
1468static PyObject *
1469odict_repr(PyODictObject *self)
1470{
1471 int i;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001472 _Py_IDENTIFIER(items);
Serhiy Storchaka4575beb2015-10-22 20:18:24 +03001473 PyObject *pieces = NULL, *result = NULL;
1474 const char *classname;
1475
1476 classname = strrchr(Py_TYPE(self)->tp_name, '.');
1477 if (classname == NULL)
1478 classname = Py_TYPE(self)->tp_name;
1479 else
1480 classname++;
1481
1482 if (PyODict_SIZE(self) == 0)
1483 return PyUnicode_FromFormat("%s()", classname);
Eric Snow96c6af92015-05-29 22:21:39 -06001484
1485 i = Py_ReprEnter((PyObject *)self);
1486 if (i != 0) {
1487 return i > 0 ? PyUnicode_FromString("...") : NULL;
1488 }
1489
Eric Snow96c6af92015-05-29 22:21:39 -06001490 if (PyODict_CheckExact(self)) {
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001491 Py_ssize_t count = 0;
Eric Snowa762af72015-06-01 22:59:08 -06001492 _ODictNode *node;
Eric Snow96c6af92015-05-29 22:21:39 -06001493 pieces = PyList_New(PyODict_SIZE(self));
1494 if (pieces == NULL)
1495 goto Done;
1496
1497 _odict_FOREACH(self, node) {
Eric Snowa762af72015-06-01 22:59:08 -06001498 PyObject *pair;
1499 PyObject *key = _odictnode_KEY(node);
1500 PyObject *value = _odictnode_VALUE(node, self);
1501 if (value == NULL) {
1502 if (!PyErr_Occurred())
1503 PyErr_SetObject(PyExc_KeyError, key);
Eric Snowdb4061c2015-06-03 11:09:48 -06001504 goto Done;
Eric Snowa762af72015-06-01 22:59:08 -06001505 }
1506 pair = PyTuple_Pack(2, key, value);
Eric Snow96c6af92015-05-29 22:21:39 -06001507 if (pair == NULL)
1508 goto Done;
1509
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001510 if (count < PyList_GET_SIZE(pieces))
1511 PyList_SET_ITEM(pieces, count, pair); /* steals reference */
1512 else {
1513 if (PyList_Append(pieces, pair) < 0) {
1514 Py_DECREF(pair);
1515 goto Done;
1516 }
1517 Py_DECREF(pair);
1518 }
1519 count++;
Eric Snow96c6af92015-05-29 22:21:39 -06001520 }
Serhiy Storchaka710cd342015-11-04 22:33:07 +02001521 if (count < PyList_GET_SIZE(pieces))
1522 PyList_GET_SIZE(pieces) = count;
Eric Snow96c6af92015-05-29 22:21:39 -06001523 }
1524 else {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001525 PyObject *items = _PyObject_CallMethodIdObjArgs((PyObject *)self,
1526 &PyId_items, NULL);
Eric Snow96c6af92015-05-29 22:21:39 -06001527 if (items == NULL)
1528 goto Done;
1529 pieces = PySequence_List(items);
1530 Py_DECREF(items);
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03001531 if (pieces == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -06001532 goto Done;
1533 }
1534
Serhiy Storchaka4575beb2015-10-22 20:18:24 +03001535 result = PyUnicode_FromFormat("%s(%R)", classname, pieces);
Eric Snow96c6af92015-05-29 22:21:39 -06001536
Eric Snow96c6af92015-05-29 22:21:39 -06001537Done:
1538 Py_XDECREF(pieces);
Eric Snow96c6af92015-05-29 22:21:39 -06001539 Py_ReprLeave((PyObject *)self);
1540 return result;
Victor Stinnere1871952016-06-08 10:18:18 +02001541}
Eric Snow96c6af92015-05-29 22:21:39 -06001542
1543/* tp_doc */
1544
1545PyDoc_STRVAR(odict_doc,
1546 "Dictionary that remembers insertion order");
1547
1548/* tp_traverse */
1549
1550static int
1551odict_traverse(PyODictObject *od, visitproc visit, void *arg)
1552{
Serhiy Storchakad205d012016-01-19 14:46:25 +02001553 _ODictNode *node;
1554
Eric Snow96c6af92015-05-29 22:21:39 -06001555 Py_VISIT(od->od_inst_dict);
1556 Py_VISIT(od->od_weakreflist);
Serhiy Storchakad205d012016-01-19 14:46:25 +02001557 _odict_FOREACH(od, node) {
1558 Py_VISIT(_odictnode_KEY(node));
1559 }
Eric Snow96c6af92015-05-29 22:21:39 -06001560 return PyDict_Type.tp_traverse((PyObject *)od, visit, arg);
1561}
1562
1563/* tp_clear */
1564
1565static int
1566odict_tp_clear(PyODictObject *od)
1567{
1568 PyObject *res;
1569 Py_CLEAR(od->od_inst_dict);
1570 Py_CLEAR(od->od_weakreflist);
1571 res = odict_clear(od);
1572 if (res == NULL)
1573 return -1;
1574 Py_DECREF(res);
1575 return 0;
1576}
1577
1578/* tp_richcompare */
1579
1580static PyObject *
1581odict_richcompare(PyObject *v, PyObject *w, int op)
1582{
1583 if (!PyODict_Check(v) || !PyDict_Check(w)) {
1584 Py_RETURN_NOTIMPLEMENTED;
1585 }
1586
1587 if (op == Py_EQ || op == Py_NE) {
1588 PyObject *res, *cmp;
1589 int eq;
1590
1591 cmp = PyDict_Type.tp_richcompare(v, w, op);
1592 if (cmp == NULL)
1593 return NULL;
1594 if (!PyODict_Check(w))
1595 return cmp;
1596 if (op == Py_EQ && cmp == Py_False)
1597 return cmp;
1598 if (op == Py_NE && cmp == Py_True)
1599 return cmp;
1600 Py_DECREF(cmp);
1601
1602 /* Try comparing odict keys. */
1603 eq = _odict_keys_equal((PyODictObject *)v, (PyODictObject *)w);
1604 if (eq < 0)
1605 return NULL;
1606
1607 res = (eq == (op == Py_EQ)) ? Py_True : Py_False;
1608 Py_INCREF(res);
1609 return res;
1610 } else {
1611 Py_RETURN_NOTIMPLEMENTED;
1612 }
Victor Stinnere1871952016-06-08 10:18:18 +02001613}
Eric Snow96c6af92015-05-29 22:21:39 -06001614
1615/* tp_iter */
1616
1617static PyObject *
1618odict_iter(PyODictObject *od)
1619{
1620 return odictiter_new(od, _odict_ITER_KEYS);
Victor Stinnere1871952016-06-08 10:18:18 +02001621}
Eric Snow96c6af92015-05-29 22:21:39 -06001622
1623/* tp_init */
1624
1625static int
1626odict_init(PyObject *self, PyObject *args, PyObject *kwds)
1627{
1628 PyObject *res;
1629 Py_ssize_t len = PyObject_Length(args);
1630
1631 if (len == -1)
1632 return -1;
1633 if (len > 1) {
1634 char *msg = "expected at most 1 arguments, got %d";
1635 PyErr_Format(PyExc_TypeError, msg, len);
1636 return -1;
1637 }
1638
1639 /* __init__() triggering update() is just the way things are! */
1640 res = odict_update(self, args, kwds);
1641 if (res == NULL) {
1642 return -1;
1643 } else {
1644 Py_DECREF(res);
1645 return 0;
1646 }
Victor Stinnere1871952016-06-08 10:18:18 +02001647}
Eric Snow96c6af92015-05-29 22:21:39 -06001648
1649/* tp_new */
1650
1651static PyObject *
1652odict_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
1653{
Victor Stinnerca30b022015-09-03 17:50:04 +02001654 PyODictObject *od;
1655
Victor Stinnerca30b022015-09-03 17:50:04 +02001656 od = (PyODictObject *)PyDict_Type.tp_new(type, args, kwds);
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001657 if (od == NULL)
Victor Stinnerca30b022015-09-03 17:50:04 +02001658 return NULL;
Victor Stinnerca30b022015-09-03 17:50:04 +02001659
Victor Stinnerca30b022015-09-03 17:50:04 +02001660 /* type constructor fills the memory with zeros (see
1661 PyType_GenericAlloc()), there is no need to set them to zero again */
1662 if (_odict_resize(od) < 0) {
1663 Py_DECREF(od);
1664 return NULL;
1665 }
1666
1667 return (PyObject*)od;
Eric Snow96c6af92015-05-29 22:21:39 -06001668}
1669
1670/* PyODict_Type */
1671
1672PyTypeObject PyODict_Type = {
1673 PyVarObject_HEAD_INIT(&PyType_Type, 0)
1674 "collections.OrderedDict", /* tp_name */
1675 sizeof(PyODictObject), /* tp_basicsize */
1676 0, /* tp_itemsize */
1677 (destructor)odict_dealloc, /* tp_dealloc */
1678 0, /* tp_print */
1679 0, /* tp_getattr */
1680 0, /* tp_setattr */
1681 0, /* tp_reserved */
1682 (reprfunc)odict_repr, /* tp_repr */
1683 0, /* tp_as_number */
1684 0, /* tp_as_sequence */
1685 &odict_as_mapping, /* tp_as_mapping */
1686 0, /* tp_hash */
1687 0, /* tp_call */
1688 0, /* tp_str */
1689 0, /* tp_getattro */
1690 0, /* tp_setattro */
1691 0, /* tp_as_buffer */
1692 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,/* tp_flags */
1693 odict_doc, /* tp_doc */
1694 (traverseproc)odict_traverse, /* tp_traverse */
1695 (inquiry)odict_tp_clear, /* tp_clear */
1696 (richcmpfunc)odict_richcompare, /* tp_richcompare */
1697 offsetof(PyODictObject, od_weakreflist), /* tp_weaklistoffset */
1698 (getiterfunc)odict_iter, /* tp_iter */
1699 0, /* tp_iternext */
1700 odict_methods, /* tp_methods */
Serhiy Storchakad2962f12016-02-08 16:39:05 +02001701 0, /* tp_members */
1702 odict_getset, /* tp_getset */
Eric Snow96c6af92015-05-29 22:21:39 -06001703 &PyDict_Type, /* tp_base */
1704 0, /* tp_dict */
1705 0, /* tp_descr_get */
1706 0, /* tp_descr_set */
1707 offsetof(PyODictObject, od_inst_dict), /* tp_dictoffset */
1708 (initproc)odict_init, /* tp_init */
1709 PyType_GenericAlloc, /* tp_alloc */
1710 (newfunc)odict_new, /* tp_new */
1711 0, /* tp_free */
1712};
1713
1714
1715/* ----------------------------------------------
1716 * the public OrderedDict API
1717 */
1718
1719PyObject *
1720PyODict_New(void) {
1721 return odict_new(&PyODict_Type, NULL, NULL);
Victor Stinnere1871952016-06-08 10:18:18 +02001722}
Eric Snow96c6af92015-05-29 22:21:39 -06001723
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001724static int
1725_PyODict_SetItem_KnownHash(PyObject *od, PyObject *key, PyObject *value,
1726 Py_hash_t hash)
1727{
1728 int res = _PyDict_SetItem_KnownHash(od, key, value, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001729 if (res == 0) {
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001730 res = _odict_add_new_node((PyODictObject *)od, key, hash);
Serhiy Storchakad5f353e2015-11-06 11:07:11 +02001731 if (res < 0) {
1732 /* Revert setting the value on the dict */
1733 PyObject *exc, *val, *tb;
1734 PyErr_Fetch(&exc, &val, &tb);
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001735 (void) _PyDict_DelItem_KnownHash(od, key, hash);
Serhiy Storchakad5f353e2015-11-06 11:07:11 +02001736 _PyErr_ChainExceptions(exc, val, tb);
1737 }
Eric Snow96c6af92015-05-29 22:21:39 -06001738 }
1739 return res;
Victor Stinnere1871952016-06-08 10:18:18 +02001740}
Eric Snow96c6af92015-05-29 22:21:39 -06001741
1742int
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001743PyODict_SetItem(PyObject *od, PyObject *key, PyObject *value)
1744{
1745 Py_hash_t hash = PyObject_Hash(key);
1746 if (hash == -1)
1747 return -1;
1748 return _PyODict_SetItem_KnownHash(od, key, value, hash);
Victor Stinnere1871952016-06-08 10:18:18 +02001749}
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001750
1751int
1752PyODict_DelItem(PyObject *od, PyObject *key)
1753{
1754 int res;
1755 Py_hash_t hash = PyObject_Hash(key);
1756 if (hash == -1)
1757 return -1;
1758 res = _odict_clear_node((PyODictObject *)od, NULL, key, hash);
Eric Snow96c6af92015-05-29 22:21:39 -06001759 if (res < 0)
1760 return -1;
Serhiy Storchaka19a70e72015-11-13 14:48:36 +02001761 return _PyDict_DelItem_KnownHash(od, key, hash);
Victor Stinnere1871952016-06-08 10:18:18 +02001762}
Eric Snow96c6af92015-05-29 22:21:39 -06001763
1764
1765/* -------------------------------------------
1766 * The OrderedDict views (keys/values/items)
1767 */
1768
1769typedef struct {
1770 PyObject_HEAD
1771 int kind;
1772 PyODictObject *di_odict;
Eric Snowb952ab42015-06-01 23:35:13 -06001773 Py_ssize_t di_size;
Eric Snow4fabf022015-06-04 00:09:56 -06001774 size_t di_state;
Eric Snow96c6af92015-05-29 22:21:39 -06001775 PyObject *di_current;
1776 PyObject *di_result; /* reusable result tuple for iteritems */
1777} odictiterobject;
1778
1779static void
1780odictiter_dealloc(odictiterobject *di)
1781{
1782 _PyObject_GC_UNTRACK(di);
Eric Snowa762af72015-06-01 22:59:08 -06001783 Py_XDECREF(di->di_odict);
Eric Snow96c6af92015-05-29 22:21:39 -06001784 Py_XDECREF(di->di_current);
1785 if (di->kind & (_odict_ITER_KEYS | _odict_ITER_VALUES)) {
1786 Py_DECREF(di->di_result);
1787 }
1788 PyObject_GC_Del(di);
1789}
1790
1791static int
1792odictiter_traverse(odictiterobject *di, visitproc visit, void *arg)
1793{
1794 Py_VISIT(di->di_odict);
1795 Py_VISIT(di->di_current); /* A key could be any type, not just str. */
1796 Py_VISIT(di->di_result);
1797 return 0;
1798}
1799
Eric Snowa762af72015-06-01 22:59:08 -06001800/* In order to protect against modifications during iteration, we track
1801 * the current key instead of the current node. */
Eric Snow96c6af92015-05-29 22:21:39 -06001802static PyObject *
1803odictiter_nextkey(odictiterobject *di)
1804{
Eric Snowa762af72015-06-01 22:59:08 -06001805 PyObject *key = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001806 _ODictNode *node;
1807 int reversed = di->kind & _odict_ITER_REVERSED;
1808
Eric Snowa762af72015-06-01 22:59:08 -06001809 if (di->di_odict == NULL)
Eric Snow96c6af92015-05-29 22:21:39 -06001810 return NULL;
Eric Snowa762af72015-06-01 22:59:08 -06001811 if (di->di_current == NULL)
1812 goto done; /* We're already done. */
1813
Eric Snowb952ab42015-06-01 23:35:13 -06001814 /* Check for unsupported changes. */
Eric Snow4fabf022015-06-04 00:09:56 -06001815 if (di->di_odict->od_state != di->di_state) {
1816 PyErr_SetString(PyExc_RuntimeError,
1817 "OrderedDict mutated during iteration");
1818 goto done;
1819 }
Eric Snowb952ab42015-06-01 23:35:13 -06001820 if (di->di_size != PyODict_SIZE(di->di_odict)) {
1821 PyErr_SetString(PyExc_RuntimeError,
1822 "OrderedDict changed size during iteration");
1823 di->di_size = -1; /* Make this state sticky */
1824 return NULL;
1825 }
1826
Eric Snowa762af72015-06-01 22:59:08 -06001827 /* Get the key. */
Eric Snow96c6af92015-05-29 22:21:39 -06001828 node = _odict_find_node(di->di_odict, di->di_current);
1829 if (node == NULL) {
Serhiy Storchakab45b7b22015-11-04 22:05:38 +02001830 if (!PyErr_Occurred())
1831 PyErr_SetObject(PyExc_KeyError, di->di_current);
Eric Snow96c6af92015-05-29 22:21:39 -06001832 /* Must have been deleted. */
Eric Snowe3dfa9e2015-05-30 12:51:15 -06001833 Py_CLEAR(di->di_current);
Eric Snow96c6af92015-05-29 22:21:39 -06001834 return NULL;
1835 }
1836 key = di->di_current;
1837
1838 /* Advance to the next key. */
1839 node = reversed ? _odictnode_PREV(node) : _odictnode_NEXT(node);
1840 if (node == NULL) {
1841 /* Reached the end. */
1842 di->di_current = NULL;
1843 }
1844 else {
1845 di->di_current = _odictnode_KEY(node);
1846 Py_INCREF(di->di_current);
1847 }
1848
1849 return key;
Eric Snowa762af72015-06-01 22:59:08 -06001850
1851done:
1852 Py_CLEAR(di->di_odict);
1853 return key;
Eric Snow96c6af92015-05-29 22:21:39 -06001854}
1855
1856static PyObject *
1857odictiter_iternext(odictiterobject *di)
1858{
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001859 PyObject *result, *value;
Eric Snow96c6af92015-05-29 22:21:39 -06001860 PyObject *key = odictiter_nextkey(di); /* new reference */
1861
1862 if (key == NULL)
1863 return NULL;
1864
1865 /* Handle the keys case. */
1866 if (! (di->kind & _odict_ITER_VALUES)) {
1867 return key;
1868 }
1869
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001870 value = PyODict_GetItem((PyObject *)di->di_odict, key); /* borrowed */
1871 if (value == NULL) {
1872 if (!PyErr_Occurred())
1873 PyErr_SetObject(PyExc_KeyError, key);
Eric Snow96c6af92015-05-29 22:21:39 -06001874 Py_DECREF(key);
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001875 goto done;
1876 }
1877 Py_INCREF(value);
1878
1879 /* Handle the values case. */
1880 if (!(di->kind & _odict_ITER_KEYS)) {
1881 Py_DECREF(key);
Eric Snow96c6af92015-05-29 22:21:39 -06001882 return value;
1883 }
Eric Snowa762af72015-06-01 22:59:08 -06001884
Serhiy Storchaka9c967612015-11-06 10:39:51 +02001885 /* Handle the items case. */
1886 result = di->di_result;
1887
1888 if (Py_REFCNT(result) == 1) {
1889 /* not in use so we can reuse it
1890 * (the common case during iteration) */
1891 Py_INCREF(result);
1892 Py_DECREF(PyTuple_GET_ITEM(result, 0)); /* borrowed */
1893 Py_DECREF(PyTuple_GET_ITEM(result, 1)); /* borrowed */
1894 }
1895 else {
1896 result = PyTuple_New(2);
1897 if (result == NULL) {
1898 Py_DECREF(key);
1899 Py_DECREF(value);
1900 goto done;
1901 }
1902 }
1903
1904 PyTuple_SET_ITEM(result, 0, key); /* steals reference */
1905 PyTuple_SET_ITEM(result, 1, value); /* steals reference */
1906 return result;
1907
Eric Snowa762af72015-06-01 22:59:08 -06001908done:
1909 Py_CLEAR(di->di_current);
1910 Py_CLEAR(di->di_odict);
1911 return NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06001912}
1913
1914/* No need for tp_clear because odictiterobject is not mutable. */
1915
1916PyDoc_STRVAR(reduce_doc, "Return state information for pickling");
1917
1918static PyObject *
1919odictiter_reduce(odictiterobject *di)
1920{
Eric Snowc5e59602015-05-30 11:28:56 -06001921 PyObject *list, *iter;
Eric Snow96c6af92015-05-29 22:21:39 -06001922
1923 list = PyList_New(0);
1924 if (!list)
1925 return NULL;
1926
1927 /* iterate the temporary into a list */
1928 for(;;) {
1929 PyObject *element = odictiter_iternext(di);
1930 if (element) {
1931 if (PyList_Append(list, element)) {
1932 Py_DECREF(element);
1933 Py_DECREF(list);
1934 return NULL;
1935 }
1936 Py_DECREF(element);
1937 }
1938 else {
1939 /* done iterating? */
1940 break;
1941 }
1942 }
1943 if (PyErr_Occurred()) {
1944 Py_DECREF(list);
1945 return NULL;
1946 }
Eric Snowc5e59602015-05-30 11:28:56 -06001947 iter = _PyObject_GetBuiltin("iter");
1948 if (iter == NULL) {
1949 Py_DECREF(list);
1950 return NULL;
1951 }
1952 return Py_BuildValue("N(N)", iter, list);
Eric Snow96c6af92015-05-29 22:21:39 -06001953}
1954
1955static PyMethodDef odictiter_methods[] = {
1956 {"__reduce__", (PyCFunction)odictiter_reduce, METH_NOARGS, reduce_doc},
1957 {NULL, NULL} /* sentinel */
1958};
1959
1960PyTypeObject PyODictIter_Type = {
1961 PyVarObject_HEAD_INIT(&PyType_Type, 0)
1962 "odict_iterator", /* tp_name */
1963 sizeof(odictiterobject), /* tp_basicsize */
1964 0, /* tp_itemsize */
1965 /* methods */
1966 (destructor)odictiter_dealloc, /* tp_dealloc */
1967 0, /* tp_print */
1968 0, /* tp_getattr */
1969 0, /* tp_setattr */
1970 0, /* tp_reserved */
1971 0, /* tp_repr */
1972 0, /* tp_as_number */
1973 0, /* tp_as_sequence */
1974 0, /* tp_as_mapping */
1975 0, /* tp_hash */
1976 0, /* tp_call */
1977 0, /* tp_str */
1978 PyObject_GenericGetAttr, /* tp_getattro */
1979 0, /* tp_setattro */
1980 0, /* tp_as_buffer */
1981 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC, /* tp_flags */
1982 0, /* tp_doc */
1983 (traverseproc)odictiter_traverse, /* tp_traverse */
1984 0, /* tp_clear */
1985 0, /* tp_richcompare */
1986 0, /* tp_weaklistoffset */
1987 PyObject_SelfIter, /* tp_iter */
1988 (iternextfunc)odictiter_iternext, /* tp_iternext */
1989 odictiter_methods, /* tp_methods */
1990 0,
1991};
1992
1993static PyObject *
1994odictiter_new(PyODictObject *od, int kind)
1995{
1996 odictiterobject *di;
1997 _ODictNode *node;
1998 int reversed = kind & _odict_ITER_REVERSED;
1999
2000 di = PyObject_GC_New(odictiterobject, &PyODictIter_Type);
2001 if (di == NULL)
2002 return NULL;
2003
2004 if (kind & (_odict_ITER_KEYS | _odict_ITER_VALUES)){
2005 di->di_result = PyTuple_Pack(2, Py_None, Py_None);
2006 if (di->di_result == NULL) {
2007 Py_DECREF(di);
2008 return NULL;
2009 }
2010 }
2011 else
2012 di->di_result = NULL;
2013
2014 di->kind = kind;
2015 node = reversed ? _odict_LAST(od) : _odict_FIRST(od);
2016 di->di_current = node ? _odictnode_KEY(node) : NULL;
2017 Py_XINCREF(di->di_current);
Eric Snowb952ab42015-06-01 23:35:13 -06002018 di->di_size = PyODict_SIZE(od);
Eric Snow4fabf022015-06-04 00:09:56 -06002019 di->di_state = od->od_state;
Eric Snow96c6af92015-05-29 22:21:39 -06002020 di->di_odict = od;
2021 Py_INCREF(od);
2022
2023 _PyObject_GC_TRACK(di);
2024 return (PyObject *)di;
2025}
2026
2027/* keys() */
2028
2029static PyObject *
2030odictkeys_iter(_PyDictViewObject *dv)
2031{
2032 if (dv->dv_dict == NULL) {
2033 Py_RETURN_NONE;
2034 }
2035 return odictiter_new((PyODictObject *)dv->dv_dict,
2036 _odict_ITER_KEYS);
2037}
2038
2039static PyObject *
2040odictkeys_reversed(_PyDictViewObject *dv)
2041{
2042 if (dv->dv_dict == NULL) {
2043 Py_RETURN_NONE;
2044 }
2045 return odictiter_new((PyODictObject *)dv->dv_dict,
2046 _odict_ITER_KEYS|_odict_ITER_REVERSED);
2047}
2048
2049static PyMethodDef odictkeys_methods[] = {
2050 {"__reversed__", (PyCFunction)odictkeys_reversed, METH_NOARGS, NULL},
2051 {NULL, NULL} /* sentinel */
2052};
2053
2054PyTypeObject PyODictKeys_Type = {
2055 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2056 "odict_keys", /* tp_name */
2057 0, /* tp_basicsize */
2058 0, /* tp_itemsize */
2059 0, /* tp_dealloc */
2060 0, /* tp_print */
2061 0, /* tp_getattr */
2062 0, /* tp_setattr */
2063 0, /* tp_reserved */
2064 0, /* tp_repr */
2065 0, /* tp_as_number */
2066 0, /* tp_as_sequence */
2067 0, /* tp_as_mapping */
2068 0, /* tp_hash */
2069 0, /* tp_call */
2070 0, /* tp_str */
2071 0, /* tp_getattro */
2072 0, /* tp_setattro */
2073 0, /* tp_as_buffer */
2074 0, /* tp_flags */
2075 0, /* tp_doc */
2076 0, /* tp_traverse */
2077 0, /* tp_clear */
2078 0, /* tp_richcompare */
2079 0, /* tp_weaklistoffset */
2080 (getiterfunc)odictkeys_iter, /* tp_iter */
2081 0, /* tp_iternext */
2082 odictkeys_methods, /* tp_methods */
2083 0, /* tp_members */
2084 0, /* tp_getset */
2085 &PyDictKeys_Type, /* tp_base */
2086};
2087
2088static PyObject *
2089odictkeys_new(PyObject *od)
2090{
2091 return _PyDictView_New(od, &PyODictKeys_Type);
2092}
2093
2094/* items() */
2095
2096static PyObject *
2097odictitems_iter(_PyDictViewObject *dv)
2098{
2099 if (dv->dv_dict == NULL) {
2100 Py_RETURN_NONE;
2101 }
2102 return odictiter_new((PyODictObject *)dv->dv_dict,
2103 _odict_ITER_KEYS|_odict_ITER_VALUES);
2104}
2105
2106static PyObject *
2107odictitems_reversed(_PyDictViewObject *dv)
2108{
2109 if (dv->dv_dict == NULL) {
2110 Py_RETURN_NONE;
2111 }
2112 return odictiter_new((PyODictObject *)dv->dv_dict,
2113 _odict_ITER_KEYS|_odict_ITER_VALUES|_odict_ITER_REVERSED);
2114}
2115
2116static PyMethodDef odictitems_methods[] = {
2117 {"__reversed__", (PyCFunction)odictitems_reversed, METH_NOARGS, NULL},
2118 {NULL, NULL} /* sentinel */
2119};
2120
2121PyTypeObject PyODictItems_Type = {
2122 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2123 "odict_items", /* tp_name */
2124 0, /* tp_basicsize */
2125 0, /* tp_itemsize */
2126 0, /* tp_dealloc */
2127 0, /* tp_print */
2128 0, /* tp_getattr */
2129 0, /* tp_setattr */
2130 0, /* tp_reserved */
2131 0, /* tp_repr */
2132 0, /* tp_as_number */
2133 0, /* tp_as_sequence */
2134 0, /* tp_as_mapping */
2135 0, /* tp_hash */
2136 0, /* tp_call */
2137 0, /* tp_str */
2138 0, /* tp_getattro */
2139 0, /* tp_setattro */
2140 0, /* tp_as_buffer */
2141 0, /* tp_flags */
2142 0, /* tp_doc */
2143 0, /* tp_traverse */
2144 0, /* tp_clear */
2145 0, /* tp_richcompare */
2146 0, /* tp_weaklistoffset */
2147 (getiterfunc)odictitems_iter, /* tp_iter */
2148 0, /* tp_iternext */
2149 odictitems_methods, /* tp_methods */
2150 0, /* tp_members */
2151 0, /* tp_getset */
2152 &PyDictItems_Type, /* tp_base */
2153};
2154
2155static PyObject *
2156odictitems_new(PyObject *od)
2157{
2158 return _PyDictView_New(od, &PyODictItems_Type);
2159}
2160
2161/* values() */
2162
2163static PyObject *
2164odictvalues_iter(_PyDictViewObject *dv)
2165{
2166 if (dv->dv_dict == NULL) {
2167 Py_RETURN_NONE;
2168 }
2169 return odictiter_new((PyODictObject *)dv->dv_dict,
2170 _odict_ITER_VALUES);
2171}
2172
2173static PyObject *
2174odictvalues_reversed(_PyDictViewObject *dv)
2175{
2176 if (dv->dv_dict == NULL) {
2177 Py_RETURN_NONE;
2178 }
2179 return odictiter_new((PyODictObject *)dv->dv_dict,
2180 _odict_ITER_VALUES|_odict_ITER_REVERSED);
2181}
2182
2183static PyMethodDef odictvalues_methods[] = {
2184 {"__reversed__", (PyCFunction)odictvalues_reversed, METH_NOARGS, NULL},
2185 {NULL, NULL} /* sentinel */
2186};
2187
2188PyTypeObject PyODictValues_Type = {
2189 PyVarObject_HEAD_INIT(&PyType_Type, 0)
2190 "odict_values", /* tp_name */
2191 0, /* tp_basicsize */
2192 0, /* tp_itemsize */
2193 0, /* tp_dealloc */
2194 0, /* tp_print */
2195 0, /* tp_getattr */
2196 0, /* tp_setattr */
2197 0, /* tp_reserved */
2198 0, /* tp_repr */
2199 0, /* tp_as_number */
2200 0, /* tp_as_sequence */
2201 0, /* tp_as_mapping */
2202 0, /* tp_hash */
2203 0, /* tp_call */
2204 0, /* tp_str */
2205 0, /* tp_getattro */
2206 0, /* tp_setattro */
2207 0, /* tp_as_buffer */
2208 0, /* tp_flags */
2209 0, /* tp_doc */
2210 0, /* tp_traverse */
2211 0, /* tp_clear */
2212 0, /* tp_richcompare */
2213 0, /* tp_weaklistoffset */
2214 (getiterfunc)odictvalues_iter, /* tp_iter */
2215 0, /* tp_iternext */
2216 odictvalues_methods, /* tp_methods */
2217 0, /* tp_members */
2218 0, /* tp_getset */
2219 &PyDictValues_Type, /* tp_base */
2220};
2221
2222static PyObject *
2223odictvalues_new(PyObject *od)
2224{
2225 return _PyDictView_New(od, &PyODictValues_Type);
2226}
2227
2228
2229/* ----------------------------------------------
Martin Panter536d70e2017-01-14 08:23:08 +00002230 MutableMapping implementations
Eric Snow96c6af92015-05-29 22:21:39 -06002231
2232Mapping:
2233
2234============ ===========
2235method uses
2236============ ===========
2237__contains__ __getitem__
2238__eq__ items
2239__getitem__ +
2240__iter__ +
2241__len__ +
2242__ne__ __eq__
2243get __getitem__
2244items ItemsView
2245keys KeysView
2246values ValuesView
2247============ ===========
2248
2249ItemsView uses __len__, __iter__, and __getitem__.
2250KeysView uses __len__, __iter__, and __contains__.
2251ValuesView uses __len__, __iter__, and __getitem__.
2252
2253MutableMapping:
2254
2255============ ===========
2256method uses
2257============ ===========
2258__delitem__ +
2259__setitem__ +
2260clear popitem
2261pop __getitem__
2262 __delitem__
2263popitem __iter__
2264 _getitem__
2265 __delitem__
2266setdefault __getitem__
2267 __setitem__
2268update __setitem__
2269============ ===========
2270*/
2271
2272static int
2273mutablemapping_add_pairs(PyObject *self, PyObject *pairs)
2274{
2275 PyObject *pair, *iterator, *unexpected;
2276 int res = 0;
2277
2278 iterator = PyObject_GetIter(pairs);
2279 if (iterator == NULL)
2280 return -1;
2281 PyErr_Clear();
2282
2283 while ((pair = PyIter_Next(iterator)) != NULL) {
2284 /* could be more efficient (see UNPACK_SEQUENCE in ceval.c) */
Eric Snowc5e59602015-05-30 11:28:56 -06002285 PyObject *key = NULL, *value = NULL;
Eric Snow96c6af92015-05-29 22:21:39 -06002286 PyObject *pair_iterator = PyObject_GetIter(pair);
Eric Snowc5e59602015-05-30 11:28:56 -06002287 if (pair_iterator == NULL)
2288 goto Done;
Eric Snow96c6af92015-05-29 22:21:39 -06002289
2290 key = PyIter_Next(pair_iterator);
2291 if (key == NULL) {
2292 if (!PyErr_Occurred())
2293 PyErr_SetString(PyExc_ValueError,
2294 "need more than 0 values to unpack");
2295 goto Done;
2296 }
2297
2298 value = PyIter_Next(pair_iterator);
2299 if (value == NULL) {
2300 if (!PyErr_Occurred())
2301 PyErr_SetString(PyExc_ValueError,
2302 "need more than 1 value to unpack");
2303 goto Done;
2304 }
2305
2306 unexpected = PyIter_Next(pair_iterator);
2307 if (unexpected != NULL) {
2308 Py_DECREF(unexpected);
2309 PyErr_SetString(PyExc_ValueError,
2310 "too many values to unpack (expected 2)");
2311 goto Done;
2312 }
2313 else if (PyErr_Occurred())
2314 goto Done;
2315
2316 res = PyObject_SetItem(self, key, value);
2317
2318Done:
2319 Py_DECREF(pair);
Eric Snowc5e59602015-05-30 11:28:56 -06002320 Py_XDECREF(pair_iterator);
Eric Snow96c6af92015-05-29 22:21:39 -06002321 Py_XDECREF(key);
2322 Py_XDECREF(value);
2323 if (PyErr_Occurred())
2324 break;
2325 }
2326 Py_DECREF(iterator);
2327
2328 if (res < 0 || PyErr_Occurred() != NULL)
2329 return -1;
2330 else
2331 return 0;
2332}
2333
2334static PyObject *
2335mutablemapping_update(PyObject *self, PyObject *args, PyObject *kwargs)
2336{
2337 int res = 0;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002338 Py_ssize_t len;
2339 _Py_IDENTIFIER(items);
2340 _Py_IDENTIFIER(keys);
Eric Snow96c6af92015-05-29 22:21:39 -06002341
2342 /* first handle args, if any */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002343 assert(args == NULL || PyTuple_Check(args));
2344 len = (args != NULL) ? PyTuple_GET_SIZE(args) : 0;
Benjamin Peterson99e96f22015-06-06 23:20:32 -05002345 if (len > 1) {
Eric Snow96c6af92015-05-29 22:21:39 -06002346 char *msg = "update() takes at most 1 positional argument (%d given)";
2347 PyErr_Format(PyExc_TypeError, msg, len);
2348 return NULL;
2349 }
Eric Snow96c6af92015-05-29 22:21:39 -06002350
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002351 if (len) {
Benjamin Peterson0718de92015-06-07 00:00:42 -05002352 PyObject *other = PyTuple_GET_ITEM(args, 0); /* borrowed reference */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002353 assert(other != NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002354 Py_INCREF(other);
Eric Snow06aed902016-09-09 11:59:08 -07002355 if PyDict_CheckExact(other) {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002356 PyObject *items;
2357 if (PyDict_CheckExact(other))
2358 items = PyDict_Items(other);
2359 else
2360 items = _PyObject_CallMethodId(other, &PyId_items, NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002361 Py_DECREF(other);
2362 if (items == NULL)
2363 return NULL;
2364 res = mutablemapping_add_pairs(self, items);
2365 Py_DECREF(items);
2366 if (res == -1)
2367 return NULL;
2368 }
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002369 else if (_PyObject_HasAttrId(other, &PyId_keys)) { /* never fails */
Benjamin Peterson0718de92015-06-07 00:00:42 -05002370 PyObject *keys, *iterator, *key;
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002371 keys = _PyObject_CallMethodIdObjArgs(other, &PyId_keys, NULL);
Benjamin Peterson0718de92015-06-07 00:00:42 -05002372 if (keys == NULL) {
2373 Py_DECREF(other);
2374 return NULL;
2375 }
2376 iterator = PyObject_GetIter(keys);
2377 Py_DECREF(keys);
2378 if (iterator == NULL) {
2379 Py_DECREF(other);
2380 return NULL;
2381 }
2382 while (res == 0 && (key = PyIter_Next(iterator))) {
2383 PyObject *value = PyObject_GetItem(other, key);
2384 if (value != NULL) {
2385 res = PyObject_SetItem(self, key, value);
2386 Py_DECREF(value);
2387 }
2388 else {
2389 res = -1;
2390 }
2391 Py_DECREF(key);
2392 }
2393 Py_DECREF(other);
2394 Py_DECREF(iterator);
2395 if (res != 0 || PyErr_Occurred())
2396 return NULL;
2397 }
Eric Snow06aed902016-09-09 11:59:08 -07002398 else if (_PyObject_HasAttrId(other, &PyId_items)) { /* never fails */
2399 PyObject *items;
2400 if (PyDict_CheckExact(other))
2401 items = PyDict_Items(other);
2402 else
2403 items = _PyObject_CallMethodId(other, &PyId_items, NULL);
2404 Py_DECREF(other);
2405 if (items == NULL)
2406 return NULL;
2407 res = mutablemapping_add_pairs(self, items);
2408 Py_DECREF(items);
2409 if (res == -1)
2410 return NULL;
2411 }
Benjamin Peterson0718de92015-06-07 00:00:42 -05002412 else {
2413 res = mutablemapping_add_pairs(self, other);
2414 Py_DECREF(other);
2415 if (res != 0)
2416 return NULL;
2417 }
2418 }
2419
2420 /* now handle kwargs */
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002421 assert(kwargs == NULL || PyDict_Check(kwargs));
Serhiy Storchaka5ab81d72016-12-16 16:18:57 +02002422 if (kwargs != NULL && PyDict_GET_SIZE(kwargs)) {
Serhiy Storchaka8003baf2015-10-18 09:53:17 +03002423 PyObject *items = PyDict_Items(kwargs);
Eric Snow96c6af92015-05-29 22:21:39 -06002424 if (items == NULL)
2425 return NULL;
2426 res = mutablemapping_add_pairs(self, items);
2427 Py_DECREF(items);
2428 if (res == -1)
2429 return NULL;
2430 }
Eric Snow96c6af92015-05-29 22:21:39 -06002431
2432 Py_RETURN_NONE;
2433}