Thomas Wouters | 49fd7fa | 2006-04-21 10:40:58 +0000 | [diff] [blame] | 1 | """Thread-local objects. |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 2 | |
Thomas Wouters | 49fd7fa | 2006-04-21 10:40:58 +0000 | [diff] [blame] | 3 | (Note that this module provides a Python version of the threading.local |
| 4 | class. Depending on the version of Python you're using, there may be a |
| 5 | faster one available. You should always import the `local` class from |
| 6 | `threading`.) |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 7 | |
| 8 | Thread-local objects support the management of thread-local data. |
| 9 | If you have data that you want to be local to a thread, simply create |
Andrew M. Kuchling | 3fc2fde | 2004-07-15 12:17:26 +0000 | [diff] [blame] | 10 | a thread-local object and use its attributes: |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 11 | |
| 12 | >>> mydata = local() |
| 13 | >>> mydata.number = 42 |
| 14 | >>> mydata.number |
| 15 | 42 |
| 16 | |
| 17 | You can also access the local-object's dictionary: |
| 18 | |
| 19 | >>> mydata.__dict__ |
| 20 | {'number': 42} |
| 21 | >>> mydata.__dict__.setdefault('widgets', []) |
| 22 | [] |
| 23 | >>> mydata.widgets |
| 24 | [] |
| 25 | |
| 26 | What's important about thread-local objects is that their data are |
| 27 | local to a thread. If we access the data in a different thread: |
| 28 | |
| 29 | >>> log = [] |
| 30 | >>> def f(): |
Guido van Rossum | cc2b016 | 2007-02-11 06:12:03 +0000 | [diff] [blame] | 31 | ... items = sorted(mydata.__dict__.items()) |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 32 | ... log.append(items) |
| 33 | ... mydata.number = 11 |
| 34 | ... log.append(mydata.number) |
| 35 | |
| 36 | >>> import threading |
| 37 | >>> thread = threading.Thread(target=f) |
| 38 | >>> thread.start() |
| 39 | >>> thread.join() |
| 40 | >>> log |
| 41 | [[], 11] |
| 42 | |
| 43 | we get different data. Furthermore, changes made in the other thread |
| 44 | don't affect data seen in this thread: |
| 45 | |
| 46 | >>> mydata.number |
| 47 | 42 |
| 48 | |
| 49 | Of course, values you get from a local object, including a __dict__ |
| 50 | attribute, are for whatever thread was current at the time the |
| 51 | attribute was read. For that reason, you generally don't want to save |
| 52 | these values across threads, as they apply only to the thread they |
| 53 | came from. |
| 54 | |
| 55 | You can create custom local objects by subclassing the local class: |
| 56 | |
| 57 | >>> class MyLocal(local): |
| 58 | ... number = 2 |
Serhiy Storchaka | 2085bd0 | 2019-06-01 11:00:15 +0300 | [diff] [blame] | 59 | ... def __init__(self, /, **kw): |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 60 | ... self.__dict__.update(kw) |
| 61 | ... def squared(self): |
| 62 | ... return self.number ** 2 |
| 63 | |
| 64 | This can be useful to support default values, methods and |
| 65 | initialization. Note that if you define an __init__ method, it will be |
| 66 | called each time the local object is used in a separate thread. This |
| 67 | is necessary to initialize each thread's dictionary. |
| 68 | |
| 69 | Now if we create a local object: |
| 70 | |
| 71 | >>> mydata = MyLocal(color='red') |
| 72 | |
| 73 | Now we have a default number: |
| 74 | |
| 75 | >>> mydata.number |
| 76 | 2 |
| 77 | |
| 78 | an initial color: |
Tim Peters | 182b5ac | 2004-07-18 06:16:08 +0000 | [diff] [blame] | 79 | |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 80 | >>> mydata.color |
| 81 | 'red' |
| 82 | >>> del mydata.color |
| 83 | |
| 84 | And a method that operates on the data: |
| 85 | |
| 86 | >>> mydata.squared() |
| 87 | 4 |
| 88 | |
| 89 | As before, we can access the data in a separate thread: |
| 90 | |
| 91 | >>> log = [] |
| 92 | >>> thread = threading.Thread(target=f) |
| 93 | >>> thread.start() |
| 94 | >>> thread.join() |
| 95 | >>> log |
Aaron Gallagher | 5fb632e | 2018-02-25 07:03:40 -0800 | [diff] [blame] | 96 | [[('color', 'red')], 11] |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 97 | |
Andrew M. Kuchling | 3fc2fde | 2004-07-15 12:17:26 +0000 | [diff] [blame] | 98 | without affecting this thread's data: |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 99 | |
| 100 | >>> mydata.number |
| 101 | 2 |
| 102 | >>> mydata.color |
| 103 | Traceback (most recent call last): |
| 104 | ... |
| 105 | AttributeError: 'MyLocal' object has no attribute 'color' |
| 106 | |
| 107 | Note that subclasses can define slots, but they are not thread |
| 108 | local. They are shared across threads: |
| 109 | |
| 110 | >>> class MyLocal(local): |
| 111 | ... __slots__ = 'number' |
| 112 | |
| 113 | >>> mydata = MyLocal() |
| 114 | >>> mydata.number = 42 |
| 115 | >>> mydata.color = 'red' |
| 116 | |
| 117 | So, the separate thread: |
| 118 | |
| 119 | >>> thread = threading.Thread(target=f) |
| 120 | >>> thread.start() |
| 121 | >>> thread.join() |
| 122 | |
| 123 | affects what we see: |
| 124 | |
| 125 | >>> mydata.number |
| 126 | 11 |
| 127 | |
| 128 | >>> del mydata |
| 129 | """ |
| 130 | |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 131 | from weakref import ref |
| 132 | from contextlib import contextmanager |
| 133 | |
Thomas Wouters | 49fd7fa | 2006-04-21 10:40:58 +0000 | [diff] [blame] | 134 | __all__ = ["local"] |
| 135 | |
| 136 | # We need to use objects from the threading module, but the threading |
| 137 | # module may also want to use our `local` class, if support for locals |
| 138 | # isn't compiled in to the `thread` module. This creates potential problems |
| 139 | # with circular imports. For that reason, we don't import `threading` |
| 140 | # until the bottom of this file (a hack sufficient to worm around the |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 141 | # potential problems). Note that all platforms on CPython do have support |
| 142 | # for locals in the `thread` module, and there is no circular import problem |
Thomas Wouters | 49fd7fa | 2006-04-21 10:40:58 +0000 | [diff] [blame] | 143 | # then, so problems introduced by fiddling the order of imports here won't |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 144 | # manifest. |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 145 | |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 146 | class _localimpl: |
| 147 | """A class managing thread-local dicts""" |
| 148 | __slots__ = 'key', 'dicts', 'localargs', 'locallock', '__weakref__' |
| 149 | |
| 150 | def __init__(self): |
| 151 | # The key used in the Thread objects' attribute dicts. |
| 152 | # We keep it a string for speed but make it unlikely to clash with |
| 153 | # a "real" attribute. |
| 154 | self.key = '_threading_local._localimpl.' + str(id(self)) |
| 155 | # { id(Thread) -> (ref(Thread), thread-local dict) } |
| 156 | self.dicts = {} |
| 157 | |
| 158 | def get_dict(self): |
| 159 | """Return the dict for the current thread. Raises KeyError if none |
| 160 | defined.""" |
| 161 | thread = current_thread() |
| 162 | return self.dicts[id(thread)][1] |
| 163 | |
| 164 | def create_dict(self): |
| 165 | """Create a new dict for the current thread, and return it.""" |
| 166 | localdict = {} |
| 167 | key = self.key |
| 168 | thread = current_thread() |
| 169 | idt = id(thread) |
| 170 | def local_deleted(_, key=key): |
| 171 | # When the localimpl is deleted, remove the thread attribute. |
| 172 | thread = wrthread() |
| 173 | if thread is not None: |
| 174 | del thread.__dict__[key] |
| 175 | def thread_deleted(_, idt=idt): |
| 176 | # When the thread is deleted, remove the local dict. |
| 177 | # Note that this is suboptimal if the thread object gets |
| 178 | # caught in a reference loop. We would like to be called |
| 179 | # as soon as the OS-level thread ends instead. |
| 180 | local = wrlocal() |
| 181 | if local is not None: |
| 182 | dct = local.dicts.pop(idt) |
| 183 | wrlocal = ref(self, local_deleted) |
| 184 | wrthread = ref(thread, thread_deleted) |
| 185 | thread.__dict__[key] = wrlocal |
| 186 | self.dicts[idt] = wrthread, localdict |
| 187 | return localdict |
| 188 | |
| 189 | |
| 190 | @contextmanager |
| 191 | def _patch(self): |
| 192 | impl = object.__getattribute__(self, '_local__impl') |
| 193 | try: |
| 194 | dct = impl.get_dict() |
| 195 | except KeyError: |
| 196 | dct = impl.create_dict() |
| 197 | args, kw = impl.localargs |
| 198 | self.__init__(*args, **kw) |
| 199 | with impl.locallock: |
| 200 | object.__setattr__(self, '__dict__', dct) |
| 201 | yield |
| 202 | |
| 203 | |
| 204 | class local: |
| 205 | __slots__ = '_local__impl', '__dict__' |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 206 | |
Serhiy Storchaka | 2085bd0 | 2019-06-01 11:00:15 +0300 | [diff] [blame] | 207 | def __new__(cls, /, *args, **kw): |
Jack Diederich | 561d5aa | 2010-02-22 19:55:46 +0000 | [diff] [blame] | 208 | if (args or kw) and (cls.__init__ is object.__init__): |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 209 | raise TypeError("Initialization arguments are not supported") |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 210 | self = object.__new__(cls) |
| 211 | impl = _localimpl() |
| 212 | impl.localargs = (args, kw) |
| 213 | impl.locallock = RLock() |
| 214 | object.__setattr__(self, '_local__impl', impl) |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 215 | # We need to create the thread dict in anticipation of |
Neal Norwitz | 7025ce6 | 2005-11-25 02:02:50 +0000 | [diff] [blame] | 216 | # __init__ being called, to make sure we don't call it |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 217 | # again ourselves. |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 218 | impl.create_dict() |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 219 | return self |
| 220 | |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 221 | def __getattribute__(self, name): |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 222 | with _patch(self): |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 223 | return object.__getattribute__(self, name) |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 224 | |
| 225 | def __setattr__(self, name, value): |
Antoine Pitrou | 1a9a9d5 | 2010-08-28 18:17:03 +0000 | [diff] [blame] | 226 | if name == '__dict__': |
| 227 | raise AttributeError( |
| 228 | "%r object attribute '__dict__' is read-only" |
| 229 | % self.__class__.__name__) |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 230 | with _patch(self): |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 231 | return object.__setattr__(self, name, value) |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 232 | |
| 233 | def __delattr__(self, name): |
Antoine Pitrou | 1a9a9d5 | 2010-08-28 18:17:03 +0000 | [diff] [blame] | 234 | if name == '__dict__': |
| 235 | raise AttributeError( |
| 236 | "%r object attribute '__dict__' is read-only" |
| 237 | % self.__class__.__name__) |
Antoine Pitrou | 783eea7 | 2010-09-07 22:06:17 +0000 | [diff] [blame] | 238 | with _patch(self): |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 239 | return object.__delattr__(self, name) |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 240 | |
Jim Fulton | d15dc06 | 2004-07-14 19:11:50 +0000 | [diff] [blame] | 241 | |
Benjamin Peterson | 672b803 | 2008-06-11 19:14:14 +0000 | [diff] [blame] | 242 | from threading import current_thread, RLock |