Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 1 | """\ |
| 2 | Pickling Algorithm |
| 3 | ------------------ |
| 4 | |
| 5 | This module implements a basic but powerful algorithm for "pickling" (a.k.a. |
| 6 | serializing, marshalling or flattening) nearly arbitrary Python objects. |
| 7 | This is a more primitive notion than persistency -- although pickle |
| 8 | reads and writes file objects, it does not handle the issue of naming |
| 9 | persistent objects, nor the (even more complicated) area of concurrent |
| 10 | access to persistent objects. The pickle module can transform a complex |
| 11 | object into a byte stream and it can transform the byte stream into |
| 12 | an object with the same internal structure. The most obvious thing to |
| 13 | do with these byte streams is to write them onto a file, but it is also |
| 14 | conceivable to send them across a network or store them in a database. |
| 15 | |
| 16 | Unlike the built-in marshal module, pickle handles the following correctly: |
| 17 | |
| 18 | - recursive objects |
| 19 | - pointer sharing |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 20 | - classes and class instances |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 21 | |
| 22 | Pickle is Python-specific. This has the advantage that there are no |
| 23 | restrictions imposed by external standards such as CORBA (which probably |
| 24 | can't represent pointer sharing or recursive objects); however it means |
| 25 | that non-Python programs may not be able to reconstruct pickled Python |
| 26 | objects. |
| 27 | |
| 28 | Pickle uses a printable ASCII representation. This is slightly more |
| 29 | voluminous than a binary representation. However, small integers actually |
| 30 | take *less* space when represented as minimal-size decimal strings than |
| 31 | when represented as 32-bit binary numbers, and strings are only much longer |
| 32 | if they contain control characters or 8-bit characters. The big advantage |
| 33 | of using printable ASCII (and of some other characteristics of pickle's |
| 34 | representation) is that for debugging or recovery purposes it is possible |
| 35 | for a human to read the pickled file with a standard text editor. (I could |
| 36 | have gone a step further and used a notation like S-expressions, but the |
| 37 | parser would have been considerably more complicated and slower, and the |
| 38 | files would probably have become much larger.) |
| 39 | |
| 40 | Pickle doesn't handle code objects, which marshal does. |
| 41 | I suppose pickle could, and maybe it should, but there's probably no |
| 42 | great need for it right now (as long as marshal continues to be used |
| 43 | for reading and writing code objects), and at least this avoids |
| 44 | the possibility of smuggling Trojan horses into a program. |
| 45 | |
| 46 | For the benefit of persistency modules written using pickle, it supports |
| 47 | the notion of a reference to an object outside the pickled data stream. |
| 48 | Such objects are referenced by a name, which is an arbitrary string of |
| 49 | printable ASCII characters. The resolution of such names is not defined |
| 50 | by the pickle module -- the persistent object module will have to implement |
| 51 | a method "persistent_load". To write references to persistent objects, |
| 52 | the persistent module must define a method "persistent_id" which returns |
| 53 | either None or the persistent ID of the object. |
| 54 | |
| 55 | There are some restrictions on the pickling of class instances. |
| 56 | |
| 57 | First of all, the class must be defined at the top level in a module. |
| 58 | |
Guido van Rossum | 37a6f16 | 1996-08-08 18:35:22 +0000 | [diff] [blame] | 59 | Next, it must normally be possible to create class instances by |
| 60 | calling the class without arguments. Usually, this is best |
| 61 | accomplished by providing default values for all arguments to its |
| 62 | __init__ method (if it has one). If this is undesirable, the |
| 63 | class can define a method __getinitargs__, which should return a |
| 64 | *tuple* containing the arguments to be passed to the class |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 65 | constructor. |
| 66 | |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 67 | Classes can influence how their instances are pickled -- if the class defines |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 68 | the method __getstate__, it is called and the return state is pickled |
| 69 | as the contents for the instance, and if the class defines the |
| 70 | method __setstate__, it is called with the unpickled state. (Note |
| 71 | that these methods can also be used to implement copying class instances.) |
| 72 | If there is no __getstate__ method, the instance's __dict__ |
| 73 | is pickled. If there is no __setstate__ method, the pickled object |
| 74 | must be a dictionary and its items are assigned to the new instance's |
| 75 | dictionary. (If a class defines both __getstate__ and __setstate__, |
| 76 | the state object needn't be a dictionary -- these methods can do what they |
| 77 | want.) |
| 78 | |
| 79 | Note that when class instances are pickled, their class's code and data |
| 80 | is not pickled along with them. Only the instance data is pickled. |
| 81 | This is done on purpose, so you can fix bugs in a class or add methods and |
| 82 | still load objects that were created with an earlier version of the |
| 83 | class. If you plan to have long-lived objects that will see many versions |
| 84 | of a class, it may be worth to put a version number in the objects so |
| 85 | that suitable conversions can be made by the class's __setstate__ method. |
| 86 | |
| 87 | The interface is as follows: |
| 88 | |
Guido van Rossum | 256cbd7 | 1995-02-16 16:30:50 +0000 | [diff] [blame] | 89 | To pickle an object x onto a file f, open for writing: |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 90 | |
| 91 | p = pickle.Pickler(f) |
| 92 | p.dump(x) |
| 93 | |
| 94 | To unpickle an object x from a file f, open for reading: |
| 95 | |
| 96 | u = pickle.Unpickler(f) |
Guido van Rossum | 48aa82e | 1995-04-10 11:34:46 +0000 | [diff] [blame] | 97 | x = u.load() |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 98 | |
| 99 | The Pickler class only calls the method f.write with a string argument |
| 100 | (XXX possibly the interface should pass f.write instead of f). |
| 101 | The Unpickler calls the methods f.read(with an integer argument) |
| 102 | and f.readline(without argument), both returning a string. |
| 103 | It is explicitly allowed to pass non-file objects here, as long as they |
| 104 | have the right methods. |
| 105 | |
| 106 | The following types can be pickled: |
| 107 | |
| 108 | - None |
| 109 | - integers, long integers, floating point numbers |
| 110 | - strings |
Guido van Rossum | 256cbd7 | 1995-02-16 16:30:50 +0000 | [diff] [blame] | 111 | - tuples, lists and dictionaries containing only picklable objects |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 112 | - class instances whose __dict__ or __setstate__() is picklable |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 113 | - classes |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 114 | |
| 115 | Attempts to pickle unpicklable objects will raise an exception |
| 116 | after having written an unspecified number of bytes to the file argument. |
| 117 | |
| 118 | It is possible to make multiple calls to Pickler.dump() or to |
| 119 | Unpickler.load(), as long as there is a one-to-one correspondence |
Guido van Rossum | 256cbd7 | 1995-02-16 16:30:50 +0000 | [diff] [blame] | 120 | between pickler and Unpickler objects and between dump and load calls |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 121 | for any pair of corresponding Pickler and Unpicklers. WARNING: this |
| 122 | is intended for pickleing multiple objects without intervening modifications |
| 123 | to the objects or their parts. If you modify an object and then pickle |
| 124 | it again using the same Pickler instance, the object is not pickled |
| 125 | again -- a reference to it is pickled and the Unpickler will return |
| 126 | the old value, not the modified one. (XXX There are two problems here: |
| 127 | (a) detecting changes, and (b) marshalling a minimal set of changes. |
| 128 | I have no answers. Garbage Collection may also become a problem here.) |
| 129 | """ |
| 130 | |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 131 | __version__ = "1.6" # Code version |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 132 | |
| 133 | from types import * |
| 134 | import string |
| 135 | |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 136 | format_version = "1.1" # File format version we write |
| 137 | compatible_formats = ["1.0"] # Old format versions we can read |
| 138 | |
Guido van Rossum | 7849da8 | 1995-03-09 14:08:35 +0000 | [diff] [blame] | 139 | PicklingError = "pickle.PicklingError" |
| 140 | |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 141 | AtomicTypes = [NoneType, IntType, FloatType, StringType] |
| 142 | |
| 143 | def safe(object): |
| 144 | t = type(object) |
| 145 | if t in AtomicTypes: |
| 146 | return 1 |
| 147 | if t is TupleType: |
| 148 | for item in object: |
| 149 | if not safe(item): return 0 |
| 150 | return 1 |
| 151 | return 0 |
| 152 | |
| 153 | MARK = '(' |
| 154 | POP = '0' |
| 155 | DUP = '2' |
| 156 | STOP = '.' |
| 157 | TUPLE = 't' |
| 158 | LIST = 'l' |
| 159 | DICT = 'd' |
| 160 | INST = 'i' |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 161 | CLASS = 'c' |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 162 | GET = 'g' |
| 163 | PUT = 'p' |
| 164 | APPEND = 'a' |
| 165 | SETITEM = 's' |
| 166 | BUILD = 'b' |
| 167 | NONE = 'N' |
| 168 | INT = 'I' |
| 169 | LONG = 'L' |
| 170 | FLOAT = 'F' |
| 171 | STRING = 'S' |
| 172 | PERSID = 'P' |
| 173 | AtomicKeys = [NONE, INT, LONG, FLOAT, STRING] |
| 174 | AtomicMap = { |
| 175 | NoneType: NONE, |
| 176 | IntType: INT, |
| 177 | LongType: LONG, |
| 178 | FloatType: FLOAT, |
| 179 | StringType: STRING, |
| 180 | } |
| 181 | |
| 182 | class Pickler: |
| 183 | |
| 184 | def __init__(self, file): |
| 185 | self.write = file.write |
| 186 | self.memo = {} |
| 187 | |
| 188 | def dump(self, object): |
| 189 | self.save(object) |
| 190 | self.write(STOP) |
| 191 | |
| 192 | def save(self, object): |
| 193 | pid = self.persistent_id(object) |
| 194 | if pid: |
| 195 | self.write(PERSID + str(pid) + '\n') |
| 196 | return |
| 197 | d = id(object) |
| 198 | if self.memo.has_key(d): |
| 199 | self.write(GET + `d` + '\n') |
| 200 | return |
| 201 | t = type(object) |
Guido van Rossum | 7849da8 | 1995-03-09 14:08:35 +0000 | [diff] [blame] | 202 | try: |
| 203 | f = self.dispatch[t] |
| 204 | except KeyError: |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 205 | if hasattr(object, '__class__'): |
| 206 | f = self.dispatch[InstanceType] |
| 207 | else: |
| 208 | raise PicklingError, \ |
| 209 | "can't pickle %s objects" % `t.__name__` |
Guido van Rossum | 7849da8 | 1995-03-09 14:08:35 +0000 | [diff] [blame] | 210 | f(self, object) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 211 | |
| 212 | def persistent_id(self, object): |
| 213 | return None |
| 214 | |
| 215 | dispatch = {} |
| 216 | |
| 217 | def save_none(self, object): |
| 218 | self.write(NONE) |
| 219 | dispatch[NoneType] = save_none |
| 220 | |
| 221 | def save_int(self, object): |
| 222 | self.write(INT + `object` + '\n') |
| 223 | dispatch[IntType] = save_int |
| 224 | |
| 225 | def save_long(self, object): |
| 226 | self.write(LONG + `object` + '\n') |
| 227 | dispatch[LongType] = save_long |
| 228 | |
| 229 | def save_float(self, object): |
| 230 | self.write(FLOAT + `object` + '\n') |
| 231 | dispatch[FloatType] = save_float |
| 232 | |
| 233 | def save_string(self, object): |
| 234 | d = id(object) |
| 235 | self.write(STRING + `object` + '\n') |
| 236 | self.write(PUT + `d` + '\n') |
| 237 | self.memo[d] = object |
| 238 | dispatch[StringType] = save_string |
| 239 | |
| 240 | def save_tuple(self, object): |
| 241 | d = id(object) |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 242 | write = self.write |
| 243 | save = self.save |
| 244 | has_key = self.memo.has_key |
| 245 | write(MARK) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 246 | n = len(object) |
| 247 | for k in range(n): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 248 | save(object[k]) |
| 249 | if has_key(d): |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 250 | # Saving object[k] has saved us! |
| 251 | while k >= 0: |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 252 | write(POP) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 253 | k = k-1 |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 254 | write(GET + `d` + '\n') |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 255 | break |
| 256 | else: |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 257 | write(TUPLE + PUT + `d` + '\n') |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 258 | self.memo[d] = object |
| 259 | dispatch[TupleType] = save_tuple |
| 260 | |
| 261 | def save_list(self, object): |
| 262 | d = id(object) |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 263 | write = self.write |
| 264 | save = self.save |
| 265 | write(MARK) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 266 | n = len(object) |
| 267 | for k in range(n): |
| 268 | item = object[k] |
| 269 | if not safe(item): |
| 270 | break |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 271 | save(item) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 272 | else: |
| 273 | k = n |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 274 | write(LIST + PUT + `d` + '\n') |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 275 | self.memo[d] = object |
| 276 | for k in range(k, n): |
| 277 | item = object[k] |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 278 | save(item) |
| 279 | write(APPEND) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 280 | dispatch[ListType] = save_list |
| 281 | |
| 282 | def save_dict(self, object): |
| 283 | d = id(object) |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 284 | write = self.write |
| 285 | save = self.save |
| 286 | write(MARK) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 287 | items = object.items() |
| 288 | n = len(items) |
| 289 | for k in range(n): |
| 290 | key, value = items[k] |
| 291 | if not safe(key) or not safe(value): |
| 292 | break |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 293 | save(key) |
| 294 | save(value) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 295 | else: |
| 296 | k = n |
| 297 | self.write(DICT + PUT + `d` + '\n') |
| 298 | self.memo[d] = object |
| 299 | for k in range(k, n): |
| 300 | key, value = items[k] |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 301 | save(key) |
| 302 | save(value) |
| 303 | write(SETITEM) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 304 | dispatch[DictionaryType] = save_dict |
| 305 | |
| 306 | def save_inst(self, object): |
| 307 | d = id(object) |
| 308 | cls = object.__class__ |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 309 | write = self.write |
| 310 | save = self.save |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 311 | module = whichmodule(cls) |
| 312 | name = cls.__name__ |
| 313 | if hasattr(object, '__getinitargs__'): |
| 314 | args = object.__getinitargs__() |
| 315 | len(args) # XXX Assert it's a sequence |
| 316 | else: |
| 317 | args = () |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 318 | write(MARK) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 319 | for arg in args: |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 320 | save(arg) |
| 321 | write(INST + module + '\n' + name + '\n' + |
| 322 | PUT + `d` + '\n') |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 323 | self.memo[d] = object |
| 324 | try: |
| 325 | getstate = object.__getstate__ |
| 326 | except AttributeError: |
| 327 | stuff = object.__dict__ |
| 328 | else: |
| 329 | stuff = getstate() |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 330 | save(stuff) |
| 331 | write(BUILD) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 332 | dispatch[InstanceType] = save_inst |
| 333 | |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 334 | def save_class(self, object): |
| 335 | d = id(object) |
| 336 | module = whichmodule(object) |
| 337 | name = object.__name__ |
| 338 | self.write(CLASS + module + '\n' + name + '\n' + |
| 339 | PUT + `d` + '\n') |
| 340 | dispatch[ClassType] = save_class |
| 341 | |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 342 | |
| 343 | classmap = {} |
| 344 | |
| 345 | def whichmodule(cls): |
| 346 | """Figure out the module in which a class occurs. |
| 347 | |
| 348 | Search sys.modules for the module. |
| 349 | Cache in classmap. |
| 350 | Return a module name. |
| 351 | If the class cannot be found, return __main__. |
| 352 | """ |
| 353 | if classmap.has_key(cls): |
| 354 | return classmap[cls] |
| 355 | import sys |
| 356 | clsname = cls.__name__ |
| 357 | for name, module in sys.modules.items(): |
Guido van Rossum | f71c79b | 1995-06-22 18:51:23 +0000 | [diff] [blame] | 358 | if name != '__main__' and \ |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 359 | hasattr(module, clsname) and \ |
| 360 | getattr(module, clsname) is cls: |
| 361 | break |
| 362 | else: |
| 363 | name = '__main__' |
| 364 | classmap[cls] = name |
| 365 | return name |
| 366 | |
| 367 | |
| 368 | class Unpickler: |
| 369 | |
| 370 | def __init__(self, file): |
| 371 | self.readline = file.readline |
| 372 | self.read = file.read |
| 373 | self.memo = {} |
| 374 | |
| 375 | def load(self): |
| 376 | self.mark = ['spam'] # Any new unique object |
| 377 | self.stack = [] |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 378 | self.append = self.stack.append |
| 379 | read = self.read |
| 380 | dispatch = self.dispatch |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 381 | try: |
| 382 | while 1: |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 383 | key = read(1) |
| 384 | dispatch[key](self) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 385 | except STOP, value: |
| 386 | return value |
| 387 | |
| 388 | def marker(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 389 | stack = self.stack |
| 390 | mark = self.mark |
| 391 | k = len(stack)-1 |
| 392 | while stack[k] is not mark: k = k-1 |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 393 | return k |
| 394 | |
| 395 | dispatch = {} |
| 396 | |
Guido van Rossum | 7b5430f | 1995-03-04 22:25:21 +0000 | [diff] [blame] | 397 | def load_eof(self): |
| 398 | raise EOFError |
| 399 | dispatch[''] = load_eof |
| 400 | |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 401 | def load_persid(self): |
| 402 | pid = self.readline()[:-1] |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 403 | self.append(self.persistent_load(pid)) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 404 | dispatch[PERSID] = load_persid |
| 405 | |
| 406 | def load_none(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 407 | self.append(None) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 408 | dispatch[NONE] = load_none |
| 409 | |
Guido van Rossum | 7853647 | 1996-04-12 13:36:27 +0000 | [diff] [blame] | 410 | def load_int(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 411 | self.append(string.atoi(self.readline()[:-1], 0)) |
Guido van Rossum | 7853647 | 1996-04-12 13:36:27 +0000 | [diff] [blame] | 412 | dispatch[INT] = load_int |
| 413 | |
| 414 | def load_long(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 415 | self.append(string.atol(self.readline()[:-1], 0)) |
Guido van Rossum | 7853647 | 1996-04-12 13:36:27 +0000 | [diff] [blame] | 416 | dispatch[LONG] = load_long |
| 417 | |
| 418 | def load_float(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 419 | self.append(string.atof(self.readline()[:-1])) |
Guido van Rossum | 7853647 | 1996-04-12 13:36:27 +0000 | [diff] [blame] | 420 | dispatch[FLOAT] = load_float |
| 421 | |
| 422 | def load_string(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 423 | self.append(eval(self.readline()[:-1], |
| 424 | {'__builtins__': {}})) # Let's be careful |
Guido van Rossum | 7853647 | 1996-04-12 13:36:27 +0000 | [diff] [blame] | 425 | dispatch[STRING] = load_string |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 426 | |
| 427 | def load_tuple(self): |
| 428 | k = self.marker() |
| 429 | self.stack[k:] = [tuple(self.stack[k+1:])] |
| 430 | dispatch[TUPLE] = load_tuple |
| 431 | |
| 432 | def load_list(self): |
| 433 | k = self.marker() |
| 434 | self.stack[k:] = [self.stack[k+1:]] |
| 435 | dispatch[LIST] = load_list |
| 436 | |
| 437 | def load_dict(self): |
| 438 | k = self.marker() |
| 439 | d = {} |
| 440 | items = self.stack[k+1:] |
| 441 | for i in range(0, len(items), 2): |
| 442 | key = items[i] |
| 443 | value = items[i+1] |
| 444 | d[key] = value |
| 445 | self.stack[k:] = [d] |
| 446 | dispatch[DICT] = load_dict |
| 447 | |
| 448 | def load_inst(self): |
| 449 | k = self.marker() |
| 450 | args = tuple(self.stack[k+1:]) |
| 451 | del self.stack[k:] |
| 452 | module = self.readline()[:-1] |
| 453 | name = self.readline()[:-1] |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 454 | klass = self.find_class(module, name) |
| 455 | value = apply(klass, args) |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 456 | self.append(value) |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 457 | dispatch[INST] = load_inst |
| 458 | |
| 459 | def load_class(self): |
| 460 | module = self.readline()[:-1] |
| 461 | name = self.readline()[:-1] |
| 462 | klass = self.find_class(module, name) |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 463 | self.append(klass) |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 464 | return klass |
| 465 | dispatch[CLASS] = load_class |
| 466 | |
| 467 | def find_class(self, module, name): |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 468 | env = {} |
| 469 | try: |
| 470 | exec 'from %s import %s' % (module, name) in env |
| 471 | except ImportError: |
| 472 | raise SystemError, \ |
| 473 | "Failed to import class %s from module %s" % \ |
| 474 | (name, module) |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 475 | klass = env[name] |
Guido van Rossum | 36fc11e | 1996-10-07 14:34:20 +0000 | [diff] [blame] | 476 | if type(klass) is BuiltinFunctionType: |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 477 | raise SystemError, \ |
| 478 | "Imported object %s from module %s is not a class" % \ |
| 479 | (name, module) |
| 480 | return klass |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 481 | |
| 482 | def load_pop(self): |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 483 | del self.stack[-1] |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 484 | dispatch[POP] = load_pop |
| 485 | |
| 486 | def load_dup(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 487 | self.append(stack[-1]) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 488 | dispatch[DUP] = load_dup |
| 489 | |
| 490 | def load_get(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 491 | self.append(self.memo[self.readline()[:-1]]) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 492 | dispatch[GET] = load_get |
| 493 | |
| 494 | def load_put(self): |
Guido van Rossum | 7853647 | 1996-04-12 13:36:27 +0000 | [diff] [blame] | 495 | self.memo[self.readline()[:-1]] = self.stack[-1] |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 496 | dispatch[PUT] = load_put |
| 497 | |
| 498 | def load_append(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 499 | stack = self.stack |
| 500 | value = stack[-1] |
| 501 | del stack[-1] |
| 502 | list = stack[-1] |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 503 | list.append(value) |
| 504 | dispatch[APPEND] = load_append |
| 505 | |
| 506 | def load_setitem(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 507 | stack = self.stack |
| 508 | value = stack[-1] |
| 509 | key = stack[-2] |
| 510 | del stack[-2:] |
| 511 | dict = stack[-1] |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 512 | dict[key] = value |
| 513 | dispatch[SETITEM] = load_setitem |
| 514 | |
| 515 | def load_build(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 516 | stack = self.stack |
| 517 | value = stack[-1] |
| 518 | del stack[-1] |
| 519 | inst = stack[-1] |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 520 | try: |
| 521 | setstate = inst.__setstate__ |
| 522 | except AttributeError: |
| 523 | for key in value.keys(): |
Guido van Rossum | abfd806 | 1996-08-08 20:26:45 +0000 | [diff] [blame] | 524 | setattr(inst, key, value[key]) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 525 | else: |
| 526 | setstate(value) |
| 527 | dispatch[BUILD] = load_build |
| 528 | |
| 529 | def load_mark(self): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 530 | self.append(self.mark) |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 531 | dispatch[MARK] = load_mark |
| 532 | |
| 533 | def load_stop(self): |
| 534 | value = self.stack[-1] |
| 535 | del self.stack[-1] |
| 536 | raise STOP, value |
| 537 | dispatch[STOP] = load_stop |
| 538 | |
| 539 | |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 540 | # Shorthands |
| 541 | |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 542 | from StringIO import StringIO |
| 543 | |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 544 | def dump(object, file): |
| 545 | Pickler(file).dump(object) |
| 546 | |
| 547 | def dumps(object): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 548 | file = StringIO() |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 549 | Pickler(file).dump(object) |
| 550 | return file.getvalue() |
| 551 | |
| 552 | def load(file): |
| 553 | return Unpickler(file).load() |
| 554 | |
| 555 | def loads(str): |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 556 | file = StringIO(str) |
Guido van Rossum | 0c891ce | 1995-03-14 15:09:05 +0000 | [diff] [blame] | 557 | return Unpickler(file).load() |
| 558 | |
| 559 | |
| 560 | # The rest is used for testing only |
| 561 | |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 562 | class C: |
| 563 | def __cmp__(self, other): |
| 564 | return cmp(self.__dict__, other.__dict__) |
| 565 | |
| 566 | def test(): |
| 567 | fn = 'pickle_tmp' |
| 568 | c = C() |
| 569 | c.foo = 1 |
Guido van Rossum | 955c5d1 | 1996-05-15 22:49:57 +0000 | [diff] [blame] | 570 | c.bar = 2L |
Guido van Rossum | c7c5e69 | 1996-07-22 22:26:07 +0000 | [diff] [blame] | 571 | x = [0, 1, 2, 3] |
Guido van Rossum | a48061a | 1995-01-10 00:31:14 +0000 | [diff] [blame] | 572 | y = ('abc', 'abc', c, c) |
| 573 | x.append(y) |
| 574 | x.append(y) |
| 575 | x.append(5) |
| 576 | f = open(fn, 'w') |
| 577 | F = Pickler(f) |
| 578 | F.dump(x) |
| 579 | f.close() |
| 580 | f = open(fn, 'r') |
| 581 | U = Unpickler(f) |
| 582 | x2 = U.load() |
| 583 | print x |
| 584 | print x2 |
| 585 | print x == x2 |
| 586 | print map(id, x) |
| 587 | print map(id, x2) |
| 588 | print F.memo |
| 589 | print U.memo |
| 590 | |
| 591 | if __name__ == '__main__': |
| 592 | test() |