Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 1 | Project: core implementation |
| 2 | **************************** |
| 3 | |
| 4 | Tasks: |
| 5 | |
| 6 | Do binary operators properly. nb_add should try to call self.__add__ |
| 7 | and other.__radd__. I think I'll exclude base types that define any |
| 8 | binary operator without setting the CHECKTYPES flag. |
| 9 | |
| 10 | Fix comparisons. There's some nasty stuff here: when two types are |
| 11 | not the same, and they're not instances, the fallback code doesn't |
| 12 | account for the possibility that they might be subtypes of a common |
| 13 | base type that defines a comparison. |
| 14 | |
| 15 | Fix subtype_dealloc(). This currently searches through the list of |
| 16 | base types until it finds a type whose tp_dealloc is not |
| 17 | subtype_dealloc. I think this is not safe. I think the alloc/dealloc |
| 18 | policy needs to be rethought. *** There's an idea here that I haven't |
| 19 | worked out yet: just as object creation now has separate API's tp_new, |
| 20 | tp_alloc, and tp_init, destruction has tp_dealloc and tp_free. (Maybe |
| 21 | tp_fini should be added to correspond to tp_init?) Something |
| 22 | could/should be done with this. *** |
| 23 | |
| 24 | Clean up isinstance(), issubclass() and their C equivalents. There |
| 25 | are a bunch of different APIs here and not all of them do the right |
| 26 | thing yet. There should be fewer APIs and their implementation should |
| 27 | be simpler. The old "abstract subclass" test should probably |
| 28 | disappear (if we want to root out ExtensionClass). *** I think I've |
| 29 | done 90% of this by creating PyType_IsSubtype() and using it |
| 30 | appropriately. For now, the old "abstract subclass" test is still |
| 31 | there, and there may be some places where PyObject_IsSubclass() is |
| 32 | called where PyType_IsSubtype() would be more appropriate. *** |
| 33 | |
| 34 | Check for conflicts between base classes. I fear that the rules used |
| 35 | to decide whether multiple bases have conflicting instance variables |
| 36 | aren't strict enough. I think that sometimes two different classes |
| 37 | adding __dict__ may be incompatible after all. |
| 38 | |
| 39 | Check for order conflicts. Suppose there are two base classes X and |
| 40 | Y. Suppose class B derives from X and Y, and class C from Y and X (in |
| 41 | that order). Now suppose class D derives from B and C. In which |
| 42 | order should the base classes X and Y be searched? This is an order |
| 43 | conflict, and should be disallowed; currently the test for this is not |
| 44 | implemented. |
| 45 | |
| 46 | Clean up the GC interface. Currently, tp_basicsize includes the GC |
| 47 | head size iff tp_flags includes the GC flag bit. This makes object |
| 48 | size math a pain (e.g. to see if two object types have the same |
| 49 | instance size, you can't just compare the tp_basicsize fields -- you |
| 50 | have to conditionally subtract the GC head size). Neil has a patch |
| 51 | that improves the API in this area, but it's backwards incompatible. |
| 52 | (http://sf.net/tracker/?func=detail&aid=421893&group_id=5470&atid=305470) |
| 53 | I think I know of a way to fix the incompatibility (by switching to a |
| 54 | different flag bit). *** Tim proposed a better idea: macros to access |
| 55 | tp_basicsize while hiding the nastiness. This is done now, so I think |
| 56 | the rest of this task needn't be done. *** |
| 57 | |
| 58 | Make the __dict__ of types declared with Python class statements |
| 59 | writable -- only statically declared types must have an immutable |
| 60 | dict, because they're shared between interpreter instances. Possibly |
| 61 | trap writes to the __dict__ to update the corresponding tp_<slot> if |
| 62 | an __<slot>__ name is affected. *** Done as part of the next task. *** |
| 63 | |
| 64 | It should be an option (maybe a different metaclass, maybe a flag) to |
| 65 | *not* merge __dict__ with all the bases, but instead search the |
| 66 | __dict__ (or __introduced__?) of all bases in __mro__ order. (This is |
| 67 | needed anyway to unify classes completely.) *** Partly done. |
| 68 | Inheritance of slots from bases is still icky: (1) MRO is not always |
| 69 | respected when inheriting slots; (2) dynamic classes can't add slot |
| 70 | implementations in Python after creation (e.g., setting C.__hash__ |
| 71 | doesn't set the tp_hash slot). *** |
| 72 | |
| 73 | Universal base class (object). How can we make the object class |
| 74 | subclassable and define simple default methods for everything without |
| 75 | having these inherited by built-in types that don't want these |
| 76 | defaults? *** Done, really. *** |
| 77 | |
| 78 | Add error checking to the MRO calculation. *** Done. *** |
| 79 | |
| 80 | Make __new__ overridable through a Python class method (!). Make more |
| 81 | of the sub-algorithms of type construction available as methods. *** |
| 82 | After I implemented class methods, I found that in order to be able |
| 83 | to make an upcall to Base.__new__() and have it create an instance of |
| 84 | your class (rather than a Base instance), you can't use class methods |
| 85 | -- you must use static methods. So I've implemented those too. I've |
| 86 | hooked up __new__ in the right places, so the first part of this is |
| 87 | now done. I've also exported the MRO calculation and made it |
| 88 | overridable, as metamethod mro(). I believe that closes this topic |
| 89 | for now. I expect that some warts will only be really debugged when |
| 90 | we try to use this for some, eh, interesting types such as tuples. *** |
| 91 | |
| 92 | More -- I'm sure new issues will crop up as we go. |
| 93 | |
| 94 | |
| 95 | Project: loose ends and follow-through |
| 96 | ************************************** |
| 97 | |
| 98 | Tasks: |
| 99 | |
| 100 | Make more (most?) built-in types act as their own factory functions. |
| 101 | |
| 102 | Make more (most?) built-in types subtypable -- with or without |
| 103 | overridable allocation. *** This includes descriptors! It should be |
| 104 | possible to write descriptors in Python, so metaclasses can do clever |
| 105 | things with them. *** |
| 106 | |
| 107 | Exceptions should be types. This changes the rules, since now almost |
| 108 | anything can be raised (as maybe it should). Or should we strive for |
| 109 | enforcement of the convention that all exceptions should be derived |
| 110 | from Exception? String exceptions will be another hassle, to be |
| 111 | deprecated and eventually ruled out. |
| 112 | |
| 113 | Standardize a module containing names for all built-in types, and |
| 114 | standardize on names. E.g. should the official name of the string |
| 115 | type be 'str', 'string', or 'StringType'? |
| 116 | |
| 117 | Create a hierarchy of types, so that e.g. int and long are both |
| 118 | subtypes of an abstract base type integer, which is itself a subtype |
| 119 | of number, etc. A lot of thinking can go into this! |
| 120 | |
| 121 | *** NEW TASK??? *** |
| 122 | Implement "signature" objects. These are alluded to in PEP 252 but |
| 123 | not yet specified. Supposedly they provide an easily usable API to |
| 124 | find out about function/method arguments. Building these for Python |
| 125 | functions is simple. Building these for built-in functions will |
| 126 | require a change to the PyMethodDef structure, so that a type can |
| 127 | provide signature information for its C methods. (This would also |
| 128 | help in supporting keyword arguments for C methods with less work than |
| 129 | PyArg_ParseTupleAndKeywords() currently requires.) But should we do |
| 130 | this? It's additional work and not required for any of the other |
| 131 | parts. |
| 132 | |
| 133 | |
| 134 | Project: making classes use the new machinery |
| 135 | ********************************************* |
| 136 | |
| 137 | Tasks: |
| 138 | |
| 139 | Try to get rid of all code in classobject.c by deferring to the new |
| 140 | mechanisms. How far can we get without breaking backwards |
| 141 | compatibility? This is underspecified because I haven't thought much |
| 142 | about it yet. Can we lose the use of PyInstance_Check() everywhere? |
| 143 | I would hope so! |
| 144 | |
| 145 | |
| 146 | Project: backwards compatibility |
| 147 | ******************************** |
| 148 | |
| 149 | Tasks: |
| 150 | |
| 151 | Make sure all code checks the proper tp_flags bit before accessing |
| 152 | type object fields. |
| 153 | |
| 154 | Identify areas of incompatibility with Python 2.1. Design solutions. |
| 155 | Implement and test. |
| 156 | |
| 157 | Some specific areas: a fair amount of code probably depends on |
| 158 | specific types having __members__ and/or __methods__ attributes. |
| 159 | These are currently not present (conformant to PEP 252, which proposes |
| 160 | to drop them) but we may have to add them back. This can be done in a |
| 161 | generic way with not too much effort. Tim adds: Perhaps that dir(object) |
| 162 | rarely returns anything but [] now is a consequence of this. I'm very |
| 163 | used to doing, e.g., dir([]) or dir("") in an interactive shell to jog my |
| 164 | memory; also one of the reasons test_generators failed. |
| 165 | |
| 166 | Another area: going all the way with classes and instances means that |
| 167 | type(x) == types.InstanceType won't work any more to detect instances. |
| 168 | Should there be a mode where this still works? Maybe this should be |
| 169 | the default mode, with a warning, and an explicit way to get the new |
| 170 | way to work? (Instead of a __future__ statement, I'm thinking of a |
| 171 | module global __metaclass__ which would provide the default metaclass |
| 172 | for baseless class statements.) |
| 173 | |
| 174 | |
| 175 | Project: testing |
| 176 | **************** |
| 177 | |
| 178 | Tasks: |
| 179 | |
| 180 | Identify new functionality that needs testing. Conceive unit tests |
| 181 | for all new functionality. Conceive stress tests for critical |
| 182 | features. Run the tests. Fix bugs. Repeat until satisfied. |
| 183 | |
| 184 | Note: this may interact with the branch integration task. |
| 185 | |
| 186 | |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 187 | Project: integration with main branch *** This is done - tim *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 188 | ************************************* |
| 189 | |
| 190 | Tasks: |
| 191 | |
| 192 | Merge changes in the HEAD branch into the descr-branch. Then merge |
| 193 | the descr-branch back into the HEAD branch. |
| 194 | |
| 195 | The longer we wait, the more effort this will be -- the descr-branch |
| 196 | forked off quite a long time ago, and there are changes everywhere in |
| 197 | the HEAD branch (e.g. the dict object has been radically rewritten). |
| 198 | |
| 199 | On the other hand, if we do this too early, we'll have to do it again |
| 200 | later. |
| 201 | |
| 202 | Note from Tim: We should never again wait until literally 100s of files |
| 203 | are out of synch. I don't care how often I need to do this, provided only |
| 204 | that it's a tractable task each time. Once per week sounds like a good |
| 205 | idea. As is, even the trunk change to rangeobject.c created more than its |
| 206 | proper share of merge headaches, because it confused all the other reasons |
| 207 | include file merges were getting conflicts (the more changes there are, the |
| 208 | worse diff does; indeed, I came up with the ndiff algorithm in the 80s |
| 209 | precisely because the source-control diff program Cray used at the time |
| 210 | produced minimal but *senseless* diffs, thus creating artificial conflicts; |
| 211 | paying unbounded attention to context does a much better job of putting |
| 212 | changes where they make semantic sense too; but we're stuck with Unix diff |
| 213 | here, and it isn't robust in this sense; if we don't keep its job simple, |
| 214 | it will make my job hell). |
| 215 | |
| 216 | Done: |
| 217 | To undo or rename before final merge: Modules/spam.c has worked its |
| 218 | way into the branch Unix and Windows builds (pythoncore.dsp and |
| 219 | PC/config.c); also imported by test_descr.py. How about renaming to |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 220 | xxsubtype.c (whatever) now? *** this is done - tim *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 221 | |
| 222 | |
| 223 | Project: performance tuning |
| 224 | *************************** |
| 225 | |
| 226 | Tasks: |
| 227 | |
| 228 | Pick or create a general performance benchmark for Python. Benchmark |
| 229 | the new system vs. the old system. Profile the new system. Improve |
| 230 | hotspots. Repeat until satisfied. |
| 231 | |
| 232 | Note: this may interact with the branch integration task. |
| 233 | |
| 234 | |
| 235 | Project: documentation |
| 236 | ********************** |
| 237 | |
| 238 | Tasks: |
| 239 | |
| 240 | Update PEP 252 (descriptors). Describe more of the prototype |
| 241 | implementation |
| 242 | |
| 243 | Update PEP 253 (subtyping). Complicated architectural wrangling with |
| 244 | metaclasses. There is an interaction between implementation and |
| 245 | description. |
| 246 | |
| 247 | Write PEP 254 (unification of classes). This should discuss what |
| 248 | changes for ordinary classes, and how we can make it more b/w |
| 249 | compatible. |
| 250 | |
| 251 | Other documentation. There needs to be user documentation, |
| 252 | eventually. |
| 253 | |
| 254 | |
| 255 | Project: community interaction |
| 256 | ****************************** |
| 257 | |
| 258 | Tasks: |
| 259 | |
| 260 | Once the PEPs are written, solicit community feedback, and formulate |
| 261 | responses to the feedback. Give the community enough time to think |
| 262 | over this complicated proposal. Provide the community with a |
| 263 | prototype implementation to test. Try to do this *before* casting |
| 264 | everything in stone! |
| 265 | |
| 266 | MERGE BEGIN **************************************************************** |
| 267 | Merge details (this section is Tim's scratchpad, but should help a lot if |
| 268 | he dies of frustration while wrestling with CVS <0.9 wink>). |
| 269 | ---------------------------------------------------------------------------- |
| 270 | 2001-08-01 Merging descr-branch back into trunk. |
| 271 | |
| 272 | Tagged trunk about 22:05: |
| 273 | cvs tag date2001-08-01 python |
| 274 | |
| 275 | Merged trunk delta into branch: |
| 276 | cvs -q -z3 up -j date2001-07-30 -j date2001-08-01 descr |
| 277 | |
| 278 | No conflicts (! first time ever!) ... but problems with pythoncore.dsp. |
| 279 | Resolved. |
| 280 | |
| 281 | Rebuilt from scratch; ran all tests; checked into branch about 22:40. |
| 282 | |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 283 | Merged descr-branch back into trunk (SEE BELOW -- this specific way of |
| 284 | doing it was a bad idea): |
| 285 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 286 | cvs -q -z3 up -j descr-branch python |
| 287 | |
| 288 | 34 conflicts. Hmm! OK, looks like every file in the project with an |
| 289 | embedded RCS Id is "a conflict". Others make no sense, e.g., a dozen |
| 290 | conflicts in dictobject.c, sometimes enclosing identical(!) blobs of |
| 291 | source code. And CVS remains utterly baffled by Python type object decls. |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 292 | Every line of ceval.c's generator code is in conflict blocks ... OK, |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 293 | there's no pattern or sense here, I'll just deal with it. |
| 294 | |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 295 | Conflicts resolved; rebuilt from scratch; test_weakref fails. Didn't find |
| 296 | an obvious reason and it was late, so committed it anyway. Tagged the |
| 297 | trunk then with tag: |
| 298 | |
| 299 | after-descr-branch-merge |
| 300 | |
| 301 | Tracked the test_weakref failure to a botched conflict resolution in |
| 302 | classobject.c; checked in a fix. |
| 303 | |
| 304 | LATER: The merge should have been done via: |
| 305 | |
| 306 | upd -j date2001-08-01 -j descr-branch python |
| 307 | |
| 308 | instead. This would have caused only one conflict, a baffler in |
| 309 | bltinmodule.c. It would have avoided the classobject.c error I made. |
| 310 | Luckily, except for that one, we got to the same place in the end anyway, |
| 311 | apart from a few curious tabs-vs-spaces differences. |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 312 | ---------------------------------------------------------------------------- |
| 313 | 2001-07-30 |
| 314 | |
| 315 | Doing this again while the expat and Windows installer changes are still |
| 316 | fresh on my mind. |
| 317 | |
| 318 | Tagged trunk about 23:50 EDT on the 29th: |
| 319 | cvs tag date2001-07-30 python |
| 320 | |
| 321 | Merged trunk delta into branch: |
| 322 | |
| 323 | cvs -q -z3 up -j date2001-07-28 -j date2001-07-30 descr |
| 324 | |
| 325 | 2 conflicts, resolved. |
| 326 | ---------------------------------------------------------------------------- |
| 327 | 2001-07-28 |
| 328 | |
| 329 | Tagged trunk about 00:31 EDT: |
| 330 | cvs tag date2001-07-28 python |
| 331 | |
| 332 | Merged trunk delta into branch: |
| 333 | cvs -q -z3 up -j date2001-07-21 -j date2001-07-28 descr |
| 334 | |
| 335 | 4 conflicts, all RCS Ids. Resolved. |
| 336 | ---------------------------------------------------------------------------- |
| 337 | 2001-07-21 |
| 338 | |
| 339 | Tagged trunk about 01:00 EDT: |
| 340 | cvs tag date2001-07-21 python |
| 341 | |
| 342 | Merged trunk delta into branch: |
| 343 | cvs -q -z3 up -j date2001-07-17b -j date2001-07-21 descr |
| 344 | |
| 345 | 4 conflicts, mostly RCS Id thingies. Resolved. |
| 346 | |
| 347 | Legit failure in new test_repr, because repr.py dispatches on the exact |
| 348 | string returned by type(x). type(1L) and type('s') differ in descr-branch |
| 349 | now, and repr.py didn't realize that, falling back to the "unknown type" |
| 350 | case for longs and strings. Repaired descr-branch repr.py. |
| 351 | ---------------------------------------------------------------------------- |
| 352 | 2001-07-19 |
| 353 | |
| 354 | Removed the r22a1-branch tag (see next entry). Turns out Guido did add a |
| 355 | r22a1 tag, so the r22a1-branch tag served no point anymore. |
| 356 | ---------------------------------------------------------------------------- |
| 357 | 2001-07-18 2.2a1 releaase |
| 358 | |
| 359 | Immediately after the merge just below, I tagged descr-branch via |
| 360 | |
| 361 | cvs tag r22a1-branch descr |
| 362 | |
| 363 | Guido may or may not want to add another tag here (? maybe he wants to do |
| 364 | some more Unix fiddling first). |
| 365 | ---------------------------------------------------------------------------- |
| 366 | 2001-07-17 building 2.2a1 release, from descr-branch |
| 367 | |
| 368 | Tagged trunk about 22:00 EDT, like so: |
| 369 | cvs tag date2001-07-17b python |
| 370 | |
| 371 | Merged trunk delta into branch via: |
| 372 | cvs -q -z3 up -j date2001-07-17a -j date2001-07-17b descr |
| 373 | ---------------------------------------------------------------------------- |
| 374 | 2001-07-17 |
| 375 | |
| 376 | Tagged trunk about 00:05 EDT, like so: |
| 377 | cvs tag date2001-07-17a python |
| 378 | |
| 379 | Merged trunk delta into branch via: |
| 380 | cvs -q -z3 up -j date2001-07-16 -j date2001-07-17a descr |
| 381 | ---------------------------------------------------------------------------- |
| 382 | 2001-07-16 |
| 383 | |
| 384 | Tagged trunk about 15:20 EDT, like so: |
| 385 | cvs tag date2001-07-16 python |
| 386 | |
| 387 | Guido then added all the other dist/ directories to descr-branch from that |
| 388 | trunk tag. |
| 389 | |
| 390 | Tim then merged trunk delta into the branch via: |
| 391 | cvs -q -z3 up -j date2001-07-15 -j date2001-07-16 descr |
| 392 | ---------------------------------------------------------------------------- |
| 393 | 2001-07-15 |
| 394 | |
| 395 | Tagged trunk about 15:44 EDT, like so: |
| 396 | cvs tag date2001-07-15 python |
| 397 | |
| 398 | Merged trunk delta into branch via: |
| 399 | cvs -q -z3 up -j date2001-07-13 -j date2001-07-15 descr |
| 400 | |
| 401 | Four files with conflicts, all artificial RCS Id & Revision thingies. |
| 402 | Resolved and committed. |
| 403 | ---------------------------------------------------------------------------- |
| 404 | 2001-07-13 |
| 405 | |
| 406 | Tagged trunk about 22:13 EDT, like so: |
| 407 | cvs tag date2001-07-13 python |
| 408 | |
| 409 | Merged trunk delta into branch via: |
| 410 | cvs -q -z3 up -j date2001-07-06 -j date2001-07-13 descr |
| 411 | |
| 412 | Six(!) files with conflicts, mostly related to NeilS's generator gc patches. |
| 413 | Unsure why, but CVS seems always to think there are conflicts whenever a |
| 414 | line in a type object decl gets changed, and the conflict marking seems |
| 415 | maximally confused in these cases. Anyway, since I reviewed those patches |
| 416 | on the trunk, good thing I'm merging them, and darned glad it's still fresh |
| 417 | on my mind. |
| 418 | |
| 419 | Resolved the conflicts, and committed the changes in a few hours total. |
| 420 | ---------------------------------------------------------------------------- |
| 421 | 2001-07-07 |
| 422 | |
| 423 | Merge of trunk tag date2001-07-06 into descr-branch, via |
| 424 | cvs -q -z3 up -j date2001-07-06 mergedescr |
| 425 | was committed on 2001-07-07. |
| 426 | |
| 427 | Merge issues: |
| 428 | |
| 429 | (all resolved -- GvR) |
| 430 | ---------------------------------------------------------------------------- |
| 431 | 2001-07-06 |
| 432 | |
| 433 | Tagged trunk a bit after midnight, like so: |
| 434 | |
| 435 | C:\Code>cvs tag date2001-07-06 python |
| 436 | cvs server: Tagging python |
| 437 | cvs server: Tagging python/dist |
| 438 | cvs server: Tagging python/dist/src |
| 439 | T python/dist/src/.cvsignore |
| 440 | T python/dist/src/LICENSE |
| 441 | T python/dist/src/Makefile.pre.in |
| 442 | T python/dist/src/README |
| 443 | ... [& about 3000 lines more] ... |
| 444 | |
| 445 | This is the first trunk snapshot to be merged into the descr-branch. |
| 446 | Gave it a date instead of a goofy name because there's going to be more |
| 447 | than one of these, and at least it's obvious which of two ISO dates comes |
| 448 | earlier. These tags should go away after all merging is complete. |
| 449 | MERGE END ****************************************************************** |