Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 1 | Project: core implementation |
| 2 | **************************** |
| 3 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 4 | Still to do |
| 5 | ----------- |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 6 | |
Guido van Rossum | 50fda3b | 2001-10-04 19:46:06 +0000 | [diff] [blame] | 7 | Add __del__ handlers? |
Guido van Rossum | 43b9a08 | 2001-09-28 18:19:21 +0000 | [diff] [blame] | 8 | |
| 9 | Allow assignment to __bases__ and __dict__? |
| 10 | |
| 11 | Support mixed multiple inheritance from classic and new-style classes? |
| 12 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 13 | Check for conflicts between base classes. I fear that the rules used |
| 14 | to decide whether multiple bases have conflicting instance variables |
| 15 | aren't strict enough. I think that sometimes two different classes |
| 16 | adding __dict__ may be incompatible after all. |
| 17 | |
| 18 | Check for order conflicts. Suppose there are two base classes X and |
| 19 | Y. Suppose class B derives from X and Y, and class C from Y and X (in |
| 20 | that order). Now suppose class D derives from B and C. In which |
| 21 | order should the base classes X and Y be searched? This is an order |
| 22 | conflict, and should be disallowed; currently the test for this is not |
| 23 | implemented. |
| 24 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 25 | Done (mostly) |
| 26 | ------------- |
| 27 | |
Guido van Rossum | 27b7f9f | 2001-10-12 17:43:43 +0000 | [diff] [blame] | 28 | More performance work -- one particular test, test_descr.inherits(), |
| 29 | is still about 50% slower with dynamic classes. :-( The approach of |
| 30 | choice would be: |
| 31 | |
| 32 | Add a list of weak refs to derived classes to each dynamic |
| 33 | class, and trap setattr+delattr on the base class so that they |
| 34 | update the tp_XXX slot in each derived class when the base class |
| 35 | __XXX__ gets set or deleted. More work, but more gain (zero waste |
| 36 | in slot_tp_XXX when __XXX__ is not overridden). |
| 37 | *** That's done now, with great success. *** |
| 38 | |
Guido van Rossum | 50fda3b | 2001-10-04 19:46:06 +0000 | [diff] [blame] | 39 | Make __dynamic__ the default. *** done (but more performance work |
| 40 | needs to be done). *** |
| 41 | |
Guido van Rossum | f49a913 | 2001-10-03 03:00:56 +0000 | [diff] [blame] | 42 | Treat all binary operators the same way as I just did for rich |
| 43 | comparison: in a <op> b, if type(a) is not type(b) and isinstance(b, |
| 44 | type(a)), try b.__rop__(a) before trying a.__op__(b). *** Done. *** |
| 45 | |
Guido van Rossum | c39553a | 2001-09-25 06:20:52 +0000 | [diff] [blame] | 46 | Fix comparisons. There's some nasty stuff here: when two types are |
| 47 | not the same, and they're not instances, the fallback code doesn't |
| 48 | account for the possibility that they might be subtypes of a common |
| 49 | base type that defines a comparison. *** I believe this is now done, |
| 50 | but it's a bit of a mess. *** |
| 51 | |
| 52 | Allow __class__ assignment. *** done *** |
| 53 | |
| 54 | Change __getattr__ to be more like classic __getattr__, and introduce |
| 55 | a new name for new-style __getattr__. *** Done. The new-style method |
| 56 | is called __getattribute__. *** |
| 57 | |
Guido van Rossum | ed8fa72 | 2001-09-20 05:27:24 +0000 | [diff] [blame] | 58 | Make inspect and pydoc do the right thing for new-style classes. *** |
| 59 | done *** |
| 60 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 61 | Do binary operators properly. nb_add should try to call self.__add__ |
| 62 | and other.__radd__. I think I'll exclude base types that define any |
| 63 | binary operator without setting the CHECKTYPES flag. *** This is |
| 64 | done, AFAICT. Even supports __truediv__ and __floordiv__. *** |
| 65 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 66 | Fix subtype_dealloc(). This currently searches through the list of |
| 67 | base types until it finds a type whose tp_dealloc is not |
| 68 | subtype_dealloc. I think this is not safe. I think the alloc/dealloc |
| 69 | policy needs to be rethought. *** There's an idea here that I haven't |
| 70 | worked out yet: just as object creation now has separate API's tp_new, |
| 71 | tp_alloc, and tp_init, destruction has tp_dealloc and tp_free. (Maybe |
| 72 | tp_fini should be added to correspond to tp_init?) Something |
| 73 | could/should be done with this. *** |
| 74 | |
| 75 | Clean up isinstance(), issubclass() and their C equivalents. There |
| 76 | are a bunch of different APIs here and not all of them do the right |
| 77 | thing yet. There should be fewer APIs and their implementation should |
| 78 | be simpler. The old "abstract subclass" test should probably |
| 79 | disappear (if we want to root out ExtensionClass). *** I think I've |
| 80 | done 90% of this by creating PyType_IsSubtype() and using it |
| 81 | appropriately. For now, the old "abstract subclass" test is still |
| 82 | there, and there may be some places where PyObject_IsSubclass() is |
| 83 | called where PyType_IsSubtype() would be more appropriate. *** |
| 84 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 85 | Clean up the GC interface. Currently, tp_basicsize includes the GC |
| 86 | head size iff tp_flags includes the GC flag bit. This makes object |
| 87 | size math a pain (e.g. to see if two object types have the same |
| 88 | instance size, you can't just compare the tp_basicsize fields -- you |
| 89 | have to conditionally subtract the GC head size). Neil has a patch |
| 90 | that improves the API in this area, but it's backwards incompatible. |
| 91 | (http://sf.net/tracker/?func=detail&aid=421893&group_id=5470&atid=305470) |
| 92 | I think I know of a way to fix the incompatibility (by switching to a |
| 93 | different flag bit). *** Tim proposed a better idea: macros to access |
| 94 | tp_basicsize while hiding the nastiness. This is done now, so I think |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 95 | the rest of this task needn't be done. *** *** Neil checked in a |
| 96 | much improved version of his idea, and it's all squared away. *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 97 | |
| 98 | Make the __dict__ of types declared with Python class statements |
| 99 | writable -- only statically declared types must have an immutable |
| 100 | dict, because they're shared between interpreter instances. Possibly |
| 101 | trap writes to the __dict__ to update the corresponding tp_<slot> if |
| 102 | an __<slot>__ name is affected. *** Done as part of the next task. *** |
| 103 | |
| 104 | It should be an option (maybe a different metaclass, maybe a flag) to |
| 105 | *not* merge __dict__ with all the bases, but instead search the |
| 106 | __dict__ (or __introduced__?) of all bases in __mro__ order. (This is |
| 107 | needed anyway to unify classes completely.) *** Partly done. |
| 108 | Inheritance of slots from bases is still icky: (1) MRO is not always |
| 109 | respected when inheriting slots; (2) dynamic classes can't add slot |
| 110 | implementations in Python after creation (e.g., setting C.__hash__ |
| 111 | doesn't set the tp_hash slot). *** |
| 112 | |
| 113 | Universal base class (object). How can we make the object class |
| 114 | subclassable and define simple default methods for everything without |
| 115 | having these inherited by built-in types that don't want these |
| 116 | defaults? *** Done, really. *** |
| 117 | |
| 118 | Add error checking to the MRO calculation. *** Done. *** |
| 119 | |
| 120 | Make __new__ overridable through a Python class method (!). Make more |
| 121 | of the sub-algorithms of type construction available as methods. *** |
| 122 | After I implemented class methods, I found that in order to be able |
| 123 | to make an upcall to Base.__new__() and have it create an instance of |
| 124 | your class (rather than a Base instance), you can't use class methods |
| 125 | -- you must use static methods. So I've implemented those too. I've |
| 126 | hooked up __new__ in the right places, so the first part of this is |
| 127 | now done. I've also exported the MRO calculation and made it |
| 128 | overridable, as metamethod mro(). I believe that closes this topic |
| 129 | for now. I expect that some warts will only be really debugged when |
| 130 | we try to use this for some, eh, interesting types such as tuples. *** |
| 131 | |
Guido van Rossum | 0415620 | 2001-08-08 16:57:43 +0000 | [diff] [blame] | 132 | There was a sequel to the __new__ story (see checkins). There |
| 133 | still is a problem: object.__new__ now no longer exists, because |
| 134 | it was inherited by certain extension types that could break. But |
| 135 | now when I write |
| 136 | |
| 137 | class C(object): |
| 138 | def __new__(cls, *args): |
| 139 | "How do I call the default __new__ implementation???" |
| 140 | |
Guido van Rossum | 42a8c2b | 2001-08-09 20:25:58 +0000 | [diff] [blame] | 141 | This was resolved nicely by putting object.__new__ back but not |
| 142 | inheriting __new__ from object when the subtype is a built-in or |
| 143 | extension type. |
| 144 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 145 | More -- I'm sure new issues will crop up as we go. |
| 146 | |
| 147 | |
| 148 | Project: loose ends and follow-through |
| 149 | ************************************** |
| 150 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 151 | Still to do |
| 152 | ----------- |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 153 | |
| 154 | Exceptions should be types. This changes the rules, since now almost |
| 155 | anything can be raised (as maybe it should). Or should we strive for |
| 156 | enforcement of the convention that all exceptions should be derived |
| 157 | from Exception? String exceptions will be another hassle, to be |
| 158 | deprecated and eventually ruled out. |
| 159 | |
| 160 | Standardize a module containing names for all built-in types, and |
| 161 | standardize on names. E.g. should the official name of the string |
| 162 | type be 'str', 'string', or 'StringType'? |
| 163 | |
| 164 | Create a hierarchy of types, so that e.g. int and long are both |
| 165 | subtypes of an abstract base type integer, which is itself a subtype |
| 166 | of number, etc. A lot of thinking can go into this! |
| 167 | |
| 168 | *** NEW TASK??? *** |
| 169 | Implement "signature" objects. These are alluded to in PEP 252 but |
| 170 | not yet specified. Supposedly they provide an easily usable API to |
| 171 | find out about function/method arguments. Building these for Python |
| 172 | functions is simple. Building these for built-in functions will |
| 173 | require a change to the PyMethodDef structure, so that a type can |
| 174 | provide signature information for its C methods. (This would also |
| 175 | help in supporting keyword arguments for C methods with less work than |
| 176 | PyArg_ParseTupleAndKeywords() currently requires.) But should we do |
| 177 | this? It's additional work and not required for any of the other |
| 178 | parts. |
| 179 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 180 | Done (mostly) |
| 181 | ------------- |
| 182 | |
| 183 | Make more (most?) built-in types act as their own factory functions. |
| 184 | *** Done for all reasonable built-in types. *** |
| 185 | |
| 186 | Make more (most?) built-in types subtypable -- with or without |
| 187 | overridable allocation. *** This includes descriptors! It should be |
| 188 | possible to write descriptors in Python, so metaclasses can do clever |
| 189 | things with them. *** *** Done for most reasonable built-in types, |
| 190 | except for descriptors *** |
| 191 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 192 | |
| 193 | Project: making classes use the new machinery |
| 194 | ********************************************* |
| 195 | |
| 196 | Tasks: |
| 197 | |
| 198 | Try to get rid of all code in classobject.c by deferring to the new |
| 199 | mechanisms. How far can we get without breaking backwards |
| 200 | compatibility? This is underspecified because I haven't thought much |
| 201 | about it yet. Can we lose the use of PyInstance_Check() everywhere? |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 202 | I would hope so! *** I'm dropping this goal for now -- classic |
| 203 | classes will be 99% unchanged. *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 204 | |
| 205 | |
| 206 | Project: backwards compatibility |
| 207 | ******************************** |
| 208 | |
| 209 | Tasks: |
| 210 | |
| 211 | Make sure all code checks the proper tp_flags bit before accessing |
| 212 | type object fields. |
| 213 | |
| 214 | Identify areas of incompatibility with Python 2.1. Design solutions. |
| 215 | Implement and test. |
| 216 | |
| 217 | Some specific areas: a fair amount of code probably depends on |
| 218 | specific types having __members__ and/or __methods__ attributes. |
| 219 | These are currently not present (conformant to PEP 252, which proposes |
| 220 | to drop them) but we may have to add them back. This can be done in a |
| 221 | generic way with not too much effort. Tim adds: Perhaps that dir(object) |
| 222 | rarely returns anything but [] now is a consequence of this. I'm very |
| 223 | used to doing, e.g., dir([]) or dir("") in an interactive shell to jog my |
| 224 | memory; also one of the reasons test_generators failed. |
| 225 | |
| 226 | Another area: going all the way with classes and instances means that |
| 227 | type(x) == types.InstanceType won't work any more to detect instances. |
| 228 | Should there be a mode where this still works? Maybe this should be |
| 229 | the default mode, with a warning, and an explicit way to get the new |
| 230 | way to work? (Instead of a __future__ statement, I'm thinking of a |
| 231 | module global __metaclass__ which would provide the default metaclass |
| 232 | for baseless class statements.) |
| 233 | |
| 234 | |
| 235 | Project: testing |
| 236 | **************** |
| 237 | |
| 238 | Tasks: |
| 239 | |
| 240 | Identify new functionality that needs testing. Conceive unit tests |
| 241 | for all new functionality. Conceive stress tests for critical |
| 242 | features. Run the tests. Fix bugs. Repeat until satisfied. |
| 243 | |
| 244 | Note: this may interact with the branch integration task. |
| 245 | |
| 246 | |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 247 | Project: integration with main branch *** This is done - tim *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 248 | ************************************* |
| 249 | |
| 250 | Tasks: |
| 251 | |
| 252 | Merge changes in the HEAD branch into the descr-branch. Then merge |
| 253 | the descr-branch back into the HEAD branch. |
| 254 | |
| 255 | The longer we wait, the more effort this will be -- the descr-branch |
| 256 | forked off quite a long time ago, and there are changes everywhere in |
| 257 | the HEAD branch (e.g. the dict object has been radically rewritten). |
| 258 | |
| 259 | On the other hand, if we do this too early, we'll have to do it again |
| 260 | later. |
| 261 | |
| 262 | Note from Tim: We should never again wait until literally 100s of files |
| 263 | are out of synch. I don't care how often I need to do this, provided only |
| 264 | that it's a tractable task each time. Once per week sounds like a good |
| 265 | idea. As is, even the trunk change to rangeobject.c created more than its |
| 266 | proper share of merge headaches, because it confused all the other reasons |
| 267 | include file merges were getting conflicts (the more changes there are, the |
| 268 | worse diff does; indeed, I came up with the ndiff algorithm in the 80s |
| 269 | precisely because the source-control diff program Cray used at the time |
| 270 | produced minimal but *senseless* diffs, thus creating artificial conflicts; |
| 271 | paying unbounded attention to context does a much better job of putting |
| 272 | changes where they make semantic sense too; but we're stuck with Unix diff |
| 273 | here, and it isn't robust in this sense; if we don't keep its job simple, |
| 274 | it will make my job hell). |
| 275 | |
| 276 | Done: |
| 277 | To undo or rename before final merge: Modules/spam.c has worked its |
| 278 | way into the branch Unix and Windows builds (pythoncore.dsp and |
| 279 | PC/config.c); also imported by test_descr.py. How about renaming to |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 280 | xxsubtype.c (whatever) now? *** this is done - tim *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 281 | |
| 282 | |
| 283 | Project: performance tuning |
| 284 | *************************** |
| 285 | |
| 286 | Tasks: |
| 287 | |
| 288 | Pick or create a general performance benchmark for Python. Benchmark |
| 289 | the new system vs. the old system. Profile the new system. Improve |
| 290 | hotspots. Repeat until satisfied. |
| 291 | |
| 292 | Note: this may interact with the branch integration task. |
| 293 | |
| 294 | |
| 295 | Project: documentation |
| 296 | ********************** |
| 297 | |
| 298 | Tasks: |
| 299 | |
| 300 | Update PEP 252 (descriptors). Describe more of the prototype |
| 301 | implementation |
| 302 | |
| 303 | Update PEP 253 (subtyping). Complicated architectural wrangling with |
| 304 | metaclasses. There is an interaction between implementation and |
| 305 | description. |
| 306 | |
| 307 | Write PEP 254 (unification of classes). This should discuss what |
| 308 | changes for ordinary classes, and how we can make it more b/w |
| 309 | compatible. |
| 310 | |
| 311 | Other documentation. There needs to be user documentation, |
| 312 | eventually. |
| 313 | |
| 314 | |
| 315 | Project: community interaction |
| 316 | ****************************** |
| 317 | |
| 318 | Tasks: |
| 319 | |
| 320 | Once the PEPs are written, solicit community feedback, and formulate |
| 321 | responses to the feedback. Give the community enough time to think |
| 322 | over this complicated proposal. Provide the community with a |
| 323 | prototype implementation to test. Try to do this *before* casting |
| 324 | everything in stone! |