Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 1 | Project: core implementation |
| 2 | **************************** |
| 3 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 4 | Still to do |
| 5 | ----------- |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 6 | |
| 7 | Fix comparisons. There's some nasty stuff here: when two types are |
| 8 | not the same, and they're not instances, the fallback code doesn't |
| 9 | account for the possibility that they might be subtypes of a common |
| 10 | base type that defines a comparison. |
| 11 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 12 | Check for conflicts between base classes. I fear that the rules used |
| 13 | to decide whether multiple bases have conflicting instance variables |
| 14 | aren't strict enough. I think that sometimes two different classes |
| 15 | adding __dict__ may be incompatible after all. |
| 16 | |
| 17 | Check for order conflicts. Suppose there are two base classes X and |
| 18 | Y. Suppose class B derives from X and Y, and class C from Y and X (in |
| 19 | that order). Now suppose class D derives from B and C. In which |
| 20 | order should the base classes X and Y be searched? This is an order |
| 21 | conflict, and should be disallowed; currently the test for this is not |
| 22 | implemented. |
| 23 | |
| 24 | Allow __class__ assignment. |
| 25 | |
| 26 | Make __dynamic__ the default. |
| 27 | |
| 28 | Add __del__ handlers. |
| 29 | |
| 30 | Done (mostly) |
| 31 | ------------- |
| 32 | |
| 33 | Do binary operators properly. nb_add should try to call self.__add__ |
| 34 | and other.__radd__. I think I'll exclude base types that define any |
| 35 | binary operator without setting the CHECKTYPES flag. *** This is |
| 36 | done, AFAICT. Even supports __truediv__ and __floordiv__. *** |
| 37 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 38 | Fix subtype_dealloc(). This currently searches through the list of |
| 39 | base types until it finds a type whose tp_dealloc is not |
| 40 | subtype_dealloc. I think this is not safe. I think the alloc/dealloc |
| 41 | policy needs to be rethought. *** There's an idea here that I haven't |
| 42 | worked out yet: just as object creation now has separate API's tp_new, |
| 43 | tp_alloc, and tp_init, destruction has tp_dealloc and tp_free. (Maybe |
| 44 | tp_fini should be added to correspond to tp_init?) Something |
| 45 | could/should be done with this. *** |
| 46 | |
| 47 | Clean up isinstance(), issubclass() and their C equivalents. There |
| 48 | are a bunch of different APIs here and not all of them do the right |
| 49 | thing yet. There should be fewer APIs and their implementation should |
| 50 | be simpler. The old "abstract subclass" test should probably |
| 51 | disappear (if we want to root out ExtensionClass). *** I think I've |
| 52 | done 90% of this by creating PyType_IsSubtype() and using it |
| 53 | appropriately. For now, the old "abstract subclass" test is still |
| 54 | there, and there may be some places where PyObject_IsSubclass() is |
| 55 | called where PyType_IsSubtype() would be more appropriate. *** |
| 56 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 57 | Clean up the GC interface. Currently, tp_basicsize includes the GC |
| 58 | head size iff tp_flags includes the GC flag bit. This makes object |
| 59 | size math a pain (e.g. to see if two object types have the same |
| 60 | instance size, you can't just compare the tp_basicsize fields -- you |
| 61 | have to conditionally subtract the GC head size). Neil has a patch |
| 62 | that improves the API in this area, but it's backwards incompatible. |
| 63 | (http://sf.net/tracker/?func=detail&aid=421893&group_id=5470&atid=305470) |
| 64 | I think I know of a way to fix the incompatibility (by switching to a |
| 65 | different flag bit). *** Tim proposed a better idea: macros to access |
| 66 | tp_basicsize while hiding the nastiness. This is done now, so I think |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 67 | the rest of this task needn't be done. *** *** Neil checked in a |
| 68 | much improved version of his idea, and it's all squared away. *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 69 | |
| 70 | Make the __dict__ of types declared with Python class statements |
| 71 | writable -- only statically declared types must have an immutable |
| 72 | dict, because they're shared between interpreter instances. Possibly |
| 73 | trap writes to the __dict__ to update the corresponding tp_<slot> if |
| 74 | an __<slot>__ name is affected. *** Done as part of the next task. *** |
| 75 | |
| 76 | It should be an option (maybe a different metaclass, maybe a flag) to |
| 77 | *not* merge __dict__ with all the bases, but instead search the |
| 78 | __dict__ (or __introduced__?) of all bases in __mro__ order. (This is |
| 79 | needed anyway to unify classes completely.) *** Partly done. |
| 80 | Inheritance of slots from bases is still icky: (1) MRO is not always |
| 81 | respected when inheriting slots; (2) dynamic classes can't add slot |
| 82 | implementations in Python after creation (e.g., setting C.__hash__ |
| 83 | doesn't set the tp_hash slot). *** |
| 84 | |
| 85 | Universal base class (object). How can we make the object class |
| 86 | subclassable and define simple default methods for everything without |
| 87 | having these inherited by built-in types that don't want these |
| 88 | defaults? *** Done, really. *** |
| 89 | |
| 90 | Add error checking to the MRO calculation. *** Done. *** |
| 91 | |
| 92 | Make __new__ overridable through a Python class method (!). Make more |
| 93 | of the sub-algorithms of type construction available as methods. *** |
| 94 | After I implemented class methods, I found that in order to be able |
| 95 | to make an upcall to Base.__new__() and have it create an instance of |
| 96 | your class (rather than a Base instance), you can't use class methods |
| 97 | -- you must use static methods. So I've implemented those too. I've |
| 98 | hooked up __new__ in the right places, so the first part of this is |
| 99 | now done. I've also exported the MRO calculation and made it |
| 100 | overridable, as metamethod mro(). I believe that closes this topic |
| 101 | for now. I expect that some warts will only be really debugged when |
| 102 | we try to use this for some, eh, interesting types such as tuples. *** |
| 103 | |
Guido van Rossum | 0415620 | 2001-08-08 16:57:43 +0000 | [diff] [blame] | 104 | There was a sequel to the __new__ story (see checkins). There |
| 105 | still is a problem: object.__new__ now no longer exists, because |
| 106 | it was inherited by certain extension types that could break. But |
| 107 | now when I write |
| 108 | |
| 109 | class C(object): |
| 110 | def __new__(cls, *args): |
| 111 | "How do I call the default __new__ implementation???" |
| 112 | |
Guido van Rossum | 42a8c2b | 2001-08-09 20:25:58 +0000 | [diff] [blame] | 113 | This was resolved nicely by putting object.__new__ back but not |
| 114 | inheriting __new__ from object when the subtype is a built-in or |
| 115 | extension type. |
| 116 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 117 | More -- I'm sure new issues will crop up as we go. |
| 118 | |
| 119 | |
| 120 | Project: loose ends and follow-through |
| 121 | ************************************** |
| 122 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 123 | Still to do |
| 124 | ----------- |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 125 | |
| 126 | Exceptions should be types. This changes the rules, since now almost |
| 127 | anything can be raised (as maybe it should). Or should we strive for |
| 128 | enforcement of the convention that all exceptions should be derived |
| 129 | from Exception? String exceptions will be another hassle, to be |
| 130 | deprecated and eventually ruled out. |
| 131 | |
| 132 | Standardize a module containing names for all built-in types, and |
| 133 | standardize on names. E.g. should the official name of the string |
| 134 | type be 'str', 'string', or 'StringType'? |
| 135 | |
| 136 | Create a hierarchy of types, so that e.g. int and long are both |
| 137 | subtypes of an abstract base type integer, which is itself a subtype |
| 138 | of number, etc. A lot of thinking can go into this! |
| 139 | |
| 140 | *** NEW TASK??? *** |
| 141 | Implement "signature" objects. These are alluded to in PEP 252 but |
| 142 | not yet specified. Supposedly they provide an easily usable API to |
| 143 | find out about function/method arguments. Building these for Python |
| 144 | functions is simple. Building these for built-in functions will |
| 145 | require a change to the PyMethodDef structure, so that a type can |
| 146 | provide signature information for its C methods. (This would also |
| 147 | help in supporting keyword arguments for C methods with less work than |
| 148 | PyArg_ParseTupleAndKeywords() currently requires.) But should we do |
| 149 | this? It's additional work and not required for any of the other |
| 150 | parts. |
| 151 | |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 152 | Done (mostly) |
| 153 | ------------- |
| 154 | |
| 155 | Make more (most?) built-in types act as their own factory functions. |
| 156 | *** Done for all reasonable built-in types. *** |
| 157 | |
| 158 | Make more (most?) built-in types subtypable -- with or without |
| 159 | overridable allocation. *** This includes descriptors! It should be |
| 160 | possible to write descriptors in Python, so metaclasses can do clever |
| 161 | things with them. *** *** Done for most reasonable built-in types, |
| 162 | except for descriptors *** |
| 163 | |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 164 | |
| 165 | Project: making classes use the new machinery |
| 166 | ********************************************* |
| 167 | |
| 168 | Tasks: |
| 169 | |
| 170 | Try to get rid of all code in classobject.c by deferring to the new |
| 171 | mechanisms. How far can we get without breaking backwards |
| 172 | compatibility? This is underspecified because I haven't thought much |
| 173 | about it yet. Can we lose the use of PyInstance_Check() everywhere? |
Guido van Rossum | eb9f384 | 2001-08-30 21:18:04 +0000 | [diff] [blame] | 174 | I would hope so! *** I'm dropping this goal for now -- classic |
| 175 | classes will be 99% unchanged. *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 176 | |
| 177 | |
| 178 | Project: backwards compatibility |
| 179 | ******************************** |
| 180 | |
| 181 | Tasks: |
| 182 | |
| 183 | Make sure all code checks the proper tp_flags bit before accessing |
| 184 | type object fields. |
| 185 | |
| 186 | Identify areas of incompatibility with Python 2.1. Design solutions. |
| 187 | Implement and test. |
| 188 | |
| 189 | Some specific areas: a fair amount of code probably depends on |
| 190 | specific types having __members__ and/or __methods__ attributes. |
| 191 | These are currently not present (conformant to PEP 252, which proposes |
| 192 | to drop them) but we may have to add them back. This can be done in a |
| 193 | generic way with not too much effort. Tim adds: Perhaps that dir(object) |
| 194 | rarely returns anything but [] now is a consequence of this. I'm very |
| 195 | used to doing, e.g., dir([]) or dir("") in an interactive shell to jog my |
| 196 | memory; also one of the reasons test_generators failed. |
| 197 | |
| 198 | Another area: going all the way with classes and instances means that |
| 199 | type(x) == types.InstanceType won't work any more to detect instances. |
| 200 | Should there be a mode where this still works? Maybe this should be |
| 201 | the default mode, with a warning, and an explicit way to get the new |
| 202 | way to work? (Instead of a __future__ statement, I'm thinking of a |
| 203 | module global __metaclass__ which would provide the default metaclass |
| 204 | for baseless class statements.) |
| 205 | |
| 206 | |
| 207 | Project: testing |
| 208 | **************** |
| 209 | |
| 210 | Tasks: |
| 211 | |
| 212 | Identify new functionality that needs testing. Conceive unit tests |
| 213 | for all new functionality. Conceive stress tests for critical |
| 214 | features. Run the tests. Fix bugs. Repeat until satisfied. |
| 215 | |
| 216 | Note: this may interact with the branch integration task. |
| 217 | |
| 218 | |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 219 | Project: integration with main branch *** This is done - tim *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 220 | ************************************* |
| 221 | |
| 222 | Tasks: |
| 223 | |
| 224 | Merge changes in the HEAD branch into the descr-branch. Then merge |
| 225 | the descr-branch back into the HEAD branch. |
| 226 | |
| 227 | The longer we wait, the more effort this will be -- the descr-branch |
| 228 | forked off quite a long time ago, and there are changes everywhere in |
| 229 | the HEAD branch (e.g. the dict object has been radically rewritten). |
| 230 | |
| 231 | On the other hand, if we do this too early, we'll have to do it again |
| 232 | later. |
| 233 | |
| 234 | Note from Tim: We should never again wait until literally 100s of files |
| 235 | are out of synch. I don't care how often I need to do this, provided only |
| 236 | that it's a tractable task each time. Once per week sounds like a good |
| 237 | idea. As is, even the trunk change to rangeobject.c created more than its |
| 238 | proper share of merge headaches, because it confused all the other reasons |
| 239 | include file merges were getting conflicts (the more changes there are, the |
| 240 | worse diff does; indeed, I came up with the ndiff algorithm in the 80s |
| 241 | precisely because the source-control diff program Cray used at the time |
| 242 | produced minimal but *senseless* diffs, thus creating artificial conflicts; |
| 243 | paying unbounded attention to context does a much better job of putting |
| 244 | changes where they make semantic sense too; but we're stuck with Unix diff |
| 245 | here, and it isn't robust in this sense; if we don't keep its job simple, |
| 246 | it will make my job hell). |
| 247 | |
| 248 | Done: |
| 249 | To undo or rename before final merge: Modules/spam.c has worked its |
| 250 | way into the branch Unix and Windows builds (pythoncore.dsp and |
| 251 | PC/config.c); also imported by test_descr.py. How about renaming to |
Tim Peters | f980301 | 2001-08-02 22:06:35 +0000 | [diff] [blame] | 252 | xxsubtype.c (whatever) now? *** this is done - tim *** |
Tim Peters | 6d6c1a3 | 2001-08-02 04:15:00 +0000 | [diff] [blame] | 253 | |
| 254 | |
| 255 | Project: performance tuning |
| 256 | *************************** |
| 257 | |
| 258 | Tasks: |
| 259 | |
| 260 | Pick or create a general performance benchmark for Python. Benchmark |
| 261 | the new system vs. the old system. Profile the new system. Improve |
| 262 | hotspots. Repeat until satisfied. |
| 263 | |
| 264 | Note: this may interact with the branch integration task. |
| 265 | |
| 266 | |
| 267 | Project: documentation |
| 268 | ********************** |
| 269 | |
| 270 | Tasks: |
| 271 | |
| 272 | Update PEP 252 (descriptors). Describe more of the prototype |
| 273 | implementation |
| 274 | |
| 275 | Update PEP 253 (subtyping). Complicated architectural wrangling with |
| 276 | metaclasses. There is an interaction between implementation and |
| 277 | description. |
| 278 | |
| 279 | Write PEP 254 (unification of classes). This should discuss what |
| 280 | changes for ordinary classes, and how we can make it more b/w |
| 281 | compatible. |
| 282 | |
| 283 | Other documentation. There needs to be user documentation, |
| 284 | eventually. |
| 285 | |
| 286 | |
| 287 | Project: community interaction |
| 288 | ****************************** |
| 289 | |
| 290 | Tasks: |
| 291 | |
| 292 | Once the PEPs are written, solicit community feedback, and formulate |
| 293 | responses to the feedback. Give the community enough time to think |
| 294 | over this complicated proposal. Provide the community with a |
| 295 | prototype implementation to test. Try to do this *before* casting |
| 296 | everything in stone! |