| Jack Jansen | a630813 | 1996-03-18 13:38:52 +0000 | [diff] [blame^] | 1 | <HTML><HEAD><TITLE>Creating a C extension module on the Macintosh</TITLE></HEAD> |
| 2 | <BODY> |
| 3 | <H1>Creating a C extension module on the Macintosh</H1> |
| 4 | <HR> |
| 5 | |
| 6 | This document gives a step-by-step example of how to create a new C |
| 7 | extension module on the mac. For this example, we will create a module |
| 8 | to interface to the programmers' API of InterSLIP, a package that |
| 9 | allows you to use MacTCP (and, hence, all internet services) over a |
| 10 | modem connection. <p> |
| 11 | |
| 12 | <H2>Prerequisites</H2> |
| 13 | |
| 14 | There are a few things you need to pull this off. First and foremost, |
| 15 | you need a C development environment. Actually, you need a specific |
| 16 | development environment, CodeWarrior by <A |
| 17 | HREF="http://www.metrowerks.com/">MetroWerks</A>. You will probably |
| 18 | need the latest version. You may be able to get by with an older |
| 19 | version of CodeWarrior or with another development environment (Up to |
| 20 | about 1994 python was developed with THINK C, and in the dim past it |
| 21 | was compiled with MPW C) assuming you have managed to get Python to |
| 22 | compile under your development environment, but the step-by-step |
| 23 | character of this document will be lost. <p> |
| 24 | |
| 25 | Next, you need a <A HREF="http://www.python.org/python/Sources.html">python |
| 26 | source distribution</A>. There is a <A |
| 27 | HREF="update-to-1.3/into-PlugIns.hqx"> fixed project template</A> that |
| 28 | you also need if you are going to make a dynamically loaded |
| 29 | module. For PowerPC development you can actually get by without a full |
| 30 | source distribution, using the PPC Development distribution (if I have |
| 31 | gotten around to putting it together by the time you read |
| 32 | this). You'll also need a functional python interpreter, and the |
| 33 | Modulator program (which lives in <CODE>Tools:Modulator</CODE> in the |
| 34 | standard source distribution). You may also find that Guido's <A |
| 35 | HREF="http://www.python.org/doc/ext/ext.html">Extending and embedding |
| 36 | the Python interpreter</A> is a very handy piece of documentation. I |
| 37 | will skip lots of details that are handled there, like complete |
| 38 | descriptions of <CODE>Py_ParseTuple</CODE> and such utility routines, |
| 39 | or the general structure of extension modules. <p> |
| 40 | |
| 41 | <H2>InterSLIP and the C API to it</H2> |
| 42 | |
| 43 | InterSLIP, the utility to which we are going to create a python |
| 44 | interface, is a system extension that does all the work of connecting |
| 45 | to the internet over a modem connection. InterSLIP is provided |
| 46 | free-of-charge by <A |
| 47 | HREF="http://www.intercon.com/">InterCon</A>. First it connects to |
| 48 | your modem, then it goes through the whole process of dialling, |
| 49 | logging in and possibly starting the SLIP software on the remote |
| 50 | computer and finally it starts with the real work: packing up IP |
| 51 | packets handed to it by MacTCP and sending them to the remote side |
| 52 | (and, of course, the reverse action of receiving incoming packets, |
| 53 | unpacking them and handing them to MacTCP). InterSLIP is a device |
| 54 | driver, and you control it using a application supplied with it, |
| 55 | InterSLIP Setup. The API that InterSLIP Setup uses to talk to the |
| 56 | device driver is published in the documentation and, hence, also |
| 57 | useable by other applications. <p> |
| 58 | |
| 59 | I happened to have a C interface to the API, which is all ugly |
| 60 | low-level device-driver calls by itself. The C interface is in <A |
| 61 | HREF="interslip/InterslipLib.c">InterslipLib.c</A> and <A |
| 62 | HREF="interslip/InterslipLib.h">InterslipLib.h</A>, we'll |
| 63 | concentrate here on how to build the Python wrapper module around |
| 64 | it. Note that this is the "normal" situation when you are writing a |
| 65 | Python extension module: you have some sort of functionality available |
| 66 | to C programmers and want to make a Python interface to it. <p> |
| 67 | |
| 68 | <H2>Using Modulator</H2> |
| 69 | |
| 70 | The method we describe in this document, using Modulator, is the best |
| 71 | method for small interfaces. For large interfaces there is another |
| 72 | tool, Bgen, which actually generates the complete module without you |
| 73 | lifting a single finger. Bgen, however, has the disadvantage of having |
| 74 | a very steep learning curve, so an example using it will have to wait |
| 75 | until another document, when I have more time. <p> |
| 76 | |
| 77 | First, let us look at the <A |
| 78 | HREF="interslip/InterslipLib.h">InterslipLib.h</A> header file, |
| 79 | and see that the whole interface consists of six routines: |
| 80 | <CODE>is_open</CODE>, <CODE>is_connect</CODE>, |
| 81 | <CODE>is_disconnect</CODE>, <CODE>is_status</CODE>, |
| 82 | <CODE>is_getconfig</CODE> and <CODE>is_setconfig</CODE>. Our first |
| 83 | step will be to create a skeleton file <A |
| 84 | HREF="interslip/@interslipmodule.c">@interslipmodule.c</A>, a |
| 85 | dummy module that will contain all the glue code that python expects |
| 86 | of an extension module. Creating this glue code is a breeze with |
| 87 | modulator, a tool that we only have to tell that we want to create a |
| 88 | module with methods of the six names above and that will create the |
| 89 | complete skeleton C code for us. <p> |
| 90 | |
| 91 | Why call this dummy module <CODE>@interslipmodule.c</CODE> and not |
| 92 | <CODE>interslipmodule.c</CODE>? Self-preservation: if ever you happen |
| 93 | to repeat the whole process after you have actually turned the |
| 94 | skeleton module into a real module you would overwrite your |
| 95 | hand-written code. By calling the dummy module a different name you |
| 96 | have to make <EM>two</EM> mistakes in a row before you do this. <p> |
| 97 | |
| 98 | On systems with the Tk windowing API for Python (currently only |
| 99 | unix/X11 systems, but mac support may be available when you read this) |
| 100 | this is extremely simple. It is actually so simple that it pays to |
| 101 | create the skeleton module under unix and ship the code to your |
| 102 | mac. You start modulator and are provided with a form in which you |
| 103 | fill out the details of the module you are creating. <p> |
| 104 | |
| 105 | <IMG SRC="html.icons/modulator.gif" ALIGN=CENTER><p> |
| 106 | |
| 107 | You'll need to supply a module name (<CODE>interslip</CODE>, in our |
| 108 | case), a module abbreviation (<CODE>pyis</CODE>, which is used as a |
| 109 | prefix to all the routines and data structures modulator will create |
| 110 | for you) and you enter the names of all the methods your module will |
| 111 | export (the list above, with <CODE>is_</CODE> stripped off). Note that |
| 112 | we use <CODE>pyis</CODE> as the prefix instead of the more logical |
| 113 | <CODE>is</CODE>, since the latter would cause our routine names to |
| 114 | collide with those in the API we are interfacing to! The method names |
| 115 | are the names as seen by the python program, and the C routine names |
| 116 | will have the prefix and an underscore prepended. Modulator can do |
| 117 | much more, like generating code for objects and such, but that is a |
| 118 | topic for a later example. <p> |
| 119 | |
| 120 | Once you have told modulator all about the module you want to create |
| 121 | you press "check", which checks that you haven't omitted any |
| 122 | information and "Generate code". This will prompt you for a C output |
| 123 | file and generate your module for you. <p> |
| 124 | |
| 125 | <H2>Using Modulator without Tk</H2> |
| 126 | |
| 127 | |
| 128 | Modulator actually uses a two-stage process to create your code: first |
| 129 | the information you provided is turned into a number of python |
| 130 | statements and then these statements are executed to generate your |
| 131 | code. This is done so that you can even use modulator if you don't |
| 132 | have Tk support in Python: you'll just have to write the modulator |
| 133 | python statements by hand (about 10 lines, in our example) and |
| 134 | modulator will generate the C code (about 150 lines, in our |
| 135 | example). Here is the Python code you'll want to execute to generate |
| 136 | our skeleton module: <p> |
| 137 | |
| 138 | <CODE><PRE> |
| 139 | import addpack |
| 140 | addpack.addpack('Tools') |
| 141 | addpack.addpack('modulator') |
| 142 | import genmodule |
| 143 | |
| 144 | m = genmodule.module() |
| 145 | m.name = 'interslip' |
| 146 | m.abbrev = 'pyis' |
| 147 | m.methodlist = ['open', 'connect', 'disconnect', 'status', \ |
| 148 | 'getconfig', 'setconfig'] |
| 149 | m.objects = [] |
| 150 | |
| 151 | fp = open('@interslipmodule.c', 'w') |
| 152 | genmodule.write(fp, m) |
| 153 | </PRE></CODE> |
| 154 | |
| 155 | Drop this program on the python interpreter and out will come your |
| 156 | skeleton module. <p> |
| 157 | |
| 158 | Now, rename the file to interslipmodule.c and you're all set to start |
| 159 | developing. The module is complete in the sense that it should |
| 160 | compile, and that if you import it in a python program you will see |
| 161 | all the methods. It is, of course, not yet complete in a functional |
| 162 | way... <p> |
| 163 | |
| 164 | <H2>Adding a module to 68K Python</H2> |
| 165 | |
| 166 | What you do now depends on whether you're developing for PowerPC (or |
| 167 | for CFM68K) or for "traditional" mac. For a traditional 68K Python, |
| 168 | you will have to add your new module to the project file of the Python |
| 169 | interpreter, and you have to edit "config.c" to add the module to the |
| 170 | set of builtin modules. In config.c you will add the module at two |
| 171 | places: near the start of the file there is a list of external |
| 172 | declarations for all init() routines. Add a line of the form |
| 173 | <CODE><PRE> |
| 174 | extern void initinterslip(); |
| 175 | </PRE></CODE> |
| 176 | here. Further down the file there is an array that is initialized with |
| 177 | modulename/initfunction pairs. Add a line of the form |
| 178 | <CODE><PRE> |
| 179 | {"interslip", initinterslip}, |
| 180 | </PRE></CODE> |
| 181 | here. You may want to bracket these two lines with |
| 182 | <CODE><PRE> |
| 183 | #ifdef USE_INTERSLIP |
| 184 | #endif |
| 185 | </PRE></CODE> |
| 186 | lines, that way you can easily control whether the module is |
| 187 | incorporated into python at compile time. If you decide to do the |
| 188 | latter edit your config file (you can find the name in the "C/C++ |
| 189 | language" section of the MW preferences dialog, it will probably be |
| 190 | "mwerks_nonshared_config.h") and add a |
| 191 | <CODE><PRE> |
| 192 | #define USE_INTERSLIP |
| 193 | </PRE></CODE> |
| 194 | |
| 195 | Make the new interpreter and check that you can import the module, see |
| 196 | the methods (with "dir(interslip)") and call them. <p> |
| 197 | |
| 198 | <H2>Creating a PowerPC plugin module</H2> |
| 199 | |
| 200 | For PowerPC development you could follow the same path, but it is |
| 201 | actually a better idea to use a dynamically loadable module. The |
| 202 | advantage of dynamically loadable modules is that they are not loaded |
| 203 | until a python program actually uses them (resulting in less memory |
| 204 | usage by the interpreter) and that development is a lot simpler (since |
| 205 | your projects will all be smaller). Moreover, you can distribute a |
| 206 | plugin module by itself without haveing to distribute a complete |
| 207 | python interpreter. <p> |
| 208 | |
| 209 | Go to the "PlugIns" folder and copy the files xxmodule.µ, |
| 210 | xxmodule_config.h and xxmodule.µ.exp to interslipmodule.µ, |
| 211 | interslipmodule_config.h and interslipmodule.µ.exp, respectively. Edit |
| 212 | interslipmodule.µ.exp and change the name of the exported routine |
| 213 | "initxx" to "initinterslip". Open interslipmodule.µ with CodeWarrior, |
| 214 | remove the file xxmodule.c and add interslipmodule.c and make a number |
| 215 | of adjustments to the preferences: |
| 216 | <UL> |
| 217 | <LI> in C/C++ language, set the header file to interslipmodule_config.h |
| 218 | <LI> in PPC linker, set the entry point to "initinterslip" |
| 219 | <LI> in PPC PEF, set the fragment name to "interslipmodule" |
| 220 | <LI> in PPC Project, set the output file name to "interslipmodule.slb". |
| 221 | </UL> |
| 222 | Next, compile and link your module, fire up python and do the same |
| 223 | tests as for 68K python. <p> |
| 224 | |
| 225 | <H2>Getting the module to do real work</H2> |
| 226 | |
| 227 | So far, so good. In half an hour or so we have created a complete new |
| 228 | extension module for Python. The downside, however, is that the module |
| 229 | does not do anything useful. So, in the next half hour we will turn |
| 230 | our beautiful skeleton module into something that is at least as |
| 231 | beautiful but also gets some serious work done. For this once, |
| 232 | <EM>I</EM> have spent that half hour for you, and you can see the |
| 233 | results in <A |
| 234 | HREF="interslip/interslipmodule.c">interslipmodule.c</A>. <p> |
| 235 | |
| 236 | We add |
| 237 | <CODE><PRE> |
| 238 | #include "InterslipLib.h" |
| 239 | #include "macglue.h" |
| 240 | </PRE></CODE> |
| 241 | to the top of the file, and work our way through each of the methods |
| 242 | to add the functionality needed. Starting with open, we fill in the |
| 243 | template docstring, the value accessible from Python by looking at |
| 244 | <CODE>interslip.open.__doc__</CODE>. There are not many tools using |
| 245 | this information at the moment, but as soon as class browsers for |
| 246 | python become available having this minimal documentation available is |
| 247 | a good idea. We put "Load the interslip driver" as the comment |
| 248 | here. <p> |
| 249 | |
| 250 | Next, we tackle the body of <CODE>pyis_open()</CODE>. Since it has no |
| 251 | arguments and no return value we don't need to mess with that, we just |
| 252 | have to add a call to <CODE>is_open()</CODE> and check the return for |
| 253 | an error code, in which case we raise an error: |
| 254 | <CODE><PRE> |
| 255 | err = is_open(); |
| 256 | if ( err ) { |
| 257 | PyErr_Mac(ErrorObject, err); |
| 258 | return NULL; |
| 259 | } |
| 260 | </PRE></CODE> |
| 261 | The routine <CODE><A NAME="PyErr_Mac">PyErr_Mac()</A></CODE> is a |
| 262 | useful routine that raises the exception passed as its first |
| 263 | argument. The data passed with the exception is based on the standard |
| 264 | MacOS error code given, and PyErr_Mac() attempts to locate a textual |
| 265 | description of the error code (which sure beats the "error -14021" |
| 266 | messages that so many macintosh applications tell their poor |
| 267 | users). <p> |
| 268 | |
| 269 | We will skip pyis_connect and pyis_disconnect here, which are pretty |
| 270 | much identical to pyis_open: no arguments, no return value, just a |
| 271 | call and an error check. With pyis_status() things get interesting |
| 272 | again: this call still takes 3 arguments, and all happen to be values |
| 273 | returned (a numeric connection status indicator, a message sequence |
| 274 | number and a pointer to the message itself, in MacOS pascal-style |
| 275 | string form). We declare variables to receive the returned values, do |
| 276 | the call, check the error and format the return value. <p> |
| 277 | |
| 278 | Building the return value is done using <CODE><A |
| 279 | NAME="Py_BuildValue">Py_BuildValue</A></CODE>: |
| 280 | <CODE><PRE> |
| 281 | return Py_BuildValue("iiO&", (int)status, (int)seqnum, PyMac_BuildStr255, message); |
| 282 | </PRE></CODE> |
| 283 | Py_BuildValue() is a very handy routine that builds tuples according |
| 284 | to a format string, somewhat similar to the way <CODE>printf()</CODE> |
| 285 | works. The format string specifies the arguments expected after the |
| 286 | string, and turns them from C objects into python objects. The |
| 287 | resulting objects are put in a python tuple object and returned. The |
| 288 | "i" format specifier signifies an "int" (hence the cast: status and |
| 289 | seqnum are declared as "long", which is what the is_status() routine |
| 290 | wants, and even though we use a 4-byte project there is really no |
| 291 | reason not to put the cast here). Py_BuildValue and its counterpart |
| 292 | Py_ParseTuple have format codes for all the common C types like ints, |
| 293 | shorts, C-strings, floats, etc. Also, there is a nifty escape |
| 294 | mechanism to format values about which is does not know. This is |
| 295 | invoked by the "O&" format: it expects two arguments, a routine |
| 296 | pointer and an int-sized data object. The routine is called with the |
| 297 | object as a parameter and it should return a python objects |
| 298 | representing the data. <CODE>Macglue.h</CODE> declares a number of |
| 299 | such formatting routines for common MacOS objects like Str255, FSSpec, |
| 300 | OSType, Rect, etc. See the comments in the include file for |
| 301 | details. <p> |
| 302 | |
| 303 | <CODE>Pyis_getconfig()</CODE> is again similar to pyis_getstatus, only |
| 304 | two minor points are worth noting here. First, the C API return the |
| 305 | input and output baudrate squashed together into a single 4-byte |
| 306 | long. We separate them out before returning the result to |
| 307 | python. Second, whereas the status call returned us a pointer to a |
| 308 | <CODE>Str255</CODE> it kept we are responsible for allocating the |
| 309 | <CODE>Str255</CODE> for getconfig. This is something that would have |
| 310 | been easy to get wrong had we not used prototypes everywhere. Morale: |
| 311 | always try to include the header files for interfaces to libraries and |
| 312 | other stuff, so that the compiler can catch any mistakes you make. <p> |
| 313 | |
| 314 | <CODE>Pyis_setconfig()</CODE> finally shows off |
| 315 | <CODE>Py_ParseTuple</CODE>, the companion function to |
| 316 | <CODE>Py_BuildValue</CODE>. You pass it the argument tuple "args" |
| 317 | that your method gets as its second argument, a format string and |
| 318 | pointers to where you want the arguments stored. Again, standard C |
| 319 | types such as strings and integers Py_ParseTuple knows all about and |
| 320 | through the "O&" format you can extend the functionality. For each |
| 321 | "O&" you pass a function pointer and a pointer to a data area. The |
| 322 | function will be called with a PyObject pointer and your data pointer |
| 323 | and it should convert the python object to the correct C type. It |
| 324 | should return 1 on success and 0 on failure. Again, a number of |
| 325 | converters for standard MacOS types are provided, and declared in |
| 326 | <CODE>macglue.h</CODE>. <p> |
| 327 | |
| 328 | Next in our source file comes the method table for our module, which |
| 329 | has been generated by modulator (and it did a good job too!), but |
| 330 | which is worth looking at for a moment. Entries are of the form |
| 331 | <CODE><PRE> |
| 332 | {"open", pyis_open, 1, pyis_open__doc__}, |
| 333 | </PRE></CODE> |
| 334 | where the entries are python method name, C routine pointer, flags and |
| 335 | docstring pointer. The value to note is the 1 for the flags: this |
| 336 | signifies that you want to use "new-style" Py_ParseTuple behaviour. If |
| 337 | you are writing a new module always use this, but if you are modifying |
| 338 | old code which calls something like <CODE>getargs(args, "(ii)", |
| 339 | ...)</CODE> you will have to put zero here. See "extending and |
| 340 | embedding" or possibly the getargs.c source file for details if you |
| 341 | need them. <p> |
| 342 | |
| 343 | Finally, we add some code to the init module, to put some symbolic |
| 344 | constants (codes that can by returned by the status method) in the |
| 345 | module dictionary, so the python program can use "interslip.RUN" |
| 346 | instead of the cryptic "4" when it wants to check that the interslip |
| 347 | driver is in RUN state. Modulator has already generated code to get at |
| 348 | the module dictionary using PyModule_GetDict() to store the exception |
| 349 | object, so we simply call |
| 350 | <CODE><PRE> |
| 351 | PyDict_SetItemString(d, "IDLE", PyInt_FromLong(IS_IDLE)); |
| 352 | </PRE></CODE> |
| 353 | for each of our items. Since the last bit of code in our init routine |
| 354 | checks for previous errors with <CODE>PyErr_Occurred()</CODE> and |
| 355 | since <CODE>PyDict_SetItemString()</CODE> gracefully handles the case |
| 356 | of <CODE>NULL</CODE> parameters (if <CODE>PyInt_FromLong()</CODE> |
| 357 | failed, for instance) we don't have to do error checking here. In some |
| 358 | other cases you may have to do error checking yourself. <p> |
| 359 | |
| 360 | This concludes our crash-course on writing Python extensions in C on |
| 361 | the Macintosh. If you are not done reading yet I suggest you look |
| 362 | back at the <A HREF="index.html">MacPython Crashcourse index</A> to |
| 363 | find another topic to study. <p> |