| Jack Jansen | 968cde9 | 2000-04-22 21:48:56 +0000 | [diff] [blame] | 1 | <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN"> |
| 2 | <HTML> |
| 3 | |
| 4 | <HEAD> |
| 5 | |
| 6 | <TITLE>HOWTO: Compiling Python Modules with MPW</TITLE> |
| 7 | |
| 8 | </HEAD> |
| 9 | |
| 10 | <BODY> |
| 11 | |
| 12 | <H1>HOWTO: Compiling Python Modules with MPW</H1> |
| 13 | |
| 14 | <blockquote> |
| 15 | This HOWTO is a slightly reformatted version of an original by |
| 16 | <A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, whose |
| 17 | <A HREF="http://www.nevada.edu/~cwebster/Python/">Python page</A> |
| 18 | may contain a more up-to-date version. |
| 19 | </blockquote> |
| 20 | <HR> |
| 21 | |
| 22 | <P> |
| 23 | The <A HREF="http://www.cwi.nl/~jack/macpython.html">Macintosh version</A> |
| 24 | of the <A HREF="http://www.python.org/">Python programming language</A> is |
| 25 | usually compiled with <A HREF="http://www.metrowerks.com/">Metrowerks |
| 26 | CodeWarrior</A>. As a result, C extension modules are also usually |
| 27 | compiled with CodeWarrior, and the documentation and sample code reflects |
| 28 | this. CodeWarrior is a commercial product, and may be beyond the budgets |
| 29 | of hobbyist hackers, making them dependent on others to compile C extension |
| 30 | modules. At the present time, many standard C extension modules compile |
| 31 | "out of the box" on the Macintosh, but in only a few cases is the plugin |
| 32 | for the Macintosh included in the distribution. |
| 33 | </P> |
| 34 | |
| 35 | <P> |
| 36 | The <A HREF="http://developer.apple.com/tools/mpw-tools/">Macintosh |
| 37 | Programmer's Workshop</A> (MPW) is Apple's development environment, and is |
| 38 | freely available for <A |
| 39 | HREF="ftp://ftp.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc./">download</A> |
| 40 | from Apple, as well as on their Developer CDs. Since Python was originally |
| 41 | developed using MPW, before CodeWarrior became the dominant MacOS |
| 42 | development environment, most of the idiosyncrasies of MPW are already |
| 43 | supported, and compilation of C extension modules in MPW is possible. |
| 44 | </P> |
| 45 | |
| 46 | <P> |
| 47 | This HOWTO only deals with compiling for PowerPC Macintoshes. The process |
| 48 | should be similar for 68k Macintoshes using the code fragment manager, but |
| 49 | I have not attempted this - my old Mac is running NetBSD. |
| 50 | </P> |
| 51 | |
| 52 | <P> |
| 53 | This way of compiling modules is still experimental. Please read the |
| 54 | caveats section below. |
| 55 | </P> |
| 56 | |
| 57 | <H2><A NAME="setup">Setting Up MPW for Compiling Python Modules</A></H2> |
| 58 | |
| 59 | <P> |
| 60 | This assumes that you have successfully installed both MPW and Python with |
| 61 | the Developer's Kit on your Macintosh. |
| 62 | </P> |
| 63 | |
| 64 | <P> |
| 65 | The first step is to let MPW know where you keep Python. This step is not |
| 66 | strictly necessary, but will make development easier and improve |
| 67 | portability. Create a new file in the <CODE>Startup Items</CODE> folder of |
| 68 | MPW called <A HREF="Python"><CODE>Python</CODE></A>. Type the lines: |
| 69 | </P> |
| 70 | |
| 71 | <PRE> |
| 72 | set Python "Macintosh HD:Applications:Python 1.5.2c1:" |
| 73 | set PythonIncludes "{Python}Include" |
| 74 | set PythonMacIncludes "{Python}Mac:Include" |
| 75 | set PythonCore "{Python}PythonCore" |
| 76 | |
| 77 | export Python PythonIncludes PythonMacIncludes PythonCore |
| 78 | </PRE> |
| 79 | |
| 80 | <P> |
| 81 | where <CODE>Macintosh HD:Applications:Python 1.5.2c1:</CODE> is replaced by |
| 82 | the path to the directory where you keep your copy of Python, and the other |
| 83 | variables reflect where you keep your header files and Python core files. |
| 84 | The locations here are the standard for Python 1.5.2c1, but they are |
| 85 | different for Python 1.52b2 and earlier (most notably, the PythonCore is |
| 86 | kept in the Extensions folder). |
| 87 | </P> |
| 88 | |
| 89 | <P> |
| 90 | Next, you need to update the <A HREF="config.h"><CODE>config.h</CODE></A> |
| 91 | file for the <CODE>MrC</CODE> compiler included with MPW. This header file |
| 92 | is located in the <CODE>:Mac:Include</CODE> folder in the standard |
| 93 | distribution. You can update it by hand, by adding the lines: |
| 94 | </P> |
| 95 | |
| 96 | <PRE> |
| 97 | #ifdef __MRC__ |
| 98 | #define BAD_STATIC_FORWARD |
| 99 | #endif |
| 100 | </PRE> |
| 101 | |
| 102 | <P> |
| 103 | at the after the similar defines for <CODE>__MWERKS__</CODE> and |
| 104 | <CODE>__SC__</CODE> in the file. This step is critical: many modules, |
| 105 | including ones in the standard distribution, will not compile properly |
| 106 | without this modification (see common problems below). |
| 107 | </P> |
| 108 | |
| 109 | <P> |
| 110 | Copies of both the <A HREF="Python"><CODE>Python</CODE></A> startup item |
| 111 | for MPW and the <A HREF="config.h"><CODE>config.h</CODE></A> are included |
| 112 | here for your convenience. |
| 113 | </P> |
| 114 | |
| 115 | <P> |
| 116 | If you are porting Unix modules to the mac, you may find it useful to |
| 117 | install <A |
| 118 | HREF="http://www.iis.ee.ethz.ch/~neeri/macintosh/gusi-qa.html">GUSI</A> for |
| 119 | your copy of MPW. GUSI provides some amount of POSIX compatibility, and is |
| 120 | used by Python itself for this purpose - at the very least having it's |
| 121 | header files available may be useful. Also of note for people porting Unix |
| 122 | modules, the most recent alpha version (4.1a8) of <CODE>MrC</CODE> and |
| 123 | <CODE>MrCpp</CODE> at this writing permits using unix-style pathnames for |
| 124 | includes via the <CODE>-includes unix</CODE> command line option. I have |
| 125 | not experimented heavily with this, but will be doing so in the future and |
| 126 | report my findings. |
| 127 | </P> |
| 128 | |
| 129 | <P> |
| 130 | You now have MPW and Python set up to allow compilation of modules. |
| 131 | </P> |
| 132 | |
| 133 | <H2><A NAME="compiling">Compiling a Module</A></H2> |
| 134 | |
| 135 | <P> |
| 136 | This assumes that you have a C extension module ready to compile. For |
| 137 | instructions on how to write a module, see the Python documentation. |
| 138 | </P> |
| 139 | |
| 140 | <P> |
| 141 | There are three approaches you can take to compiling in MPW: using the |
| 142 | command line interface, using the MPW <CODE>CreateMake</CODE> command |
| 143 | (available as the "Create build commands..." menu item, and writing a |
| 144 | Makefile by hand. |
| 145 | </P> |
| 146 | |
| 147 | <P> |
| 148 | Before you start any of these, you'll need to know: |
| 149 | </P> |
| 150 | |
| 151 | <UL> |
| 152 | <LI>The names and locations of the C source files. In the examples, this |
| 153 | is the file <A HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>, and is in |
| 154 | MPW's current working directory. |
| 155 | <LI>The name that Python expects to import your module under. In the |
| 156 | examples, this is <CODE>xx</CODE>, so the shared library file will be |
| 157 | called <CODE>xx.ppc.slb</CODE>. |
| 158 | <LI>The location of any additional header files use by the C source. The |
| 159 | example does not use any additional header files. |
| 160 | <LI>The location of any additional shared libraries which the module needs |
| 161 | to link to. The example does not link to any other shared libraries. |
| 162 | <LI>The name of the entry point to your module. This is usually the last |
| 163 | function in the main C source file, and the name usually starts with |
| 164 | <CODE>init</CODE>. In the examples, this is <CODE>initxx</CODE>. |
| 165 | </UL> |
| 166 | |
| 167 | <H3>Using the Command Line</H3> |
| 168 | |
| 169 | <P> |
| 170 | For simple modules consisting of one or two C files, it's often convenient |
| 171 | to simply use commands in a MPW Worksheet. Usually you will want to set |
| 172 | MPW's working directory to the directory containing the C source code. The |
| 173 | following commands compile and link the standard Python test module <A |
| 174 | HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>: |
| 175 | </P> |
| 176 | |
| 177 | <PRE> |
| 178 | MrC "xxmodule.c" -o "xx.c.x" -w off -d HAVE_CONFIG_H ∂ |
| 179 | -i "{PythonMacIncludes}" ∂ |
| 180 | -i "{PythonIncludes}" |
| 181 | PPCLink ∂ |
| 182 | -o "xx.ppc.slb" ∂ |
| 183 | "xx.c.x" ∂ |
| 184 | -t 'shlb' ∂ |
| 185 | -c 'Pyth' ∂ |
| 186 | -xm s ∂ |
| 187 | -d ∂ |
| 188 | "{PythonCore}" ∂ |
| 189 | "{SharedLibraries}InterfaceLib" ∂ |
| 190 | "{SharedLibraries}MathLib" ∂ |
| 191 | "{SharedLibraries}StdCLib" ∂ |
| 192 | "{PPCLibraries}StdCRuntime.o" ∂ |
| 193 | "{PPCLibraries}PPCCRuntime.o" ∂ |
| 194 | "{PPCLibraries}PPCToolLibs.o" ∂ |
| 195 | -export initxx |
| 196 | </PRE> |
| 197 | |
| 198 | <P> |
| 199 | (Note: The last character on each line should appear as "partial |
| 200 | derivative" symbol, which you type as <KBD>option-d</KBD> and which is |
| 201 | MPW's line continuation symbol.) |
| 202 | </P> |
| 203 | |
| 204 | <P> |
| 205 | Any additional header files should be specified by adding their directories |
| 206 | as extra <CODE>-i</CODE> options to the <CODE>MrC</CODE> command. Any |
| 207 | additional shared libraries should be added before the PythonCore library |
| 208 | in the <CODE>PPCLink</CODE> command. |
| 209 | </P> |
| 210 | |
| 211 | <P> |
| 212 | If there is more than one source file, you will need to duplicate the |
| 213 | compile command for each source file, and you will need to include all the |
| 214 | object files in the place where <CODE>"xx.c.x"</CODE> appears in the |
| 215 | <CODE>PPCLink</CODE> command. |
| 216 | </P> |
| 217 | |
| 218 | <H3>Using CreateMake</H3> |
| 219 | |
| 220 | <P> |
| 221 | For more complex modules, or modules that you are writing yourself, you |
| 222 | will probably want to use a makefile. Unfortunately MPW's makefiles are |
| 223 | incompatible with the standard Unix makefiles, so you will not be able to |
| 224 | use any makefiles which come with a C module. |
| 225 | </P> |
| 226 | |
| 227 | <P> |
| 228 | Usually, you will want the makefile to reside in the same directory as the |
| 229 | C source code, so you should set MPW's working directory to that directory |
| 230 | before proceeding. |
| 231 | </P> |
| 232 | |
| 233 | <P> |
| 234 | To create a makefile for the standard Python test module <A |
| 235 | HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>: |
| 236 | </P> |
| 237 | |
| 238 | <UL> |
| 239 | <LI>Select "Create build commands..." from the "Build" Menu. |
| 240 | <LI>Type <KBD>xx.ppc.slb</KBD> for the Program Name. |
| 241 | <LI>Select "Shared Library" for the Program Type. |
| 242 | <LI>Select "PowerPC Only" for the Target. |
| 243 | <LI>Click on the "Source Files..." button, and add your module's C source |
| 244 | files to the list. |
| 245 | <LI>Click on the "Other Options..." button and change the creator type to |
| 246 | "Pyth". If you are using additional header files, you can also add their |
| 247 | directories at this stage. Click on "Continue" once you have done this. |
| 248 | <LI>Click on the "Exported Symbols..." button and type <KBD>initxx</KBD> |
| 249 | into the entry field. Click on "Continue" once you have done this. |
| 250 | <LI>At this stage, your CreateMake window should look like this: <IMG |
| 251 | SRC="html.icons/createmake.png" ALT="[picture of commando window for CreateMake]"> |
| 252 | <LI>Click on the "CreateMake" button. |
| 253 | </UL> |
| 254 | |
| 255 | <P> |
| 256 | You will now need to edit the makefile that was just created. Open the |
| 257 | file "xx.ppc.slb.make" in the current directory and make the following |
| 258 | changes: |
| 259 | </P> |
| 260 | |
| 261 | <UL> |
| 262 | <LI>Change the line |
| 263 | |
| 264 | <PRE> |
| 265 | Includes = |
| 266 | </PRE> |
| 267 | |
| 268 | <P> |
| 269 | to read |
| 270 | </P> |
| 271 | |
| 272 | <PRE> |
| 273 | Includes = -i "{PythonIncludes}" -i "{PythonMacIncludes}" |
| 274 | </PRE> |
| 275 | |
| 276 | <P> |
| 277 | If you have any additional headers than need to be included, you can add |
| 278 | them here as well. |
| 279 | <LI>Change the line |
| 280 | |
| 281 | <PRE> |
| 282 | PPCCOptions = {Includes} {Sym•PPC} |
| 283 | </PRE> |
| 284 | |
| 285 | <P> |
| 286 | to read |
| 287 | </P> |
| 288 | |
| 289 | <PRE> |
| 290 | PPCCOptions = -w off -d HAVE_CONFIG_H {Includes} {Sym•PPC} |
| 291 | </PRE> |
| 292 | |
| 293 | <P> |
| 294 | <LI>After the line |
| 295 | |
| 296 | <PRE> |
| 297 | -xm s ∂ |
| 298 | </PRE> |
| 299 | |
| 300 | <P> |
| 301 | add |
| 302 | </P> |
| 303 | |
| 304 | <PRE> |
| 305 | -d ∂ |
| 306 | "{PythonCore}" ∂ |
| 307 | </PRE> |
| 308 | |
| 309 | <P> |
| 310 | If you have any other shared libraries you need to link to, add each on a |
| 311 | line before PythonCore, terminating each line with a <CODE>∂</CODE>. |
| 312 | </P> |
| 313 | |
| 314 | </UL> |
| 315 | |
| 316 | <P>Save the file. You are now ready to build. |
| 317 | </P> |
| 318 | |
| 319 | <P> |
| 320 | Go to the "Build" or "Full Build" menu items, type in |
| 321 | <KBD>xx.ppc.slb</KBD>, and MPW should take things from there. Any time you |
| 322 | need to rebuild the shared library, you can simply do another "Build" or |
| 323 | "Full Build". |
| 324 | </P> |
| 325 | |
| 326 | <H3>Writing a Makefile by Hand</H3> |
| 327 | |
| 328 | <P> |
| 329 | For modules which have complex interdependencies between files, you will |
| 330 | likely need a more sophisticated makefile than the one created by |
| 331 | <CODE>CreateMake</CODE>. You will need to be familiar with the MPW |
| 332 | makefile format, but you can get a start by either using |
| 333 | <CODE>CreateMake</CODE> to get a simple starting point, or taking another |
| 334 | MPW makefile as a starting point. |
| 335 | </P> |
| 336 | |
| 337 | <P> |
| 338 | It is beyond the scope of this HOWTO to go into the generalities of MPW |
| 339 | makefiles. Documentation on MPW's <CODE>Make</CODE> command can be found |
| 340 | with the MPW distribution, in particular the documents <A |
| 341 | HREF="http://developer.apple.com/tools/mpw-tools/books.html#Building">Building |
| 342 | and Maintaining Programs with MPW (2nd Edition)</A> and the <A |
| 343 | HREF="http://developer.apple.com/tools/mpw-tools/books.html#CommandRef">MPW |
| 344 | Command Reference</A>. |
| 345 | </P> |
| 346 | |
| 347 | <P> |
| 348 | There are a couple of important points to keep in mind when writing a |
| 349 | makefile by hand:</P> |
| 350 | |
| 351 | <UL> |
| 352 | <LI>When there are multiple symbols with the same name in object files or |
| 353 | shared libraries, <CODE>PPCLink</CODE> used the symbol from the file which |
| 354 | appears first in arguments of the <CODE>PPCLink</CODE> command. For this |
| 355 | reason, you will usually want the PythonCore and any other shared libraries |
| 356 | which are not part of the standard MPW runtime environment to appear before |
| 357 | the standard runtime libraries. This is particularly the case with |
| 358 | StdCLib. The "-d" option turns off the (often copious) warnings about |
| 359 | multiply defined symbols. |
| 360 | <LI>You will want to make sure that the <CODE>HAVE_CONFIG_H</CODE> |
| 361 | preprocessor symbol is defined for most C source files using the <CODE>-d |
| 362 | HAVE_CONFIG_H</CODE> option to <CODE>MrC</CODE>. |
| 363 | </UL> |
| 364 | |
| 365 | <P> |
| 366 | The file <A HREF="xx.ppc.slb.make.sit.hqx"><CODE>xx.ppc.slb.make</CODE></A> |
| 367 | is included here for you to use as a starting point. |
| 368 | </P> |
| 369 | |
| 370 | <H2><A NAME="using">Using the Extension Module</A></H2> |
| 371 | |
| 372 | <P> |
| 373 | Once you have compiled your extension module, you will need to let Python |
| 374 | know where it is. You can either move it into a place on Python's search |
| 375 | path - such as the <CODE>:Mac:Plugins</CODE> folder - or modify the path to |
| 376 | include the location of your new module using the |
| 377 | <CODE>EditPythonPrefs</CODE> applet. |
| 378 | </P> |
| 379 | |
| 380 | <P> |
| 381 | Your work may not be completely done, as many extension modules have a |
| 382 | Python wrapper around them. If the Python was not written with portability |
| 383 | in mind, you may need to do some more work to get that up and running. |
| 384 | Indeed, if the Python part uses OS-specific features, like pipes, you may |
| 385 | have to completely rewrite it if you can make it work at all. |
| 386 | </P> |
| 387 | |
| 388 | <H2><A NAME="problems">Common Problems</A></H2> |
| 389 | |
| 390 | <P> |
| 391 | There are a couple of common problems which occur when porting a module |
| 392 | from another platform. Fortunately, they are often easy to fix. |
| 393 | </P> |
| 394 | |
| 395 | <H3>Static Forward Definitions</H3> |
| 396 | |
| 397 | <P> |
| 398 | If you get a compiler error which looks something like: |
| 399 | </P> |
| 400 | |
| 401 | <PRE> |
| 402 | File "xxmodule.c"; line 135 #Error: 'Xxo_Type' is already defined |
| 403 | </PRE> |
| 404 | |
| 405 | <P> |
| 406 | then most likely either you have not set up <CODE>config.h</CODE> correctly |
| 407 | to handle static forward definitions, or the module author has not adhered |
| 408 | to the standard python conventions. If the second is the case, find where |
| 409 | the variable is first defined, and replace the <CODE>static</CODE> with |
| 410 | <CODE>staticforward</CODE>. Then find the second place it is defined |
| 411 | (usually the line where the compiler complained) and replace |
| 412 | <CODE>static</CODE> with <CODE>statichere</CODE>. |
| 413 | </P> |
| 414 | |
| 415 | <P> |
| 416 | If you have set up things correctly, you should now be able to compile. |
| 417 | </P> |
| 418 | |
| 419 | <H3>Automatic Type Conversion</H3> |
| 420 | |
| 421 | <P> |
| 422 | <CODE>MrC</CODE> seems to be a little pickier about automatically |
| 423 | converting from one type to another than some other C compilers. These can |
| 424 | often be fixed by simply adding an explicit cast to the desired type. |
| 425 | </P> |
| 426 | |
| 427 | <P> |
| 428 | XXX There may be a compiler option which relaxes this. That would be a |
| 429 | better solution. |
| 430 | </P> |
| 431 | |
| 432 | <H2><A NAME="caveats">Caveats</A></H2> |
| 433 | |
| 434 | <P> |
| 435 | As Jack Jansen pointed out on the Mac Python mailing list, there could |
| 436 | potentially be conflicts between the MetroWerks C runtime which the Python |
| 437 | core and standard modules was compiled with, and the MPW C runtime which |
| 438 | your extension module is compiled with. While things seem to work fine in |
| 439 | everyday use, it is possible that there are bugs which have not been |
| 440 | discovered yet. Most likely these world take the form of standard C |
| 441 | functions (most likely I/O functions due to conflicts between the SIOUX |
| 442 | libraries and the SIOW libraries) not working as they are supposed to, or |
| 443 | memory leaks caused by improper malloc/free. |
| 444 | </P> |
| 445 | |
| 446 | <P> |
| 447 | Some such problems have been demonstrated by compiling modules with |
| 448 | PythonCore linked after StdCLib - printf does not work properly in this |
| 449 | setup, and I suspect that there will also be malloc/free problems in |
| 450 | situations where the module allocates memory which is later disposed of by |
| 451 | Python, or vice-versa. Compiling with PythonCore taking precedence over |
| 452 | StdCLib seems to give the correct behaviour. |
| 453 | </P> |
| 454 | |
| 455 | <P> |
| 456 | This method of compiling should be considered experimental for the time |
| 457 | being. <STRONG>Use it at your own risk.</STRONG> |
| 458 | </P> |
| 459 | |
| 460 | <P> |
| 461 | If you notice any quirks in modules compiled this way, or have insight into |
| 462 | what may go wrong or right with this situation, <A |
| 463 | HREF="mailto:cwebster@nevada.edu">please contact me</A> so that I can add |
| 464 | it to the HOWTO. |
| 465 | </P> |
| 466 | |
| 467 | <P> |
| 468 | The ideal solution to this problem would be to get Python to compile using |
| 469 | MPW (and a Python MPW Tool would be very neat indeed). However, that does |
| 470 | seem to be a major project. |
| 471 | </P> |
| 472 | |
| 473 | <DIV class=footer> |
| 474 | <HR> |
| 475 | <BR> |
| 476 | ©<A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, 1999. <BR> |
| 477 | <!-- #LASTMODIFIED TEXT="Last modified" FORM="SHORT,TIME" --> |
| 478 | Last modified 14/12/99 12:17 PM |
| 479 | <!-- /#LASTMODIFIED --> |
| 480 | </DIV> |
| 481 | |
| 482 | </BODY> |
| 483 | |
| 484 | </HTML> |