<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>

<HEAD>

<TITLE>HOWTO: Compiling Python Modules with MPW</TITLE>

</HEAD>

<BODY>

<H1>HOWTO: Compiling Python Modules with MPW</H1>

<blockquote>
This HOWTO is a slightly reformatted version of an original by
<A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, whose
<A  HREF="http://www.nevada.edu/~cwebster/Python/">Python page</A>
may contain a more up-to-date version.
</blockquote>
<HR>

<P>
The <A HREF="http://www.cwi.nl/~jack/macpython.html">Macintosh version</A> 
of the <A HREF="http://www.python.org/">Python programming language</A> is 
usually compiled with <A HREF="http://www.metrowerks.com/">Metrowerks 
CodeWarrior</A>.  As a result, C extension modules are also usually 
compiled with CodeWarrior, and the documentation and sample code reflects 
this.  CodeWarrior is a commercial product, and may be beyond the budgets 
of hobbyist hackers, making them dependent on others to compile C extension 
modules.  At the present time, many standard C extension modules compile 
"out of the box" on the Macintosh, but in only a few cases is the plugin 
for the Macintosh included in the distribution.
</P>

<P>
The <A HREF="http://developer.apple.com/tools/mpw-tools/">Macintosh 
Programmer's Workshop</A> (MPW) is Apple's development environment, and is 
freely available for <A 
HREF="ftp://ftp.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc./">download</A> 
from Apple, as well as on their Developer CDs.  Since Python was originally 
developed using MPW, before CodeWarrior became the dominant MacOS 
development environment, most of the idiosyncrasies of MPW are already 
supported, and compilation of C extension modules in MPW is possible.
</P>

<P>
This HOWTO only deals with compiling for PowerPC Macintoshes.  The process 
should be similar for 68k Macintoshes using the code fragment manager, but 
I have not attempted this - my old Mac is running NetBSD.
</P>

<P>
This way of compiling modules is still experimental.  Please read the 
caveats section below.
</P>

<H2><A NAME="setup">Setting Up MPW for Compiling Python Modules</A></H2>

<P>
This assumes that you have successfully installed both MPW and Python with 
the Developer's Kit on your Macintosh.
</P>

<P>
The first step is to let MPW know where you keep Python.  This step is not 
strictly necessary, but will make development easier and improve 
portability.  Create a new file in the <CODE>Startup Items</CODE> folder of 
MPW called <A HREF="Python"><CODE>Python</CODE></A>.  Type the lines:
</P>

<PRE>
set Python "Macintosh HD:Applications:Python 1.5.2c1:"
set PythonIncludes "{Python}Include"
set PythonMacIncludes "{Python}Mac:Include"
set PythonCore "{Python}PythonCore"

export Python PythonIncludes PythonMacIncludes PythonCore
</PRE>

<P>
where <CODE>Macintosh HD:Applications:Python 1.5.2c1:</CODE> is replaced by 
the path to the directory where you keep your copy of Python, and the other 
variables reflect where you keep your header files and Python core files. 
The locations here are the standard for Python 1.5.2c1, but they are 
different for Python 1.52b2 and earlier (most notably, the PythonCore is 
kept in the Extensions folder).
</P>

<P>
Next, you need to update the <A HREF="config.h"><CODE>config.h</CODE></A> 
file for the <CODE>MrC</CODE> compiler included with MPW.  This header file 
is located in the <CODE>:Mac:Include</CODE> folder in the standard 
distribution.  You can update it by hand, by adding the lines:
</P>

<PRE>
#ifdef __MRC__
#define BAD_STATIC_FORWARD
#endif
</PRE>

<P>
at the after the similar defines for <CODE>__MWERKS__</CODE> and 
<CODE>__SC__</CODE> in the file. This step is critical: many modules, 
including ones in the standard distribution, will not compile properly 
without this modification (see common problems below).
</P>

<P>
Copies of both the <A HREF="Python"><CODE>Python</CODE></A> startup item 
for MPW and the <A HREF="config.h"><CODE>config.h</CODE></A> are included 
here for your convenience.
</P>

<P>
If you are porting Unix modules to the mac, you may find it useful to 
install <A 
HREF="http://www.iis.ee.ethz.ch/~neeri/macintosh/gusi-qa.html">GUSI</A> for 
your copy of MPW.  GUSI provides some amount of POSIX compatibility, and is 
used by Python itself for this purpose - at the very least having it's 
header files available may be useful.  Also of note for people porting Unix 
modules, the most recent alpha version (4.1a8) of <CODE>MrC</CODE> and 
<CODE>MrCpp</CODE> at this writing permits using unix-style pathnames for 
includes via the <CODE>-includes unix</CODE> command line option.  I have 
not experimented heavily with this, but will be doing so in the future and 
report my findings.
</P>

<P>
You now have MPW and Python set up to allow compilation of modules.
</P>

<H2><A NAME="compiling">Compiling a Module</A></H2>

<P>
This assumes that you have a C extension module ready to compile.  For 
instructions on how to write a module, see the Python documentation.
</P>

<P>
There are three approaches you can take to compiling in MPW: using the 
command line interface, using the MPW <CODE>CreateMake</CODE> command 
(available as the "Create build commands..." menu item, and writing a 
Makefile by hand.
</P>

<P>
Before you start any of these, you'll need to know:
</P>

<UL>
    <LI>The names and locations of the C source files.  In the examples, this 
    is the file <A HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>, and is in 
    MPW's current working directory.
    <LI>The name that Python expects to import your module under.  In the 
    examples, this is <CODE>xx</CODE>, so the shared library file will be 
    called <CODE>xx.ppc.slb</CODE>.
    <LI>The location of any additional header files use by the C source. The 
    example does not use any additional header files.
    <LI>The location of any additional shared libraries which the module needs 
    to link to.  The example does not link to any other shared libraries.
    <LI>The name of the entry point to your module.  This is usually the last 
    function in the main C source file, and the name usually starts with 
    <CODE>init</CODE>.  In the examples, this is <CODE>initxx</CODE>.
</UL>

<H3>Using the Command Line</H3>

<P>
For simple modules consisting of one or two C files, it's often convenient 
to simply use commands in a MPW Worksheet.  Usually you will want to set 
MPW's working directory to the directory containing the C source code.  The 
following commands compile and link the standard Python test module <A 
HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>:
</P>

<PRE>
MrC "xxmodule.c" -o "xx.c.x" -w off  -d HAVE_CONFIG_H &#8706;
        -i "{PythonMacIncludes}" &#8706;
        -i "{PythonIncludes}" 
PPCLink &#8706;
        -o "xx.ppc.slb"  &#8706;
        "xx.c.x" &#8706;
        -t 'shlb' &#8706;
        -c 'Pyth' &#8706;
        -xm s &#8706;
        -d &#8706;
        "{PythonCore}" &#8706;
        "{SharedLibraries}InterfaceLib" &#8706;
        "{SharedLibraries}MathLib" &#8706;
        "{SharedLibraries}StdCLib" &#8706;
        "{PPCLibraries}StdCRuntime.o" &#8706;
        "{PPCLibraries}PPCCRuntime.o" &#8706;
        "{PPCLibraries}PPCToolLibs.o" &#8706;
        -export initxx
</PRE>

<P>
(Note: The last character on each line should appear as "partial 
derivative" symbol, which you type as <KBD>option-d</KBD> and which is 
MPW's line continuation symbol.)
</P>

<P>
Any additional header files should be specified by adding their directories 
as extra <CODE>-i</CODE> options to the <CODE>MrC</CODE> command.  Any 
additional shared libraries should be added before the PythonCore library 
in the <CODE>PPCLink</CODE> command.
</P>

<P>
If there is more than one source file, you will need to duplicate the 
compile command for each source file, and you will need to include all the 
object files in the place where <CODE>"xx.c.x"</CODE> appears in the 
<CODE>PPCLink</CODE> command.
</P>

<H3>Using CreateMake</H3>

<P>
For more complex modules, or modules that you are writing yourself, you 
will probably want to use a makefile. Unfortunately MPW's makefiles are 
incompatible with the standard Unix makefiles, so you will not be able to 
use any makefiles which come with a C module.
</P>

<P>
Usually, you will want the makefile to reside in the same directory as the 
C source code, so you should set MPW's working directory to that directory 
before proceeding.
</P>

<P>
To create a makefile for the standard Python test module <A 
HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>:
</P>

<UL>
    <LI>Select "Create build commands..." from the "Build" Menu.
    <LI>Type <KBD>xx.ppc.slb</KBD> for the Program Name.
    <LI>Select "Shared Library" for the Program Type.
    <LI>Select "PowerPC Only" for the Target.
    <LI>Click on the "Source Files..." button, and add your module's C source 
    files to the list.
    <LI>Click on the "Other Options..." button and change the creator type to 
    "Pyth".  If you are using additional header files, you can also add their 
    directories at this stage.  Click on "Continue" once you have done this.
    <LI>Click on the "Exported Symbols..." button and type <KBD>initxx</KBD> 
    into the entry field.  Click on "Continue" once you have done this.
    <LI>At this stage, your CreateMake window should look like this: <IMG 
    SRC="html.icons/createmake.png" ALT="[picture of commando window for CreateMake]">
    <LI>Click on the "CreateMake" button.
</UL>

<P>
You will now need to edit the makefile that was just created.  Open the 
file "xx.ppc.slb.make" in the current directory and make the following 
changes:
</P>

<UL>
    <LI>Change the line
    
    <PRE>
Includes     =
</PRE>
    
    <P>
    to read
    </P>
    
    <PRE>
Includes     = -i "{PythonIncludes}" -i "{PythonMacIncludes}"
</PRE>
    
    <P>
    If you have any additional headers than need to be included, you can add 
    them here as well.
    <LI>Change the line
    
    <PRE>
PPCCOptions  = {Includes} {Sym&#8226;PPC} 
</PRE>
    
    <P>
    to read
    </P>
    
    <PRE>
PPCCOptions  = -w off  -d HAVE_CONFIG_H {Includes} {Sym&#8226;PPC} 
</PRE>
    
    <P>
    <LI>After the line
    
    <PRE>
                -xm s &#8706;
</PRE>
    
    <P>
    add
    </P>
    
    <PRE>
                -d &#8706;
                "{PythonCore}" &#8706;
</PRE>
    
    <P>
    If you have any other shared libraries you need to link to, add each on a 
    line before PythonCore, terminating each line with a <CODE>&#8706;</CODE>.
    </P>
    
</UL>

<P>Save the file.  You are now ready to build.
</P>

<P>
Go to the "Build" or "Full Build" menu items, type in 
<KBD>xx.ppc.slb</KBD>, and MPW should take things from there.  Any time you 
need to rebuild the shared library, you can simply do another "Build" or 
"Full Build".
</P>

<H3>Writing a Makefile by Hand</H3>

<P>
For modules which have complex interdependencies between files, you will 
likely need a more sophisticated makefile than the one created by 
<CODE>CreateMake</CODE>.  You will need to be familiar with the MPW 
makefile format, but you can get a start by either using 
<CODE>CreateMake</CODE> to get a simple starting point, or taking another 
MPW makefile as a starting point.
</P>

<P>
It is beyond the scope of this HOWTO to go into the generalities of MPW 
makefiles. Documentation on MPW's <CODE>Make</CODE> command can be found 
with the MPW distribution, in particular the documents <A 
HREF="http://developer.apple.com/tools/mpw-tools/books.html#Building">Building 
and Maintaining Programs with MPW (2nd Edition)</A> and the <A 
HREF="http://developer.apple.com/tools/mpw-tools/books.html#CommandRef">MPW 
Command Reference</A>.
</P>

<P>
There are a couple of important points to keep in mind when writing a 
makefile by hand:</P>

<UL>
    <LI>When there are multiple symbols with the same name in object files or 
    shared libraries, <CODE>PPCLink</CODE> used the symbol from the file which 
    appears first in arguments of the <CODE>PPCLink</CODE> command.  For this 
    reason, you will usually want the PythonCore and any other shared libraries 
    which are not part of the standard MPW runtime environment to appear before 
    the standard runtime libraries.  This is particularly the case with 
    StdCLib.  The "-d" option turns off the (often copious) warnings about 
    multiply defined symbols.
    <LI>You will want to make sure that the <CODE>HAVE_CONFIG_H</CODE> 
    preprocessor symbol is defined for most C source files using the <CODE>-d 
    HAVE_CONFIG_H</CODE> option to <CODE>MrC</CODE>.
</UL>

<P>
The file <A HREF="xx.ppc.slb.make.sit.hqx"><CODE>xx.ppc.slb.make</CODE></A> 
is included here for you to use as a starting point.
</P>

<H2><A NAME="using">Using the Extension Module</A></H2>

<P>
Once you have compiled your extension module, you will need to let Python 
know where it is.  You can either move it into a place on Python's search 
path - such as the <CODE>:Mac:Plugins</CODE> folder - or modify the path to 
include the location of your new module using the 
<CODE>EditPythonPrefs</CODE> applet.
</P>

<P>
Your work may not be completely done, as many extension modules have a 
Python wrapper around them.  If the Python was not written with portability 
in mind, you may need to do some more work to get that up and running. 
Indeed, if the Python part uses OS-specific features, like pipes, you may 
have to completely rewrite it if you can make it work at all.
</P>

<H2><A NAME="problems">Common Problems</A></H2>

<P>
There are a couple of common problems which occur when porting a module 
from another platform.  Fortunately, they are often easy to fix.
</P>

<H3>Static Forward Definitions</H3>

<P>
If you get a compiler error which looks something like:
</P>

<PRE>
File "xxmodule.c"; line 135 #Error: 'Xxo_Type' is already defined
</PRE>

<P>
then most likely either you have not set up <CODE>config.h</CODE> correctly 
to handle static forward definitions, or the module author has not adhered 
to the standard python conventions.  If the second is the case, find where 
the variable is first defined, and replace the <CODE>static</CODE> with 
<CODE>staticforward</CODE>. Then find the second place it is defined 
(usually the line where the compiler complained) and replace 
<CODE>static</CODE> with <CODE>statichere</CODE>.
</P>

<P>
If you have set up things correctly, you should now be able to compile.
</P>

<H3>Automatic Type Conversion</H3>

<P>
<CODE>MrC</CODE> seems to be a little pickier about automatically 
converting from one type to another than some other C compilers.  These can 
often be fixed by simply adding an explicit cast to the desired type.
</P>

<P>
XXX There may be a compiler option which relaxes this.  That would be a 
better solution.
</P>

<H2><A NAME="caveats">Caveats</A></H2>

<P>
As Jack Jansen pointed out on the Mac Python mailing list, there could 
potentially be conflicts between the MetroWerks C runtime which the Python 
core and standard modules was compiled with, and the MPW C runtime which 
your extension module is compiled with.  While things seem to work fine in 
everyday use, it is possible that there are bugs which have not been 
discovered yet.  Most likely these world take the form of standard C 
functions (most likely I/O functions due to conflicts between the SIOUX 
libraries and the SIOW libraries) not working as they are supposed to, or 
memory leaks caused by improper malloc/free.
</P>

<P>
Some such problems have been demonstrated by compiling modules with 
PythonCore linked after StdCLib - printf does not work properly in this 
setup, and I suspect that there will also be malloc/free problems in 
situations where the module allocates memory which is later disposed of by 
Python, or vice-versa. Compiling with PythonCore taking precedence over 
StdCLib seems to give the correct behaviour.
</P>

<P>
This method of compiling should be considered experimental for the time 
being.  <STRONG>Use it at your own risk.</STRONG>
</P>

<P>
If you notice any quirks in modules compiled this way, or have insight into 
what may go wrong or right with this situation, <A 
HREF="mailto:cwebster@nevada.edu">please contact me</A> so that I can add 
it to the HOWTO.
</P>

<P>
The ideal solution to this problem would be to get Python to compile using 
MPW (and a Python MPW Tool would be very neat indeed).  However, that does 
seem to be a major project.
</P>

<DIV class=footer>
<HR>
<BR>
&copy;<A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, 1999. <BR>
<!-- #LASTMODIFIED TEXT="Last modified" FORM="SHORT,TIME" -->
Last modified 14/12/99 12:17 PM
<!-- /#LASTMODIFIED -->
</DIV>

</BODY>

</HTML>