| <HTML> |
| <HEAD> |
| <TITLE>Building MacPython-OS9 from source</TITLE> |
| </HEAD> |
| <BODY> |
| <H1>Building MacPython-OS9 from source</H1> |
| <HR> |
| |
| This document explains how to build MacPython-OS9 from source. This is |
| necessary if you want to make modifications to the Python core. Building |
| Python is not something to be undertaken lightly, you need a reasonable |
| working knowledge of the CodeWarrior development environment, a good net |
| connection and probably quite some time too. <p> |
| |
| Note that if you only want to build new extension modules you don't need to |
| build Python from source, see the <a href="#extending">note on extending Python</a>.<p> |
| |
| The information density in this file is high, so you should probably |
| print it and read it at your leasure. Most things are explained only |
| once (and probably in the wrong place:-). <p> |
| |
| <blockquote> |
| First a warning: this information may become outdated if a new CodeWarrior is |
| released after MacPython. The |
| <a href="http://www.cwi.nl/~jack/macpython.html">MacPython homepage</a> will |
| hopefully have updated instructions in that case. These instructions are for CW7, |
| it is rumoured you may encounter some problems with newer versions of CodeWarrior. |
| </blockquote> |
| |
| I am interested in feedback on this document, send your |
| comments to the <A |
| HREF="http://www.python.org/sigs/pythonmac-sig/">Mac Python Special |
| Interest Group</A>. |
| |
| <H2>What you need.</H2> |
| |
| The following things you definitely need: |
| |
| <UL> |
| |
| <LI> You need a MacPython source distribution, of course. You can |
| obtain one via <A HREF="http://www.cwi.nl/~jack/macpython.html"> |
| http://www.cwi.nl/~jack/macpython.html</A> (which has up-to-date links |
| to the other packages needed too) and possibly also from the standard |
| <A HREF="ftp://ftp.python.org/pub/python/mac">python.org ftp |
| site</A>. <BR> |
| |
| A better alternative is to check the sources straight out of the CVS |
| repository, see below. Most of the packages mentioned here are also |
| available through CVS. Check the section on <a href="#cvs">CVS |
| repository use</a> below. |
| |
| <LI> You need MetroWerks CodeWarrior. The current distribution has |
| been built with CodeWarrior Pro 7.1. Ordering information is |
| available on the <A HREF="http://www.metrowerks.com/">MetroWerks |
| homepage</A>. Building Python with MPW, Think/Symantec C or the OSX |
| developer tools is impossible without major surgery. |
| |
| <LI> You need GUSI version 2, the Grand Unified Socket Interface, by |
| Matthias Neeracher. The original GUSI is obtainable from <A |
| HREF="ftp://gusi.sourceforge.net/pub/gusi/"> |
| ftp://gusi.sourceforge.net/pub/gusi/</A>. At |
| the moment Python is built with a modified version of GUSI |
| with Carbon adaptations, so it may be better to check the <A |
| HREF="http://www.cwi.nl/~jack/macpython.html">MacPython homepage</A> |
| for a GUSI that is most easily used for building Python. |
| |
| </UL> |
| |
| <A NAME="optional">The MacPython project files are configured to |
| include a plethora of optional modules</A>, and these modules need a |
| number of extra packages. To use the project files as-is you have to |
| download these packages too. Python has all such modules as |
| dynamically loaded modules, so if you don't need a certain package it |
| suffices to just refrain from builing the extension module. |
| Here are the locations for the various things |
| you need: |
| |
| <UL> |
| |
| <LI> Waste, a TextEdit replacement written by Marco Piovanelli, <A |
| HREF="mailto:piovanel@kagi.com"><piovanel@kagi.com></A>. Python |
| was built using version 2.0, which is included in the CodeWarrior |
| package. You can also obtain it from <A |
| HREF="http://www.merzwaren.com/waste"><http://www.merzwaren.com/waste></A> |
| and various other places. |
| |
| <LI> Gdbm library for the Mac. Available from Jack's Mac software page at |
| <A HREF="http://www.cwi.nl/~jack/macsoftware.html"> |
| http://www.cwi.nl/~jack/macsoftware.html</A> and <A HREF="ftp://ftp.cwi.nl/pub/jack/mac"> |
| ftp://ftp.cwi.nl/pub/jack/mac</A>. |
| |
| <LI> JPEG library by the Independent JPEG Group. A version including |
| Mac projects can be found at Jack's page mentioned above. |
| The most recent JPEG library can always be obtained from <A |
| HREF="ftp://ftp.uu.net/graphics/jpeg/">ftp://ftp.uu.net/graphics/jpeg/</A>. |
| |
| <LI> The netpbm/pbmplus, libtiff, zlib and png libraries. The netpbm distribution |
| (which includes libtiff) is generally available on Internet ftp |
| servers. For Python pbmplus, an older incarnation of netpbm, is |
| functionally identical to netpbm, since Python only uses the library |
| and not the complete applications. A distribution with correct |
| projects and library source only is available from, you guessed it, Jack's Mac software |
| page mentioned above. |
| |
| </UL> |
| |
| <H2>Setting Up</H2> |
| |
| Now that you have collected everything you should start with building |
| the various parts. If you don't want to fix |
| access paths try to set things up as follows: |
| |
| <PRE> |
| Top-level-folder: |
| GUSI2 |
| imglibs |
| jpeg |
| netpbm |
| libtiff |
| zlib |
| png |
| gdbm |
| Python |
| Modules |
| ... |
| Mac |
| Modules |
| Build |
| ... |
| </PRE> |
| |
| If your setup of the libraries is exactly the same as mine (which is |
| not very likely, unless you happen to work from the same CVS |
| repository) you can use the project <code>buildlibs.prj</code> in the |
| <code>:Mac:Build</code> folder to build all needed libraries in one |
| fell swoop, otherwise you will have to build the libraries one by |
| one. <p> |
| |
| First build GUSI, the Carbon variant. |
| <p> |
| |
| Next, in |
| <code>libjpeg</code>, <code>pbmplus</code>, |
| <code>zlib</code>, <code>libpng</code>, <code>gdbm</code>, |
| and<code>libtiff</code> you build all projects. Usually the projects are in "mac" |
| subfolders, sometimes they are in the main folder. Tcl/tk is a special |
| case, see below. |
| |
| <H2>The organization of the Python source tree</H2> |
| |
| Time for a short break, while we have a look at the organization of |
| the Python source tree. At the top level, we find the following |
| folders: |
| |
| <DL> |
| <DT> Demo |
| <DD> Demo programs that are not Mac-specific. Some of these may not |
| work. |
| |
| <DT> Extensions |
| <DD> Extensions to the interpreter that are not Mac-specific. Contains |
| the <code>img</code>, <code>Imaging</code> and <code>Numerical</code> extensions |
| in this distribution. |
| |
| <DT> Grammar |
| <DD> The Python grammar. Included for reference only, you cannot build |
| the parser on a Mac. |
| |
| <DT> Include |
| <DD> Machine-independent header files. |
| |
| <DT> Modules |
| <DD> Machine-independent optional modules. Not all of these will work |
| on the Mac. |
| |
| <DT> Lib |
| <DD> Machine-independent modules in Python. |
| |
| <DT> Lib:lib-dynload |
| <DD> This is where the dynamically-loaded plugin modules live. |
| |
| <DT> Lib:plat-mac |
| <DD> This is where most of the Mac-specific modules live. The modules here |
| are available both in MacPython-OS9 and MacPython-OSX. |
| |
| <DT> Objects |
| <DD> Machine-independent code for various object types. Most of these are |
| not really optional: the interpreter will not function without them. |
| |
| <DT> Parser |
| <DD> The Python parser (machine-independent). |
| |
| <DT> Python |
| <DD> The core interpreter. Most files are machine-independent, some |
| are unix-specific and not used on the Mac. |
| |
| <DT> Tools |
| <DD> Tools for python developers. Contains <code>modulator</code> which |
| builds skeleton C extension modules, <code>bgen</code> which generates |
| complete interface modules from information in C header files and |
| <code>freeze</code> which is used to turn Python scripts into real |
| applications (used by MacFreeze and BuildApplication) There are some |
| readme files, but more documentation is sorely needed. |
| |
| </DL> |
| |
| The mac-specific stuff lives in the <code>Mac</code> folder: |
| <DL> |
| <DT> Build |
| <DD> This is where the project files live and where you build the |
| libraries, shared libraries, executables and plugin modules. All the |
| resulting binaries, except for intermedeate results, are deposited in |
| the toplevel folder or the :Lib:lib-dynload folder (for plugin modules). |
| |
| <DT> Compat |
| <DD> Unix-compatability routines. Most of these are not used anymore, |
| since GUSI provides a rather complete emulation, but you may need |
| these if you are trying to build a non-GUSI python. |
| |
| <DT> Demo |
| <DD> Mac-specific demo programs, some of them annotated. |
| |
| <DT> Include |
| <DD> Mac-specific but compiler-independent include files. |
| |
| <DT> Lib |
| <DD> MacPython-OS9 specific standard modules which are not shared with |
| MacPython-OSX. |
| |
| <DT> Modules |
| <DD> Mac-specific builtin modules. Theoretically these are all |
| optional, but some are rather essential (like |
| <code>macosmodule</code>). A lot of these modules are generated with |
| <code>bgen</code>, in which case the bgen input files are included so |
| you can attempt to regenerate them or extend them. |
| |
| <DT> MPW |
| <DD> MPW-specific files. These have not been used or kept up-to-date |
| for a long time, so use at your own risk. |
| |
| <DT> mwerks |
| <DD> Mwerks-specific sources and headers. Contains glue code for |
| Pythons shared-library architecture, a replacement for |
| <code>malloc</code> and a directory with various projects for building |
| variations on the Python interpreter. The <code>mwerks_*.h</code> |
| files here are the option-setting files for the various interpreters |
| and such, comparable to the unix command-line <code>-D</code> options |
| to the compiler. Each project uses the correct option file as its |
| "prefix file" in the "C/C++ language" settings. Disabling optional |
| modules (for the 68K interpreter), building non-GUSI interpreters and |
| various other things are accomplished by modifying these files (and |
| possibly changing the list of files included in the project window, of |
| course). |
| |
| <DT> OSX |
| <DD> Specific to MacPython-OSX, not used by MacPython-OS9. |
| |
| <DT> OSXResources |
| <DD> Specific to MacPython-OSX, not used by MacPython-OS9. |
| |
| <DT> Python |
| <DD> Mac-specific parts of the core interpreter. |
| |
| <DT> Resources |
| <DD> Resource files needed to build the interpreter. |
| |
| <DT> Scripts |
| <DD> A collection of various mac-specific Python scripts. Some are |
| essential, some are useful but few are documented, so you will have to |
| use your imagination to work them out. |
| |
| <DT> Tools |
| <DD> A collection of tools, usually bigger than those in the scripts |
| folder. The important ones here are the IDE and macfreeze. The IDE is built |
| with the buildIDE.py script, which puts the resulting applet in the toplevel |
| folder. Macfreeze is usually invoked through the BuildApplication script, |
| but for more control over the freezing process you can run the main script here. |
| |
| |
| <DT> Unsupported |
| <DD> Modules that are not supported any longer but may still work with a little effort. |
| </DL> |
| |
| <H2>Building the PPC interpreter</H2> |
| |
| First you optionally build the external libraries with buildlibs.prj. <p> |
| |
| Then, the <code>fullbuild</code> script can be used to build |
| everything, but you need a fully-functional interpreter before you can |
| use it (and one that isn't rebuilt in the process: you cannot rebuild |
| a running program). You could copy the interpreter to a different |
| place and use that to run fullbuild. The <code>PythonStandSmall.prj</code> |
| project builds an interpreter that is suited to this, and it can also come |
| in handy if you need to debug things (which is easier in a static program). <p> |
| |
| In case you want to build by hand, or in case the <code>fullbuild</code> |
| script does not work, here is a breakdown of the various projects. <p> |
| |
| The projects for interpreter and core library are linked together, so |
| building the PythonInterpreter target |
| in <code>PythonInterpreter.prj</code> |
| will result in the whole core being built, but not the extension modules. <p> |
| |
| You will get about 100 warnings on "missing prototype" for the various module init |
| routines, ignore these. You will also get numerous warnings on functions from GUSI which |
| override functions from MSL, ignore these too. <p> |
| |
| Here is a breakdown of the projects: |
| |
| <DL> |
| |
| <DT> PythonCore |
| <DD> The shared library that contains the bulk of the interpreter and |
| its resources. |
| It is a good idea to immedeately put an alias to this |
| shared library in the <code>Extensions</code> folder of your system |
| folder. Do exactly that: put an <em>alias</em> there, copying or |
| moving the file will cause you grief later if you rebuild the library and |
| forget to copy it to the extensions folder again. The ConfigurePythonXXX applets |
| will also do this. <br> |
| |
| <DT> PythonInterpeter |
| <DD> The interpreter. This is basically a routine to call out to the |
| shared library. <p> |
| |
| <DT> Plugin projects |
| <DD> Each plugin module has a separate project, and these can be rebuilt on |
| the fly. Fullbuild (or actually it's little helper genpluginprojects) takes |
| care of this. |
| </DL> |
| |
| After creating the alias to <code>PythonCore</code> you remove any old |
| <code>Python XXXX Preferences</code> file from the <code>Preferences</code> folder |
| (if you had python installed on your system before) and run the interpreter once |
| to create the correct preferences file. <p> |
| |
| Next, you have to build the extension modules. |
| If you don't use fullbuild simply open each project and build it. |
| <p> |
| |
| Finally, you must build the standard applets: |
| <code>EditPythonPrefs</code>, <code>BuildApplet</code>, etc. For the N-th time: |
| fullbuild does this for you, but you can also manually drag/drop them onto |
| BuildApplet. <p> |
| |
| You are all set now, and should read the release notes and |
| <code>ReadMe</code> file from the <code>Mac</code> folder. |
| |
| Rebuilding .exp files is no longer needed since CodeWarrior 7. |
| |
| <H2><a name="cvs">Using the CVS source archive</a></H2> |
| |
| It is possible (and probably best) to access the Python sources through remote CVS. The |
| advantage of this is that you get the very latest sources, so any bug |
| fixed or new features will be immedeately available. This is also the |
| disadvantage, of course: as this is the same tree as is used for |
| development it may sometimes be a little less stable. <p> |
| |
| The CVS client of choice is Alexandre Parenteau's MacCVS. It can be |
| obtained through the <a href="http://www.wincvs.org">WinCVS |
| homepage</a>. MacCVS uses Internet Config to set file types correctly |
| based on the filename extension. In the maccvs preferences you should |
| also set (in the "binary files" section) "use mac encoding: |
| applesingle" and (in the "text files" section) "use ISO latin 1 |
| conversion". <p> |
| |
| It is a good idea to disable Quicktime Exchange in the Quicktime control |
| panel if you are on OS9 or before. Quicktime Exchange will magically map |
| some extensions to filetypes, and this can seriously hinder you if, for |
| instance, <code>.bmp</code> is not a Windows bitmap file. <p> |
| |
| The Python sources are checked out from the main |
| Python CVS archive on sourceforge.net, see the <a |
| href="http://www.python.org/download/cvs.html">Source access via |
| CVS</a> page for details. When you check the sources out you will get |
| something like <code>Python:dist:src</code>, and under that the |
| <code>Modules</code>, <code>Lib</code>, <code>Mac</code> etc hierarchy. The |
| <code>src</code> folder can be renamed to <code>Python</code>, and |
| is what this document refers to as the "toplevel Python folder". <P> |
| |
| The CVS repository does not contain all the projects for the plugin modules, |
| these are built with <code>fullbuild.py</code> normally. For this reason |
| it is probably a good idea to first build <code>PythonStandSmall.prj</code>, |
| which builds a fairly minimal interpreter, and then follow the |
| fullbuild instructions</a>. |
| |
| <H2>Odds and ends</H2> |
| |
| Some remarks that I could not fit in elsewhere: |
| |
| <UL> |
| |
| <LI> It may be possible to use the <code>PythonCore</code> shared |
| library to embed Python in another program, if your program can live |
| with using GUSI for I/O. Use PythonCore in stead of your MSL C library |
| (or, at the very least, link it before the normal C library). Ask for help |
| on PythonMac-SIG if you have problems with this. |
| |
| <LI> <a name="extending"></a>It is possible to build PPC extension |
| modules without building a complete Python. The binary distribution |
| installer can optionally install all the needed folders (the develop |
| option). A template for a dynamic module can be found in |
| <code>xx.prj</code>. |
| |
| <LI> The Python shared library architecture is a variant of the architecture |
| described as "application with shared libraries and dropins" in the MetroWerks |
| "Targeting MacOS" documentation. The Python Application and applet-template use |
| the <code>MSL AppRuntime.Lib</code> runtime library (with properly set CFM |
| initialization and termination routines). PythonCore uses <code>MSL Runtime.Lib</code>, |
| which is really intended for standalone programs but which we fool into working by |
| providing a dummy main program. |
| It is linked statically into PythonCore (and exported to the applications and plugins) |
| so we do not have to distribute yet another shared library. Plugin modules use |
| <code>MSL ShlibRuntime.Lib</code> (not the dropin runtime: modules are never unloaded) |
| and obtain the rest from PythonCore. PythonCore uses a |
| non-standard initialization entry point, <code>__initialize_with_resources</code>, to |
| be able to obtain resources from the library file later on. Plugins can do the same |
| (_tkinter does) or use the standard <code>__initialize</code> entry point. |
| |
| |
| </UL> |
| </BODY> |
| </HTML> |