Lots of things added. README written (mostly).
diff --git a/Tools/freeze/README b/Tools/freeze/README
new file mode 100644
index 0000000..9ab900f
--- /dev/null
+++ b/Tools/freeze/README
@@ -0,0 +1,161 @@
+THE FREEZE SCRIPT
+=================
+
+
+What is Freeze?
+---------------
+
+Freeze make it possible to ship arbitrary Python programs to people
+who don't have Python. The shipped file (called a "frozen" version of
+your Python program) is an executable, so this only works if your
+platform is compatible with that on the receiving end (this is usually
+a matter of having the same major operating system revision and CPU
+type).
+
+The shipped file contains a Python interpreter and large portions of
+the Python run-time. Some measures have been taken to avoid linking
+unneeded modules, but the resulting binary is usually not small.
+
+The Python source code of your program (and of the library modules
+written in Python that it uses) is not included in the binary --
+instead, the compiled byte-code (the instruction stream used
+internally by the interpreter) is incorporated. This gives some
+protection of your Python source code, though not much -- a
+disassembler for Python byte-code is available in the standard Python
+library. At least someone running "strings" on your binary won't see
+the source.
+
+
+How does Freeze know which modules to include?
+----------------------------------------------
+
+Freeze uses a pretty simple-minded algorithm to find the modules that
+your program uses: given a file containing Python source code, it
+scans for lines beginning with the word "import" or "from" (possibly
+preceded by whitespace) and then it knows where to find the module
+name(s) in those lines. It then recursively scans the source for
+those modules (if found, and not already processed) in the same way.
+
+Freeze will not see import statements hidden behind another statement,
+like this:
+
+ if some_test: import M # M not seen
+
+or like this:
+
+ import A; import B; import C # B and C not seen
+
+nor will it see import statements constructed using string
+operations and passed to 'exec', like this:
+
+ exec "import %s" % "M" # M not seen
+
+On the other hand, Freeze will think you are importing a module even
+if the import statement it sees will never be executed, like this:
+
+ if 0:
+ import M # M is seen
+
+One tricky issue: Freeze assumes that the Python interpreter and
+environment you're using to run Freeze is the same one that would be
+used to run your program, which should also be the same whose sources
+and installed files you will learn about in the next section. In
+particular, your PYTHONPATH setting should be the same as for running
+your program locally. (Tip: if the program doesn't run when you type
+"python hello.py" there's little chance of getting the frozen version
+to run.)
+
+
+How do I use Freeze?
+--------------------
+
+Ideally, you should be able to use it as follows:
+
+ python freeze.py hello.py
+
+where hello.py is your program and freeze.py is the main file of
+Freeze (in actuality, you'll probably specify an absolute pathname
+such as /ufs/guido/src/python/Demo/freeze/freeze.py).
+
+Unfortunately, this doesn't work. Well, it might, but somehow it's
+extremely unlikely that it'll work on the first try. (If it does,
+skip to the next section.) Most likely you'll get this error message:
+
+ needed directory /usr/local/lib/python/lib not found
+
+The reason is that Freeze require that some files that are normally
+kept inside the Python build tree are installed, and it searches for
+it in the default install location. (The default install prefix is
+/usr/local; these particular files are installed at lib/python/lib
+under the install prefix.)
+
+The particular set of files needed is installed only if you run "make
+libainstall" (note: "liba", not "lib") in the Python build tree (which
+is the tree where you build Python -- often, but not necessarily, this
+is also the Python source tree). If you have in fact done a "make
+libainstall" but used a different prefix, all you need to do is pass
+that same prefix to Freeze with the -p option:
+
+ python freeze.py -p your-prefix hello.py
+
+(If you haven't run "make libainstall" yet, go and do it now and don't
+come back until you've done it.)
+
+
+How do I configure Freeze?
+--------------------------
+
+It's a good idea to change the line marked with XXX in freeze.py (an
+assignment to variable PACK) to point to the absolute pathname of the
+directory where Freeze lives (Demo/freeze in the Python source tree.)
+This makes it possible to call Freeze from other directories.
+
+You can also edit the assignment to variable PREFIX -- this saves a
+lot of -p options.
+
+
+How do I use Freeze with extensions modules?
+--------------------------------------------
+
+XXX to be written. (In short: pass -e extensionbuilddir.)
+
+
+How do I use Freeze with dynamically loaded extension modules?
+--------------------------------------------------------------
+
+XXX to be written. (In short: pass -e modulebuilddir -- this even
+works if you built the modules in Python's own Modules directory.)
+
+
+
+What do I do next?
+------------------
+
+Freeze creates three files: frozen.c, config.c and Makefile. To
+produce the frozen version of your program, you can simply type
+"make". This should produce a binary file. If the filename argument
+to Freeze was "hello.py", the binary will be called "hello". On the
+other hand, if the argument was "hello", the binary will be called
+"hello.bin". If you passed any other filename, all bets are off. :-)
+In any case, the name of the file will be printed as the last message
+from Freeze.
+
+
+Help! I've tried everything but it doesn't work!
+-------------------------------------------------
+
+Freeze is currently beta software. You could email me a bug report.
+Please give as much context as possible -- "Freeze doesn't work" is
+not going to get much sympathy. You could fix the bug and send me a
+patch. You could learn Tcl.
+
+If you are thinking about debugging Freeze, start playing with a
+really simple program first (like "print 'hello world'"). If you
+can't get that to work there's something fundamentally wrong with your
+environment (or with your understanding of it). Gradually build it up
+to use more modules and extensions until you find where it stops
+working. After that, you're on your own -- happy hacking!
+
+
+--Guido van Rossum, CWI, Amsterdam <mailto:Guido.van.Rossum@cwi.nl>
+<http://www.cwi.nl/cwi/people/Guido.van.Rossum.html>
diff --git a/Tools/freeze/checkextensions.py b/Tools/freeze/checkextensions.py
new file mode 100644
index 0000000..a7890d8
--- /dev/null
+++ b/Tools/freeze/checkextensions.py
@@ -0,0 +1,87 @@
+# Check for a module in a set of extension directories.
+# An extension directory should contain a Setup file
+# and one or more .o files or a lib.a file.
+
+import os
+import string
+import parsesetup
+
+def checkextensions(unknown, extensions):
+ files = []
+ modules = []
+ edict = {}
+ for e in extensions:
+ setup = os.path.join(e, 'Setup')
+ liba = os.path.join(e, 'lib.a')
+ if not os.path.isfile(liba):
+ liba = None
+ edict[e] = parsesetup.getsetupinfo(setup), liba
+ for mod in unknown:
+ for e in extensions:
+ (mods, vars), liba = edict[e]
+ if not mods.has_key(mod):
+ continue
+ modules.append(mod)
+ if liba:
+ # If we find a lib.a, use it, ignore the
+ # .o files, and use *all* libraries for
+ # *all* modules in the Setup file
+ if liba in files:
+ break
+ files.append(liba)
+ for m in mods.keys():
+ files = files + select(e, mods, vars,
+ m, 1)
+ break
+ files = files + select(e, mods, vars, mod, 0)
+ break
+ return files, modules
+
+def select(e, mods, vars, mod, skipofiles):
+ files = []
+ for w in mods[mod]:
+ w = treatword(w)
+ if not w:
+ continue
+ w = expandvars(w, vars)
+ if skipofiles and w[-2:] == '.o':
+ continue
+ if w[0] != '-' and w[-2:] in ('.o', '.a'):
+ w = os.path.join(e, w)
+ files.append(w)
+ return files
+
+cc_flags = ['-I', '-D', '-U']
+cc_exts = ['.c', '.C', '.cc', '.c++']
+
+def treatword(w):
+ if w[:2] in cc_flags:
+ return None
+ if w[:1] == '-':
+ return w # Assume loader flag
+ head, tail = os.path.split(w)
+ base, ext = os.path.splitext(tail)
+ if ext in cc_exts:
+ tail = base + '.o'
+ w = os.path.join(head, tail)
+ return w
+
+def expandvars(str, vars):
+ i = 0
+ while i < len(str):
+ i = k = string.find(str, '$', i)
+ if i < 0:
+ break
+ i = i+1
+ var = str[i:i+1]
+ i = i+1
+ if var == '(':
+ j = string.find(str, ')', i)
+ if j < 0:
+ break
+ var = str[i:j]
+ i = j+1
+ if vars.has_key(var):
+ str = str[:k] + vars[var] + str[i:]
+ i = k
+ return str
diff --git a/Tools/freeze/findmodules.py b/Tools/freeze/findmodules.py
index 5041574..9e02f2b 100644
--- a/Tools/freeze/findmodules.py
+++ b/Tools/freeze/findmodules.py
@@ -21,6 +21,7 @@
for name in modules:
mod = os.path.basename(name)
if mod[-3:] == '.py': mod = mod[:-3]
+ elif mod[-4:] == '.pyc': mod = mod[:-4]
todo[mod] = name
done = closure(todo)
return done
@@ -94,7 +95,6 @@
# Return filename, or '<builtin>', or '<unknown>'.
builtins = sys.builtin_module_names
-if 'sys' not in builtins: builtins.append('sys')
tails = ['.py', '.pyc']
def findmodule(modname, path = sys.path):
diff --git a/Tools/freeze/freeze.py b/Tools/freeze/freeze.py
index c423819..7d52ccf 100755
--- a/Tools/freeze/freeze.py
+++ b/Tools/freeze/freeze.py
@@ -1,31 +1,49 @@
#! /usr/local/bin/python
# "Freeze" a Python script into a binary.
-# Usage: see first function below (before the imports!)
+# Usage: see variable usage_msg below (before the imports!)
# HINTS:
-# - Edit the line at XXX below before running!
+# - Edit the lines marked XXX below to localize.
# - You must have done "make inclinstall libainstall" in the Python
# build directory.
# - The script should not use dynamically loaded modules
# (*.so on most systems).
-# XXX Change the following line to point to your Demo/freeze directory!
-pack = '/ufs/guido/src/python/Demo/freeze'
+# Usage message
+
+usage_msg = """
+usage: freeze [-p prefix] [-e extension] ... script [module] ...
+
+-p prefix: This is the prefix used when you ran
+ 'Make inclinstall libainstall' in the Python build directory.
+ (If you never ran this, freeze won't work.)
+ The default is /usr/local.
+
+-e extension: A directory containing additional .o files that
+ may be used to resolve modules. This directory
+ should also have a Setup file describing the .o files.
+ More than one -e option may be given.
+
+script: The Python script to be executed by the resulting binary.
+
+module ...: Additional Python modules (referenced by pathname)
+ that will be included in the resulting binary. These
+ may be .py or .pyc files.
+"""
-# Print usage message and exit
+# XXX Change the following line to point to your Demo/freeze directory
+PACK = '/ufs/guido/src/python/Demo/freeze'
-def usage(msg = None):
- if msg:
- sys.stderr.write(str(msg) + '\n')
- sys.stderr.write('usage: freeze [-p prefix] script [module] ...\n')
- sys.exit(2)
+# XXX Change the following line to point to your install prefix
+PREFIX = '/usr/local'
# Import standard modules
+import cmp
import getopt
import os
import string
@@ -38,29 +56,27 @@
dir = os.path.dirname(sys.argv[0])
if dir:
pack = dir
+else:
+ pack = PACK
addpack.addpack(pack)
# Import the freeze-private modules
+import checkextensions
import findmodules
import makeconfig
import makefreeze
import makemakefile
import parsesetup
-hint = """
-Use the '-p prefix' command line option to specify the prefix used
-when you ran 'Make inclinstall libainstall' in the Python build directory.
-(Please specify an absolute path.)
-"""
-
# Main program
def main():
# overridable context
- prefix = '/usr/local' # settable with -p option
+ prefix = PREFIX # settable with -p option
+ extensions = []
path = sys.path
# output files
@@ -71,14 +87,14 @@
# parse command line
try:
- opts, args = getopt.getopt(sys.argv[1:], 'p:')
- if not args:
- raise getopt.error, 'not enough arguments'
+ opts, args = getopt.getopt(sys.argv[1:], 'e:p:')
except getopt.error, msg:
usage('getopt error: ' + str(msg))
# proces option arguments
for o, a in opts:
+ if o == '-e':
+ extensions.append(a)
if o == '-p':
prefix = a
@@ -89,13 +105,13 @@
frozenmain_c = os.path.join(binlib, 'frozenmain.c')
makefile_in = os.path.join(binlib, 'Makefile')
defines = ['-DHAVE_CONFIG_H', '-DUSE_FROZEN', '-DNO_MAIN',
- '-DPTHONPATH=\\"$(PYTHONPATH)\\"']
+ '-DPYTHONPATH=\\"$(PYTHONPATH)\\"']
includes = ['-I' + incldir, '-I' + binlib]
- # sanity check of locations
- for dir in prefix, binlib, incldir:
+ # sanity check of directories and files
+ for dir in [prefix, binlib, incldir] + extensions:
if not os.path.exists(dir):
- usage('needed directory %s not found' % dir + hint)
+ usage('needed directory %s not found' % dir)
if not os.path.isdir(dir):
usage('%s: not a directory' % dir)
for file in config_c_in, makefile_in, frozenmain_c:
@@ -103,6 +119,16 @@
usage('needed file %s not found' % file)
if not os.path.isfile(file):
usage('%s: not a plain file' % file)
+ for dir in extensions:
+ setup = os.path.join(dir, 'Setup')
+ if not os.path.exists(setup):
+ usage('needed file %s not found' % setup)
+ if not os.path.isfile(setup):
+ usage('%s: not a plain file' % setup)
+
+ # check that enough arguments are passed
+ if not args:
+ usage('at least one filename argument required')
# check that file arguments exist
for arg in args:
@@ -128,30 +154,61 @@
dict = findmodules.findmodules(scriptfile, modules, path)
+ backup = frozen_c + '~'
+ try:
+ os.rename(frozen_c, backup)
+ except os.error:
+ backup = None
+ outfp = open(frozen_c, 'w')
+ try:
+ makefreeze.makefreeze(outfp, dict)
+ finally:
+ outfp.close()
+ if backup:
+ if cmp.cmp(backup, frozen_c):
+ sys.stderr.write('%s not changed, not written\n' %
+ frozen_c)
+ os.rename(backup, frozen_c)
+
builtins = []
+ unknown = []
mods = dict.keys()
mods.sort()
for mod in mods:
if dict[mod] == '<builtin>':
builtins.append(mod)
elif dict[mod] == '<unknown>':
- sys.stderr.write(
- 'Warning: module %s not found anywhere\n' %
- mod)
+ unknown.append(mod)
- outfp = open(frozen_c, 'w')
- try:
- makefreeze.makefreeze(outfp, dict)
- finally:
- outfp.close()
+ addfiles = []
+ if unknown:
+ addfiles, addmods = \
+ checkextensions.checkextensions(unknown, extensions)
+ for mod in addmods:
+ unknown.remove(mod)
+ builtins = builtins + addmods
+ if unknown:
+ sys.stderr.write('Warning: unknown modules remain: %s\n' %
+ string.join(unknown))
+ builtins.sort()
infp = open(config_c_in)
+ backup = config_c + '~'
+ try:
+ os.rename(config_c, backup)
+ except os.error:
+ backup = None
outfp = open(config_c, 'w')
try:
makeconfig.makeconfig(infp, outfp, builtins)
finally:
outfp.close()
infp.close()
+ if backup:
+ if cmp.cmp(backup, config_c):
+ sys.stderr.write('%s not changed, not written\n' %
+ config_c)
+ os.rename(backup, config_c)
cflags = defines + includes + ['$(OPT)']
libs = []
@@ -166,7 +223,8 @@
somevars[key] = makevars[key]
somevars['CFLAGS'] = string.join(cflags) # override
- files = ['$(OPT)', config_c, frozenmain_c] + libs + \
+ files = ['$(OPT)', config_c, frozen_c, frozenmain_c] + \
+ addfiles + libs + \
['$(MODLIBS)', '$(LIBS)', '$(SYSLIBS)']
outfp = open(makefile, 'w')
@@ -179,4 +237,14 @@
print 'Now run make to build the target:', target
+
+# Print usage message and exit
+
+def usage(msg = None):
+ if msg:
+ sys.stderr.write(str(msg) + '\n')
+ sys.stderr.write(usage_msg)
+ sys.exit(2)
+
+
main()