blob: 88f5778fa5a12ccdeeb1a993987f91c63901f428 [file] [log] [blame]
Guido van Rossumaa1e1401992-04-06 14:02:49 +00001\documentstyle[11pt]{article}
Guido van Rossum16d6e711994-08-08 12:30:22 +00002\newcommand{\Cpp}{C\protect\raisebox{.18ex}{++}}
Guido van Rossum2bbb3c01992-02-11 15:52:24 +00003
4\title{
5Interactively Testing Remote Servers Using the Python Programming Language
6}
7
8\author{
9 Guido van Rossum \\
Guido van Rossumd983cde1995-03-15 12:53:31 +000010 Dept. AA, CWI, P.O. Box 94079 \\
Guido van Rossumdb65a6c1993-11-05 17:11:16 +000011 1090 GB Amsterdam, The Netherlands \\
Guido van Rossum2bbb3c01992-02-11 15:52:24 +000012 E-mail: {\tt guido@cwi.nl}
13\and
14 Jelke de Boer \\
15 HIO Enschede; P.O.Box 1326 \\
16 7500 BH Enschede, The Netherlands
17}
18
19\begin{document}
20
21\maketitle
22
23\begin{abstract}
24This paper describes how two tools that were developed quite
25independently gained in power by a well-designed connection between
26them. The tools are Python, an interpreted prototyping language, and
27AIL, a Remote Procedure Call stub generator. The context is Amoeba, a
28well-known distributed operating system developed jointly by the Free
29University and CWI in Amsterdam.
30
31As a consequence of their integration, both tools have profited:
32Python gained usability when used with Amoeba --- for which it was not
33specifically developed --- and AIL users now have a powerful
34interactive tool to test servers and to experiment with new
35client/server interfaces.%
36\footnote{
37An earlier version of this paper was presented at the Spring 1991
38EurOpen Conference in Troms{\o} under the title ``Linking a Stub
39Generator (AIL) to a Prototyping Language (Python).''
40}
41\end{abstract}
42
43\section{Introduction}
44
45Remote Procedure Call (RPC) interfaces, used in distributed systems
46like Amoeba
47\cite{Amoeba:IEEE,Amoeba:CACM},
48have a much more concrete character than local procedure call
49interfaces in traditional systems. Because clients and servers may
50run on different machines, with possibly different word size, byte
51order, etc., much care is needed to describe interfaces exactly and to
52implement them in such a way that they continue to work when a client
53or server is moved to a different machine. Since machines may fail
54independently, error handling must also be treated more carefully.
55
56A common approach to such problems is to use a {\em stub generator}.
57This is a program that takes an interface description and transforms
58it into functions that must be compiled and linked with client and
59server applications. These functions are called by the application
60code to take care of details of interfacing to the system's RPC layer,
61to implement transformations between data representations of different
62machines, to check for errors, etc. They are called `stubs' because
63they don't actually perform the action that they are called for but
64only relay the parameters to the server
65\cite{RPC}.
66
67Amoeba's stub generator is called AIL, which stands for Amoeba
68Interface Language
69\cite{AIL}.
70The first version of AIL generated only C functions, but an explicit
71goal of AIL's design was {\em retargetability}: it should be possible
72to add back-ends that generate stubs for different languages from the
73same interface descriptions. Moreover, the stubs generated by
74different back-ends must be {\em interoperable}: a client written in
75Modula-3, say, should be able to use a server written in C, and vice
76versa.
77
78This interoperability is the key to the success of the marriage
79between AIL and Python. Python is a versatile interpreted language
80developed by the first author. Originally intended as an alternative
81for the kind of odd jobs that are traditionally solved by a mixture of
82shell scripts, manually given shell commands, and an occasional ad hoc
83C program, Python has evolved into a general interactive prototyping
84language. It has been applied to a wide range of problems, from
85replacements for large shell scripts to fancy graphics demos and
86multimedia applications.
87
88One of Python's strengths is the ability for the user to type in some
89code and immediately run it: no compilation or linking is necessary.
90Interactive performance is further enhanced by Python's concise, clear
91syntax, its very-high-level data types, and its lack of declarations
92(which is compensated by run-time type checking). All this makes
93programming in Python feel like a leisure trip compared to the hard
94work involved in writing and debugging even a smallish C program.
95
96It should be clear by now that Python will be the ideal tool to test
97servers and their interfaces. Especially during the development of a
98complex server, one often needs to generate test requests on an ad hoc
99basis, to answer questions like ``what happens if request X arrives
100when the server is in state Y,'' to test the behavior of the server
101with requests that touch its limitations, to check server responses to
102all sorts of wrong requests, etc. Python's ability to immediately
103execute `improvised' code makes it a much better tool for this
104situation than C.
105
106The link to AIL extends Python with the necessary functionality to
107connect to arbitrary servers, making the server testbed sketched above
108a reality. Python's high-level data types, general programming
109features, and system interface ensure that it has all the power and
110flexibility needed for the job.
111
112One could go even further than this. Current distributed operating
113systems, based on client-server interaction, all lack a good command
114language or `shell' to give adequate access to available services.
115Python has considerable potential for becoming such a shell.
116
117\subsection{Overview of this Paper}
118
119The rest of this paper contains three major sections and a conclusion.
120First an overview of the Python programming language is given. Next
121comes a short description of AIL, together with some relevant details
122about Amoeba. Finally, the design and construction of the link
123between Python and AIL is described in much detail. The conclusion
124looks back at the work and points out weaknesses and strengths of
125Python and AIL that were discovered in the process.
126
127\section{An Overview of Python}
128
129Python%
130\footnote{
131Named after the funny TV show, not the nasty reptile.
132}
133owes much to ABC
134\cite{ABC},
135a language developed at CWI as a programming language for non-expert
136computer users. Python borrows freely from ABC's syntax and data
137types, but adds modules, exceptions and classes, extensibility, and
138the ability to call system functions. The concepts of modules,
139exceptions and (to some extent) classes are influenced strongly by
140their occurrence in Modula-3
141\cite{Modula-3}.
142
143Although Python resembles ABC in many ways, there is a a clear
144difference in application domain. ABC is intended to be the only
145programming language for those who use a computer as a tool, but
146occasionally need to write a program. For this reason, ABC is not
147just a programming language but also a programming environment, which
148comes with an integrated syntax-directed editor and some source
149manipulation commands. Python, on the other hand, aims to be a tool
150for professional (system) programmers, for whom having a choice of
151languages with different feature sets makes it possible to choose `the
152right tool for the job.' The features added to Python make it more
153useful than ABC in an environment where access to system functions
154(such as file and directory manipulations) are common. They also
155support the building of larger systems and libraries. The Python
156implementation offers little in the way of a programming environment,
157but is designed to integrate seamlessly with existing programming
158environments (e.g. UNIX and Emacs).
159
160Perhaps the best introduction to Python is a short example. The
161following is a complete Python program to list the contents of a UNIX
162directory.
163\begin{verbatim}
164import sys, posix
165
166def ls(dirname): # Print sorted directory contents
167 names = posix.listdir(dirname)
168 names.sort()
169 for name in names:
170 if name[0] != '.': print name
171
172ls(sys.argv[1])
173\end{verbatim}
174The largest part of this program, in the middle starting with {\tt
175def}, is a function definition. It defines a function named {\tt ls}
176with a single parameter called {\tt dirname}. (Comments in Python
177start with `\#' and extend to the end of the line.) The function body
178is indented: Python uses indentation for statement grouping instead of
179braces or begin/end keywords. This is shorter to type and avoids
180frustrating mismatches between the perception of grouping by the user
181and the parser. Python accepts one statement per line; long
182statements may be broken in pieces using the standard backslash
183convention. If the body of a compound statement is a single, simple
184statement, it may be placed on the same line as the head.
185
186The first statement of the function body calls the function {\tt
187listdir} defined in the module {\tt posix}. This function returns a
188list of strings representing the contents of the directory name passed
189as a string argument, here the argument {\tt dirname}. If {\tt
190dirname} were not a valid directory name, or perhaps not even a
191string, {\tt listdir} would raise an exception and the next statement
192would never be reached. (Exceptions can be caught in Python; see
193later.) Assuming {\tt listdir} returns normally, its result is
194assigned to the local variable {\tt names}.
195
196The second statement calls the method {\tt sort} of the variable {\tt
197names}. This method is defined for all lists in Python and does the
198obvious thing: the elements of the list are reordered according to
199their natural ordering relationship. Since in our example the list
Guido van Rossumd983cde1995-03-15 12:53:31 +0000200contains strings, they are sorted in ascending \ASCII{} order.
Guido van Rossum2bbb3c01992-02-11 15:52:24 +0000201
202The last two lines of the function contain a loop that prints all
203elements of the list whose first character isn't a period. In each
204iteration, the {\tt for} statement assigns an element of the list to
205the local variable {\tt name}. The {\tt print} statement is intended
206for simple-minded output; more elaborate formatting is possible with
207Python's string handling functions.
208
209The other two parts of the program are easily explained. The first
210line is an {\tt import} statement that tells the interpreter to import
211the modules {\tt sys} and {\tt posix}. As it happens these are both
212built into the interpreter. Importing a module (built-in or
213otherwise) only makes the module name available in the current scope;
214functions and data defined in the module are accessed through the dot
215notation as in {\tt posix.listdir}. The scope rules of Python are
216such that the imported module name {\tt posix} is also available in
217the function {\tt ls} (this will be discussed in more detail later).
218
219Finally, the last line of the program calls the {\tt ls} function with
220a definite argument. It must be last since Python objects must be
221defined before they can be used; in particular, the function {\tt ls}
222must be defined before it can be called. The argument to {\tt ls} is
223{\tt sys.argv[1]}, which happens to be the Python equivalent of {\tt
224\$1} in a shell script or {\tt argv[1]} in a C program's {\tt main}
225function.
226
227\subsection{Python Data Types}
228
229(This and the following subsections describe Python in quite a lot of
230detail. If you are more interested in AIL, Amoeba and how they are
231linked with Python, you can skip to section 3 now.)
232
233Python's syntax may not have big surprises (which is exactly as it
234should be), but its data types are quite different from what is found
235in languages like C, Ada or Modula-3. All data types in Python, even
236integers, are `objects'. All objects participate in a common garbage
237collection scheme (currently implemented using reference counting).
238Assignment is cheap, independent of object size and type: only a
239pointer to the assigned object is stored in the assigned-to variable.
240No type checking is performed on assignment; only specific operations
241like addition test for particular operand types.
242
243The basic object types in Python are numbers, strings, tuples, lists
244and dictionaries. Some other object types are open files, functions,
245modules, classes, and class instances; even types themselves are
246represented as objects. Extension modules written in C can define
247additional object types; examples are objects representing windows and
248Amoeba capabilities. Finally, the implementation itself makes heavy
249use of objects, and defines some private object types that aren't
250normally visible to the user. There is no explicit pointer type in
251Python.
252
253{\em Numbers}, both integers and floating point, are pretty
254straightforward. The notation for numeric literals is the same as in
255C, including octal and hexadecimal integers; precision is the same as
256{\tt long} or {\tt double} in C\@. A third numeric type, `long
257integer', written with an `L' suffix, can be used for arbitrary
258precision calculations. All arithmetic, shifting and masking
259operations from C are supported.
260
261{\em Strings} are `primitive' objects just like numbers. String
262literals are written between single quotes, using similar escape
263sequences as in C\@. Operations are built into the language to
264concatenate and to replicate strings, to extract substrings, etc.
265There is no limit to the length of the strings created by a program.
266There is no separate character data type; strings of length one do
267nicely.
268
269{\em Tuples} are a way to `pack' small amounts of heterogeneous data
270together and carry them around as a unit. Unlike structure members in
271C, tuple items are nameless. Packing and unpacking assignments allow
272access to the items, for example:
273\begin{verbatim}
274x = 'Hi', (1, 2), 'World' # x is a 3-item tuple,
275 # its middle item is (1, 2)
276p, q, r = x # unpack x into p, q and r
277a, b = q # unpack q into a and b
278\end{verbatim}
279A combination of packing and unpacking assignment can be used as
280parallel assignment, and is idiom for permutations, e.g.:
281\begin{verbatim}
282p, q = q, p # swap without temporary
283a, b, c = b, c, a # cyclic permutation
284\end{verbatim}
285Tuples are also used for function argument lists if there is more than
286one argument. A tuple object, once created, cannot be modified; but
287it is easy enough to unpack it and create a new, modified tuple from
288the unpacked items and assign this to the variable that held the
289original tuple object (which will then be garbage-collected).
290
291{\em Lists} are array-like objects. List items may be arbitrary
292objects and can be accessed and changed using standard subscription
293notation. Lists support item insertion and deletion, and can
294therefore be used as queues, stacks etc.; there is no limit to their
295size.
296
297Strings, tuples and lists together are {\em sequence} types. These
298share a common notation for generic operations on sequences such as
299subscription, concatenation, slicing (taking subsequences) and
300membership tests. As in C, subscripts start at 0.
301
302{\em Dictionaries} are `mappings' from one domain to another. The
303basic operations on dictionaries are item insertion, extraction and
304deletion, using subscript notation with the key as subscript. (The
305current implementation allows only strings in the key domain, but a
306future version of the language may remove this restriction.)
307
308\subsection{Statements}
309
310Python has various kinds of simple statements, such as assignments
311and {\tt print} statements, and several kinds of compound statements,
312like {\tt if} and {\tt for} statements. Formally, function
313definitions and {\tt import} statements are also statements, and there
314are no restrictions on the ordering of statements or their nesting:
315{\tt import} may be used inside a function, functions may be defined
316conditionally using an {\tt if} statement, etc. The effect of a
317declaration-like statement takes place only when it is executed.
318
319All statements except assignments and expression statements begin with
320a keyword: this makes the language easy to parse. An overview of the
321most common statement forms in Python follows.
322
323An {\em assignment} has the general form
324\vspace{\itemsep}
325
326\noindent
327{\em variable $=$ variable $= ... =$ variable $=$ expression}
328\vspace{\itemsep}
329
330It assigns the value of the expression to all listed variables. (As
331shown in the section on tuples, variables and expressions can in fact
332be comma-separated lists.) The assignment operator is not an
333expression operator; there are no horrible things in Python like
334\begin{verbatim}
335while (p = p->next) { ... }
336\end{verbatim}
337Expression syntax is mostly straightforward and will not be explained
338in detail here.
339
340An {\em expression statement} is just an expression on a line by
341itself. This writes the value of the expression to standard output,
342in a suitably unambiguous way, unless it is a `procedure call' (a
343function call that returns no value). Writing the value is useful
344when Python is used in `calculator mode', and reminds the programmer
345not to ignore function results.
346
347The {\tt if} statement allows conditional execution. It has optional
348{\tt elif} and {\tt else} parts; a construct like {\tt
349if...elif...elif...elif...else} can be used to compensate for the
350absence of a {\em switch} or {\em case} statement.
351
352Looping is done with {\tt while} and {\tt for} statements. The latter
353(demonstrated in the `ls' example earlier) iterates over the elements
354of a `sequence' (see the discussion of data types below). It is
355possible to terminate a loop with a {\tt break} statement or to start
356the next iteration with {\tt continue}. Both looping statements have
357an optional {\tt else} clause which is executed after the loop is
358terminated normally, but skipped when it is terminated by {\tt break}.
359This can be handy for searches, to handle the case that the item is
360not found.
361
362Python's {\em exception} mechanism is modelled after that of Modula-3.
363Exceptions are raised by the interpreter when an illegal operation is
364tried. It is also possible to explicitly raise an exception with the
365{\tt raise} statement:
366\vspace{\itemsep}
367
368\noindent
369{\tt raise {\em expression}, {\em expression}}
370\vspace{\itemsep}
371
372The first expression identifies which exception should be raised;
373there are several built-in exceptions and the user may define
374additional ones. The second, optional expression is passed to the
375handler, e.g. as a detailed error message.
376
377Exceptions may be handled (caught) with the {\tt try} statement, which
378has the following general form:
379\vspace{\itemsep}
380
381\noindent
382{\tt
383\begin{tabular}{l}
384try: {\em block} \\
385except {\em expression}, {\em variable}: {\em block} \\
386except {\em expression}, {\em variable}: {\em block} \\
387... \\
388except: {\em block}
389\end{tabular}
390}
391\vspace{\itemsep}
392
393When an exception is raised during execution of the first block, a
394search for an exception handler starts. The first {\tt except} clause
395whose {\em expression} matches the exception is executed. The
396expression may specify a list of exceptions to match against. A
397handler without an expression serves as a `catch-all'. If there is no
398match, the search for a handler continues with outer {\tt try}
399statements; if no match is found on the entire invocation stack, an
400error message and stack trace are printed, and the program is
401terminated (interactively, the interpreter returns to its main loop).
402
403Note that the form of the {\tt except} clauses encourages a style of
404programming whereby only selected exceptions are caught, passing
405unanticipated exceptions on to the caller and ultimately to the user.
406This is preferable over a simpler `catch-all' error handling
407mechanism, where a simplistic handler intended to catch a single type
408of error like `file not found' can easily mask genuine programming
409errors --- especially in a language like Python which relies strongly
410on run-time checking and allows the catching of almost any type of
411error.
412
413Other common statement forms, which we have already encountered, are
414function definitions, {\tt import} statements and {\tt print}
415statements. There is also a {\tt del} statement to delete one or more
416variables, a {\tt return} statement to return from a function, and a
417{\tt global} statement to allow assignments to global variables.
418Finally, the {\tt pass} statement is a no-op.
419
420\subsection{Execution Model}
421
422A Python program is executed by a stack-based interpreter.
423
424When a function is called, a new `execution environment' for it is
425pushed onto the stack. An execution environment contains (among other
426data) pointers to two `symbol tables' that are used to hold variables:
427the local and the global symbol table. The local symbol table
428contains local variables of the current function invocation (including
429the function arguments); the global symbol table contains variables
430defined in the module containing the current function.
431
432The `global' symbol table is thus only global with respect to the
433current function. There are no system-wide global variables; using
434the {\tt import} statement it is easy enough to reference variables
435that are defined in other modules. A system-wide read-only symbol
436table is used for built-in functions and constants though.
437
438On assignment to a variable, by default an entry for it is made in the
439local symbol table of the current execution environment. The {\tt
440global} command can override this (it is not enough that a global
441variable by the same name already exists). When a variable's value is
442needed, it is searched first in the local symbol table, then in the
443global one, and finally in the symbol table containing built-in
444functions and constants.
445
446The term `variable' in this context refers to any name: functions and
447imported modules are searched in exactly the same way.
448
449Names defined in a module's symbol table survive until the end of the
450program. This approximates the semantics of file-static global
451variables in C or module variables in Modula-3. A module is
452initialized the first time it is imported, by executing the text of
453the module as a parameterless function whose local and global symbol
454tables are the same, so names are defined in module's symbol table.
455(Modules implemented in C have another way to define symbols.)
456
457A Python main program is read from standard input or from a script
458file passed as an argument to the interpreter. It is executed as if
459an anonymous module was imported. Since {\tt import} statements are
460executed like all other statements, the initialization order of the
461modules used in a program is defined by the flow of control through
462the program.
463
464The `attribute' notation {\em m.name}, where {\em m} is a module,
465accesses the symbol {\em name} in that module's symbol table. It can
466be assigned to as well. This is in fact a special case of the
467construct {\em x.name} where {\em x} denotes an arbitrary object; the
468type of {\em x} determines how this is to be interpreted, and what
469assignment to it means.
470
471For instance, when {\tt a} is a list object, {\tt a.append} yields a
472built-in `method' object which, when called, appends an item to {\tt a}.
473(If {\tt a} and {\tt b} are distinct list objects, {\tt a.append} and
474{\tt b.append} are distinguishable method objects.) Normally, in
475statements like {\tt a.append(x)}, the method object {\tt a.append} is
476called and then discarded, but this is a matter of convention.
477
478List attributes are read-only --- the user cannot define new list
479methods. Some objects, like numbers and strings, have no attributes
480at all. Like all type checking in Python, the meaning of an attribute
481is determined at run-time --- when the parser sees {\em x.name}, it
482has no idea of the type of {\em x}. Note that {\em x} here does not
483have to be a variable --- it can be an arbitrary (perhaps
484parenthesized) expression.
485
486Given the flexibility of the attribute notation, one is tempted to use
487methods to replace all standard operations. Yet, Python has kept a
488small repertoire of built-in functions like {\tt len()} and {\tt
489abs()}. The reason is that in some cases the function notation is
490more familiar than the method notation; just like programs would
491become less readable if all infix operators were replaced by function
492calls, they would become less readable if all function calls had to be
493replaced by method calls (and vice versa!).
494
495The choice whether to make something a built-in function or a method
496is a matter of taste. For arithmetic and string operations, function
497notation is preferred, since frequently the argument to such an
498operation is an expression using infix notation, as in {\tt abs(a+b)};
499this definitely looks better than {\tt (a+b).abs()}. The choice
500between make something a built-in function or a function defined in a
501built-in method (requiring {\tt import}) is similarly guided by
502intuition; all in all, only functions needed by `general' programming
503techniques are built-in functions.
504
505\subsection{Classes}
506
507Python has a class mechanism distinct from the object-orientation
508already explained. A class in Python is not much more than a
509collection of methods and a way to create class instances. Class
510methods are ordinary functions whose first parameter is the class
511instance; they are called using the method notation.
512
513For instance, a class can be defined as follows:
514\begin{verbatim}
515class Foo:
516 def meth1(self, arg): ...
517 def meth2(self): ...
518\end{verbatim}
519A class instance is created by
520{\tt x = Foo()}
521and its methods can be called thus:
522\begin{verbatim}
523x.meth1('Hi There!')
524x.meth2()
525\end{verbatim}
526The functions used as methods are also available as attributes of the
527class object, and the above method calls could also have been written
528as follows:
529\begin{verbatim}
530Foo.meth1(x, 'Hi There!')
531Foo.meth2(x)
532\end{verbatim}
533Class methods can store instance data by assigning to instance data
534attributes, e.g.:
535\begin{verbatim}
536self.size = 100
537self.title = 'Dear John'
538\end{verbatim}
539Data attributes do not have to be declared; as with local variables,
540they spring into existence when assigned to. It is a matter of
541discretion to avoid name conflicts with method names. This facility
542is also available to class users; instances of a method-less class can
543be used as records with named fields.
544
545There is no built-in mechanism for instance initialization. Classes
546by convention provide an {\tt init()} method which initializes the
547instance and then returns it, so the user can write
548\begin{verbatim}
549x = Foo().init('Dr. Strangelove')
550\end{verbatim}
551
552Any user-defined class can be used as a base class to derive other
553classes. However, built-in types like lists cannot be used as base
Guido van Rossum16d6e711994-08-08 12:30:22 +0000554classes. (Incidentally, the same is true in \Cpp{} and Modula-3.) A
Guido van Rossum2bbb3c01992-02-11 15:52:24 +0000555class may override any method of its base classes. Instance methods
556are first searched in the method list of their class, and then,
557recursively, in the method lists of their base class. Initialization
558methods of derived classes should explicitly call the initialization
559methods of their base class.
560
561A simple form of multiple inheritance is also supported: a class can
562have multiple base classes, but the language rules for resolving name
563conflicts are somewhat simplistic, and consequently the feature has so
564far found little usage.
565
566\subsection{The Python Library}
567
568Python comes with an extensive library, structured as a collection of
569modules. A few modules are built into the interpreter: these
570generally provide access to system libraries implemented in C such as
571mathematical functions or operating system calls. Two built-in
572modules provide access to internals of the interpreter and its
573environment. Even abusing these internals will at most cause an
574exception in the Python program; the interpreter will not dump core
575because of errors in Python code.
576
577Most modules however are written in Python and distributed with the
578interpreter; they provide general programming tools like string
579operations and random number generators, provide more convenient
580interfaces to some built-in modules, or provide specialized services
581like a {\em getopt}-style command line option processor for
582stand-alone scripts.
583
584There are also some modules written in Python that dig deep in the
585internals of the interpreter; there is a module to browse the stack
586backtrace when an unhandled exception has occurred, one to disassemble
587the internal representation of Python code, and even an interactive
588source code debugger which can trace Python code, set breakpoints,
589etc.
590
591\subsection{Extensibility}
592
593It is easy to add new built-in modules written in C to the Python
594interpreter. Extensions appear to the Python user as built-in
595modules. Using a built-in module is no different from using a module
596written in Python, but obviously the author of a built-in module can
597do things that cannot be implemented purely in Python.
598
599In particular, built-in modules can contain Python-callable functions
600that call functions from particular system libraries (`wrapper
601functions'), and they can define new object types. In general, if a
602built-in module defines a new object type, it should also provide at
603least one function that creates such objects. Attributes of such
604object types are also implemented in C; they can return data
605associated with the object or methods, implemented as C functions.
606
607For instance, an extension was created for Amoeba: it provides wrapper
608functions for the basic Amoeba name server functions, and defines a
609`capability' object type, whose methods are file server operations.
610Another extension is a built-in module called {\tt posix}; it provides
611wrappers around post UNIX system calls. Extension modules also
612provide access to two different windowing/graphics interfaces: STDWIN
613\cite{STDWIN}
614(which connects to X11 on UNIX and to the Mac Toolbox on the
615Macintosh), and the Graphics Library (GL) for Silicon Graphics
616machines.
617
618Any function in an extension module is supposed to type-check its
619arguments; the interpreter contains a convenience function to
620facilitate extracting C values from arguments and type-checking them
621at the same time. Returning values is also painless, using standard
622functions to create Python objects from C values.
623
624On some systems extension modules may be dynamically loaded, thus
625avoiding the need to maintain a private copy of the Python interpreter
626in order to use a private extension.
627
628\section{A Short Description of AIL and Amoeba}
629
630An RPC stub generator takes an interface description as input. The
631designer of a stub generator has at least two choices for the input
632language: use a suitably restricted version of the target language, or
633design a new language. The first solution was chosen, for instance,
634by the designers of Flume, the stub generator for the Topaz
635distributed operating system built at DEC SRC
636\cite{Flume,Evolving}.
637
638Flume's one and only target language is Modula-2+ (the predecessor of
639Modula-3). Modula-2+, like Modula-N for any N, has an interface
640syntax that is well suited as a stub generator input language: an
641interface module declares the functions that are `exported' by a
642module implementation, with their parameter and return types, plus the
643types and constants used for the parameters. Therefore, the input to
644Flume is simply a Modula-2+ interface module. But even in this ideal
645situation, an RPC stub generator needs to know things about functions
646that are not stated explicitly in the interface module: for instance,
647the transfer direction of VAR parameters (IN, OUT or both) is not
648given. Flume solves this and other problems by a mixture of
649directives hidden in comments and a convention for the names of
650objects. Thus, one could say that the designers of Flume really
651created a new language, even though it looks remarkably like their
652target language.
653
654\subsection{The AIL Input Language}
655
656Amoeba uses C as its primary programming language. C function
657declarations (at least in `Classic' C) don't specify the types of
658the parameters, let alone their transfer direction. Using this as
659input for a stub generator would require almost all information for
660the stub generator to be hidden inside comments, which would require a
661rather contorted scanner. Therefore we decided to design the input
662syntax for Amoeba's stub generator `from scratch'. This gave us the
663liberty to invent proper syntax not only for the transfer direction of
664parameters, but also for variable-length arrays.
665
666On the other hand we decided not to abuse our freedom, and borrowed as
667much from C as we could. For instance, AIL runs its input through the
668C preprocessor, so we get macros, include files and conditional
669compilation for free. AIL's type declaration syntax is a superset of
670C's, so the user can include C header files to use the types declared
671there as function parameter types --- which are declared using
Guido van Rossum16d6e711994-08-08 12:30:22 +0000672function prototypes as in \Cpp{} or Standard C\@. It should be clear by
Guido van Rossum2bbb3c01992-02-11 15:52:24 +0000673now that AIL's lexical conventions are also identical to C's. The
674same is true for its expression syntax.
675
676Where does AIL differ from C, then? Function declarations in AIL are
677grouped in {\em classes}. Classes in AIL are mostly intended as a
678grouping mechanism: all functions implemented by a server are grouped
679together in a class. Inheritance is used to form new groups by adding
680elements to existing groups; multiple inheritance is supported to join
681groups together. Classes can also contain constant and type
682definitions, and one form of output that AIL can generate is a header
683file for use by C programmers who wish to use functions from a
684particular AIL class.
685
686Let's have a look at some (unrealistically simple) class definitions:
687\begin{verbatim}
688#include <amoeba.h> /* Defines `capability', etc. */
689
690class standard_ops [1000 .. 1999] {
691 /* Operations supported by most interfaces */
692 std_info(*, out char buf[size:100], out int size);
693 std_destroy(*);
694};
695\end{verbatim}
696This defines a class called `standard\_ops' whose request codes are
697chosen by AIL from the range 1000-1999. Request codes are small
698integers used to identify remote operations. The author of the class
699must specify a range from which AIL chooses, and class authors must
700make sure they avoid conflicts, e.g. by using an `assigned number
701administration office'. In the example, `std\_info' will be assigned
702request code 1000 and `std\_destroy' will get code 1001. There is
703also an option to explicitly assign request codes, for compatibility
704with servers with manually written interfaces.
705
706The class `standard\_ops' defines two operations, `std\_info' and
707`std\_destroy'. The first parameter of each operation is a star
708(`*'); this is a placeholder for a capability that must be passed when
709the operation is called. The description of Amoeba below explains the
710meaning and usage of capabilities; for now, it is sufficient to know
711that a capability is a small structure that uniquely identifies an
712object and a server or service.
713
714The standard operation `std\_info' has two output parameters: a
715variable-size character buffer (which will be filled with a short
716descriptive string of the object to which the operation is applied)
717and an integer giving the length of this string. The standard
718operation `std\_destroy' has no further parameters --- it just
719destroys the object, if the caller has the right to do so.
720
721The next class is called `tty':
722\begin{verbatim}
723class tty [2000 .. 2099] {
724 inherit standard_ops;
725 const TTY_MAXBUF = 1000;
726 tty_write(*, char buf[size:TTY_MAXBUF], int size);
727 tty_read(*, out char buf[size:TTY_MAXBUF], out int size);
728};
729\end{verbatim}
730The request codes for operations defined in this class lie in the
731range 2000-2099; inherited operations use the request codes already
732assigned to them. The operations defined by this class are
733`tty\_read' and `tty\_write', which pass variable-sized data buffers
734between client and server. Class `tty' inherits class
735`standard\_ops', so tty objects also support the operations
736`std\_info' and `std\_destroy'.
737
738Only the {\em interface} for `std\_info' and `std\_destroy' is shared
739between tty objects and other objects whose interface inherits
740`standard\_ops'; the implementation may differ. Even multiple
741implementations of the `tty' interface may exist, e.g. a driver for a
742console terminal and a terminal emulator in a window. To expand on
743the latter example, consider:
744\begin{verbatim}
745class window [2100 .. 2199] {
746 inherit standard_ops;
747 win_create(*, int x, int y, int width, int height,
748 out capability win_cap);
749 win_reconfigure(*, int x, int y, int width, int height);
750};
751
752class tty_emulator [2200 .. 2299] {
753 inherit tty, window;
754};
755\end{verbatim}
756Here two new interface classes are defined.
757Class `window' could be used for creating and manipulating windows.
758Note that `win\_create' returns a capability for the new window.
759This request should probably should be sent to a generic window
760server capability, or it might create a subwindow when applied to a
761window object.
762
763Class `tty\_emulator' demonstrates the essence of multiple inheritance.
764It is presumably the interface to a window-based terminal emulator.
765Inheritance is transitive, so `tty\_emulator' also implicitly inherits
766`standard\_ops'.
767In fact, it inherits it twice: once via `tty' and once via `window'.
768Since AIL class inheritance only means interface sharing, not
769implementation sharing, inheriting the same class multiple times is
770never a problem and has the same effect as inheriting it once.
771
Guido van Rossum16d6e711994-08-08 12:30:22 +0000772Note that the power of AIL classes doesn't go as far as \Cpp{}.
Guido van Rossum2bbb3c01992-02-11 15:52:24 +0000773AIL classes cannot have data members, and there is
774no mechanism for a server that implements a derived class
775to inherit the implementation of the base
776class --- other than copying the source code.
777The syntax for class definitions and inheritance is also different.
778
779\subsection{Amoeba}
780
781The smell of `object-orientedness' that the use of classes in AIL
782creates matches nicely with Amoeba's object-oriented approach to
783RPC\@. In Amoeba, almost all operating system entities (files,
784directories, processes, devices etc.) are implemented as {\em
785objects}. Objects are managed by {\em services} and represented by
786{\em capabilities}. A capability gives its holder access to the
787object it represents. Capabilities are protected cryptographically
788against forgery and can thus be kept in user space. A capability is a
789128-bit binary string, subdivided as follows:
790
791% XXX Need a better version of this picture!
792\begin{verbatim}
793 48 24 8 48 Bits
794+----------------+------------+--------+---------------+
795| Service | Object | Perm. | Check |
796| port | number | bits | word |
797+----------------+------------+--------+---------------+
798\end{verbatim}
799
800The service port is used by the RPC implementation in the Amoeba
801kernel to locate a server implementing the service that manages the
802object. In many cases there is a one-to-one correspondence between
803servers and services (each service is implemented by exactly one
804server process), but some services are replicated. For instance,
805Amoeba's directory service, which is crucial for gaining access to most
806other services, is implemented by two servers that listen on the same
807port and know about exactly the same objects.
808
809The object number in the capability is used by the server receiving
810the request for identifying the object to which the operation applies.
811The permission bits specify which operations the holder of the capability
812may apply. The last part of a capability is a 48-bit long `check
813word', which is used to prevent forgery. The check word is computed
814by the server based upon the permission bits and a random key per object
815that it keeps secret. If you change the permission bits you must compute
816the proper check word or else the server will refuse the capability.
817Due to the size of the check word and the nature of the cryptographic
818`one-way function' used to compute it, inverting this function is
819impractical, so forging capabilities is impossible.%
820\footnote{
821As computers become faster, inverting the one-way function becomes
822less impractical.
823Therefore, a next version of Amoeba will have 64-bit check words.
824}
825
826A working Amoeba system is a collection of diverse servers, managing
827files, directories, processes, devices etc. While most servers have
828their own interface, there are some requests that make sense for some
829or all object types. For instance, the {\em std\_info()} request,
830which returns a short descriptive string, applies to all object types.
831Likewise, {\em std\_destroy()} applies to files, directories and
832processes, but not to devices.
833
834Similarly, different file server implementations may want to offer the
835same interface for operations like {\em read()} and {\em write()} to
836their clients. AIL's grouping of requests into classes is ideally
837suited to describe this kind of interface sharing, and a class
838hierarchy results which clearly shows the similarities between server
839interfaces (not necessarily their implementations!).
840
841The base class of all classes defines the {\em std\_info()} request.
842Most server interfaces actually inherit a derived class that also
843defines {\em std\_destroy().} File servers inherit a class that
844defines the common operations on files, etc.
845
846\subsection{How AIL Works}
847
848The AIL stub generator functions in three phases:
849\begin{itemize}
850\item
851parsing,
852\item
853strategy determination,
854\item
855code generation.
856\end{itemize}
857
858{\bf Phase one} parses the input and builds a symbol table containing
859everything it knows about the classes and other definitions found in
860the input.
861
862{\bf Phase two} determines the strategy to use for each function
863declaration in turn and decides upon the request and reply message
864formats. This is not a simple matter, because of various optimization
865attempts. Amoeba's kernel interface for RPC requests takes a
866fixed-size header and one arbitrary-size buffer. A large part of the
867header holds the capability of the object to which the request is
868directed, but there is some space left for a few integer parameters
869whose interpretation is left up to the server. AIL tries to use these
870slots for simple integer parameters, for two reasons.
871
872First, unlike the buffer, header fields are byte-swapped by the RPC
873layer in the kernel if necessary, so it saves a few byte swapping
874instructions in the user code. Second, and more important, a common
875form of request transfers a few integers and one large buffer to or
876from a server. The {\em read()} and {\em write()} requests of most
877file servers have this form, for instance. If it is possible to place
878all integer parameters in the header, the address of the buffer
879parameter can be passed directly to the kernel RPC layer. While AIL
880is perfectly capable of handling requests that do not fit this format,
881the resulting code involves allocating a new buffer and copying all
882parameters into it. It is a top priority to avoid this copying
883(`marshalling') if at all possible, in order to maintain Amoeba's
884famous RPC performance.
885
886When AIL resorts to copying parameters into a buffer, it reorders them
887so that integers indicating the lengths of variable-size arrays are
888placed in the buffer before the arrays they describe, since otherwise
889decoding the request would be impossible. It also adds occasional
890padding bytes to ensure integers are aligned properly in the buffer ---
891this can speed up (un)marshalling.
892
893{\bf Phase three} is the code generator, or back-end. There are in
894fact many different back-ends that may be called in a single run to
895generate different types of output. The most important output types
896are header files (for inclusion by the clients of an interface),
897client stubs, and `server main loop' code. The latter decodes
898incoming requests in the server. The generated code depends on the
899programming language requested, and there are separate back-ends for
900each supported language.
901
902It is important that the strategy chosen by phase two is independent
903of the language requested for phase three --- otherwise the
904interoperability of servers and clients written in different languages
905would be compromised.
906
907\section{Linking AIL to Python}
908
909From the previous section it can be concluded that linking AIL to
910Python is a matter of writing a back-end for Python. This is indeed
911what we did.
912
913Considerable time went into the design of the back-end in order to
914make the resulting RPC interface for Python fit as smoothly as
915possible in Python's programming style. For instance, the issues of
916parameter transfer, variable-size arrays, error handling, and call
917syntax were all solved in a manner that favors ease of use in Python
918rather than strict correspondence with the stubs generated for C,
919without compromising network-level compatibility.
920
921\subsection{Mapping AIL Entities to Python}
922
923For each programming language that AIL is to support, a mapping must
924be designed between the data types in AIL and those in that language.
925Other aspects of the programming languages, such as differences in
926function call semantics, must also be taken care of.
927
928While the mapping for C is mostly straightforward, the mapping for
929Python requires a little thinking to get the best results for Python
930programmers.
931
932\subsubsection{Parameter Transfer Direction}
933
934Perhaps the simplest issue is that of parameter transfer direction.
935Parameters of functions declared in AIL are categorized as being of
936type {\tt in}, {\tt out} or {\tt in} {\tt out} (the same distinction
937as made in Ada). Python only has call-by-value parameter semantics;
938functions can return multiple values as a tuple. This means that,
939unlike the C back-end, the Python back-end cannot always generate
940Python functions with exactly the same parameter list as the AIL
941functions.
942
943Instead, the Python parameter list consists of all {\tt in} and {\tt
944in} {\tt out} parameters, in the order in which they occur in the AIL
945parameter list; similarly, the Python function returns a tuple
946containing all {\tt in} {\tt out} and {\tt out} parameters. In fact
947Python packs function parameters into a tuple as well, stressing the
948symmetry between parameters and return value. For example, a stub
949with this AIL parameter list:
950\begin{verbatim}
951(*, in int p1, in out int p2, in int p3, out int p4)
952\end{verbatim}
953will have the following parameter list and return values in Python:
954\begin{verbatim}
955(p1, p2, p3) -> (p2, p4)
956\end{verbatim}
957
958\subsubsection{Variable-size Entities}
959
960The support for variable-size objects in AIL is strongly guided by the
961limitations of C in this matter. Basically, AIL allows what is
962feasible in C: functions may have variable-size arrays as parameters
963(both input or output), provided their length is passed separately.
964In practice this is narrowed to the following rule: for each
965variable-size array parameter, there must be an integer parameter
966giving its length. (An exception for null-terminated strings is
967planned but not yet realized.)
968
969Variable-size arrays in AIL or C correspond to {\em sequences} in
970Python: lists, tuples or strings. These are much easier to use than
971their C counterparts. Given a sequence object in Python, it is always
972possible to determine its size: the built-in function {\tt len()}
973returns it. It would be annoying to require the caller of an RPC stub
974with a variable-size parameter to also pass a parameter that
975explicitly gives its size. Therefore we eliminate all parameters from
976the Python parameter list whose value is used as the size of a
977variable-size array. Such parameters are easily found: the array
978bound expression contains the name of the parameter giving its size.
979This requires the stub code to work harder (it has to recover the
980value for size parameters from the corresponding sequence parameter),
981but at least part of this work would otherwise be needed as well, to
982check that the given and actual sizes match.
983
984Because of the symmetry in Python between the parameter list and the
985return value of a function, the same elimination is performed on
986return values containing variable-size arrays: integers returned
987solely to tell the client the size of a returned array are not
988returned explicitly to the caller in Python.
989
990\subsubsection{Error Handling}
991
992Another point where Python is really better than C is the issue of
993error handling. It is a fact of life that everything involving RPC
994may fail, for a variety of reasons outside the user's control: the
995network may be disconnected, the server may be down, etc. Clients
996must be prepared to handle such failures and recover from them, or at
997least print an error message and die. In C this means that every
998function returns an error status that must be checked by the caller,
999causing programs to be cluttered with error checks --- or worse,
1000programs that ignore errors and carry on working with garbage data.
1001
1002In Python, errors are generally indicated by exceptions, which can be
1003handled out of line from the main flow of control if necessary, and
1004cause immediate program termination (with a stack trace) if ignored.
1005To profit from this feature, all RPC errors that may be encountered by
1006AIL-generated stubs in Python are turned into exceptions. An extra
1007value passed together with the exception is used to relay the error
1008code returned by the server to the handler. Since in general RPC
1009failures are rare, Python test programs can usually ignore exceptions
1010--- making the program simpler --- without the risk of occasional
Guido van Rossum16d6e711994-08-08 12:30:22 +00001011errors going undetected. (I still remember the embarrassment of a
Guido van Rossum2bbb3c01992-02-11 15:52:24 +00001012hundredfold speed improvement reported, long, long, ago, about a new
1013version of a certain program, which later had to be attributed to a
1014benchmark that silently dumped core...)
1015
1016\subsubsection{Function Call Syntax}
1017
1018Amoeba RPC operations always need a capability parameter (this is what
1019the `*' in the AIL function templates stands for); the service is
1020identified by the port field of the capability. In C, the capability
1021must always be the first parameter of the stub function, but in Python
1022we can do better.
1023
1024A Python capability is an opaque object type in its own right, which
1025is used, for instance, as parameter to and return value from Amoeba's
1026name server functions. Python objects can have methods, so it is
1027convenient to make all AIL-generated stubs methods of capabilities
1028instead of just functions. Therefore, instead of writing
1029\begin{verbatim}
1030some_stub(cap, other_parameters)
1031\end{verbatim}
1032as in C, Python programmers can write
1033\begin{verbatim}
1034cap.some_stub(other_parameters)
1035\end{verbatim}
1036This is better because it reduces name conflicts: in Python, no
1037confusion is possible between a stub and a local or global variable or
1038user-defined function with the same name.
1039
1040\subsubsection{Example}
1041
1042All the preceding principles can be seen at work in the following
1043example. Suppose a function is declared in AIL as follows:
1044\begin{verbatim}
1045some_stub(*, in char buf[size:1000], in int size,
1046 out int n_done, out int status);
1047\end{verbatim}
1048In C it might be called by the following code (including declarations,
1049for clarity, but not initializations):
1050\begin{verbatim}
1051int err, n_done, status;
1052capability cap;
1053char buf[500];
1054...
1055err = some_stub(&cap, buf, sizeof buf, &n_done, &status);
1056if (err != 0) return err;
1057printf("%d done; status = %d\n", n_done, status);
1058\end{verbatim}
1059Equivalent code in Python might be the following:
1060\begin{verbatim}
1061cap = ...
1062buf = ...
1063n_done, status = cap.some_stub(buf)
1064print n_done, 'done;', 'status =', status
1065\end{verbatim}
1066No explicit error check is required in Python: if the RPC fails, an
1067exception is raised so the {\tt print} statement is never reached.
1068
1069\subsection{The Implementation}
1070
1071More or less orthogonal to the issue of how to map AIL operations to
1072the Python language is the question of how they should be implemented.
1073
1074In principle it would be possible to use the same strategy that is
1075used for C: add an interface to Amoeba's low-level RPC primitives to
1076Python and generate Python code to marshal parameters into and out of
1077a buffer. However, Python's high-level data types are not well suited
1078for marshalling: byte-level operations are clumsy and expensive, with
1079the result that marshalling a single byte of data can take several
1080Python statements. This would mean that a large amount of code would
1081be needed to implement a stub, which would cost a lot of time to parse
1082and take up a lot of space in `compiled' form (as parse tree or pseudo
1083code). Execution of the marshalling code would be sluggish as well.
1084
1085We therefore chose an alternate approach, writing the marshalling in
1086C, which is efficient at such byte-level operations. While it is easy
1087enough to generate C code that can be linked with the Python
1088interpreter, it would obviously not stimulate the use of Python for
1089server testing if each change to an interface required relinking the
1090interpreter (dynamic loading of C code is not yet available on
1091Amoeba). This is circumvented by the following solution: the
1092marshalling is handled by a simple {\em virtual machine}, and AIL
1093generates instructions for this machine. An interpreter for the
1094machine is linked into the Python interpreter and reads its
1095instructions from a file written by AIL.
1096
1097The machine language for our virtual machine is dubbed {\em Stubcode}.
1098Stubcode is a super-specialized language. There are two sets of of
1099about a dozen instructions each: one set marshals Python objects
1100representing parameters into a buffer, the other set (similar but not
1101quite symmetric) unmarshals results from a buffer into Python objects.
1102The Stubcode interpreter uses a stack to hold Python intermediate
1103results. Other state elements are an Amoeba header and buffer, a
1104pointer indicating the current position in the buffer, and of course a
1105program counter. Besides (un)marshalling, the virtual machine must
1106also implement type checking, and raise a Python exception when a
1107parameter does not have the expected type.
1108
1109The Stubcode interpreter marshals Python data types very efficiently,
1110since each instruction can marshal a large amount of data. For
1111instance, a whole Python string is marshalled by a single Stubcode
1112instruction, which (after some checking) executes the most efficient
1113byte-copying loop possible --- it calls {\tt memcpy()}.
1114
1115
1116Construction details of the Stubcode interpreter are straightforward.
1117Most complications are caused by the peculiarities of AIL's strategy
1118module and Python's type system. By far the most complex single
1119instruction is the `loop' instruction, which is used to marshal
1120arrays.
1121
1122As an example, here is the complete Stubcode program (with spaces and
1123comments added for clarity) generated for the function {\tt
1124some\_stub()} of the example above. The stack contains pointers to
1125Python objects, and its initial contents is the parameter to the
1126function, the string {\tt buf}. The final stack contents will be the
1127function return value, the tuple {\tt (n\_done, status)}. The name
1128{\tt header} refers to the fixed size Amoeba RPC header structure.
1129\vspace{1em}
1130
1131{\tt
1132\begin{tabular}{l l l}
1133BufSize & 1000 & {\em Allocate RPC buffer of 1000 bytes} \\
1134Dup & 1 & {\em Duplicate stack top} \\
1135StringS & & {\em Replace stack top by its string size} \\
1136PutI & h\_extra int32 & {\em Store top element in }header.h\_extra \\
1137TStringSlt & 1000 & {\em Assert string size less than 1000} \\
1138PutVS & & {\em Marshal variable-size string} \\
1139 & & \\
1140Trans & 1234 & {\em Execute the RPC (request code 1234)} \\
1141 & & \\
1142GetI & h\_extra int32 & {\em Push integer from} header.h\_extra \\
1143GetI & h\_size int32 & {\em Push integer from} header.h\_size \\
1144Pack & 2 & {\em Pack top 2 elements into a tuple} \\
1145\end{tabular}
1146}
1147\vspace{1em}
1148
1149As much work as possible is done by the Python back-end in AIL, rather
1150than in the Stubcode interpreter, to make the latter both simple and
1151fast. For instance, the decision to eliminate an array size parameter
1152from the Python parameter list is taken by AIL, and Stubcode
1153instructions are generated to recover the size from the actual
1154parameter and to marshal it properly. Similarly, there is a special
1155alignment instruction (not used in the example) to meet alignment
1156requirements.
1157
1158Communication between AIL and the Stubcode generator is via the file
1159system. For each stub function, AIL creates a file in its output
1160directory, named after the stub with a specific suffix. This file
1161contains a machine-readable version of the Stubcode program for the
1162stub. The Python user can specify a search path containing
1163directories which the interpreter searches for a Stubcode file the
1164first time the definition for a particular stub is needed.
1165
1166The transformations on the parameter list and data types needed to map
1167AIL data types to Python data types make it necessary to help the
1168Python programmer a bit in figuring out the parameters to a call.
1169Although in most cases the rules are simple enough, it is sometimes
1170hard to figure out exactly what the parameter and return values of a
1171particular stub are. There are two sources of help in this case:
1172first, the exception contains enough information so that the user can
1173figure what type was expected; second, AIL's Python back-end
1174optionally generates a human-readable `interface specification' file.
1175
1176\section{Conclusion}
1177
1178We have succeeded in creating a useful extension to Python that
1179enables Amoeba server writers to test and experiment with their server
1180in a much more interactive manner. We hope that this facility will
1181add to the popularity of AIL amongst Amoeba programmers.
1182
1183Python's extensibility was proven convincingly by the exercise
1184(performed by the second author) of adding the Stubcode interpreter to
1185Python. Standard data abstraction techniques are used to insulate
1186extension modules from details of the rest of the Python interpreter.
1187In the case of the Stubcode interpreter this worked well enough that
1188it survived a major overhaul of the main Python interpreter virtually
1189unchanged.
1190
1191On the other hand, adding a new back-end to AIL turned out to be quite
1192a bit of work. One problem, specific to Python, was to be expected:
1193Python's variable-size data types differ considerably from the
1194C-derived data model that AIL favors. Two additional problems we
1195encountered were the complexity of the interface between AIL's second
1196and third phases, and a number of remaining bugs in the second phase
1197that surfaced when the implementation of the Python back-end was
1198tested. The bugs have been tracked down and fixed, but nothing
1199has been done about the complexity of the interface.
1200
1201\subsection{Future Plans}
1202
1203AIL's C back-end generates server main loop code as well as client
1204stubs. The Python back-end currently only generates client stubs, so
1205it is not yet possible to write servers in Python. While it is
1206clearly more important to be able to use Python as a client than as a
1207server, the ability to write server prototypes in Python would be a
1208valuable addition: it allows server designers to experiment with
1209interfaces in a much earlier stage of the design, with a much smaller
1210programming effort. This makes it possible to concentrate on concepts
1211first, before worrying about efficient implementation.
1212
1213The unmarshalling done in the server is almost symmetric with the
1214marshalling in the client, and vice versa, so relative small
1215extensions to the Stubcode virtual machine will allow its use in a
1216server main loop. We hope to find the time to add this feature to a
1217future version of Python.
1218
1219\section{Availability}
1220
1221The Python source distribution is available to Internet users by
1222anonymous ftp to site {\tt ftp.cwi.nl} [IP address 192.16.184.180]
1223from directory {\tt /pub}, file name {\tt python*.tar.Z} (where the
1224{\tt *} stands for a version number). This is a compressed UNIX tar
1225file containing the C source and \LaTeX documentation for the Python
1226interpreter. It includes the Python library modules and the {\em
1227Stubcode} interpreter, as well as many example Python programs. Total
1228disk space occupied by the distribution is about 3 Mb; compilation
1229requires 1-3 Mb depending on the configuration built, the compile
1230options, etc.
1231
1232\bibliographystyle{plain}
1233
1234\bibliography{quabib}
1235
1236\end{document}