skylark: byte code compiler and interpreter (#95)
A byte code is a more efficient representation of the program for
execution, as it is more compact than an AST, and it is readily
serializable, permitting the work of parsing, resolving, and
compilation to be amortized across runs when the byte code is cached
in the file system.
This change defines a simple byte code, compiler, and interpreter for
Skylark, using ideas from Python and Emacs Lisp.
It also deletes the tree-based evaluator. I evaluated (ha!) keeping
both and switching between them dynamically, compiling the program
just before the 2nd or nth invocation of a function. (In a build
system, a great many "functions" are executed exactly once, such as
the toplevel of a BUILD file.) However, the performance gain over
always compiling was barely measurable and did not justify the cost of
maintaining two different interpreters. Also, just-in-time compilation
makes program dynamics very complex.
Details:
- the toplevel code of a module is now represented as a function
of no parameters, enabling various simplifications.
- Every Frame now has a Function. Backtracing is simpler.
Only the leaf frame (the error) has a position;
all the parent frames record only the program counter of
the call instruction, and compute the position from this
as needed.
- To keep the line number table encoding simple and compact,
EvalError no longer reports column information.
It would be nice to add it back with a carefully tuned encoding.
Breaking API changes:
- Thread.{Push,Pop} are gone.
The reason they were initially exposed is exemplified by the
TestRepeatedExec test case, which is better served by the new API:
var predeclared skylark.StringDict
// parse and compile
_, prog, err = SourceProgram("foo.sky", src, predeclared.Has)
// "in between" stuff goes here
// execute toplevel
globals, err = prog.Init(thread, predeclared)
- ExecOptions.BeforeExec is gone, and thus so is ExecOptions.
This function existed so that clients could "get in between"
producing a program and executing it, but the new low-level API
(see above) above separates these operations explicitly.
Now that Exec and ExecOptions are no more expressive than ExecFile,
they have been deleted.
Thanks to Jay Conrod for earlier rounds of review,
including substantial simplifications.diff --git a/syntax/syntax.go b/syntax/syntax.go
index 3480c66..99548b1 100644
--- a/syntax/syntax.go
+++ b/syntax/syntax.go
@@ -298,7 +298,7 @@
commentsRef
Fn Expr
Lparen Position
- Args []Expr
+ Args []Expr // arg = expr | ident=expr | *expr | **expr
Rparen Position
}