Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 1 | r"""Utilities to compile possibly incomplete Python source code. |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 2 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 3 | This module provides two interfaces, broadly similar to the builtin |
| 4 | function compile(), that take progam text, a filename and a 'mode' |
| 5 | and: |
Skip Montanaro | e99d5ea | 2001-01-20 19:54:20 +0000 | [diff] [blame] | 6 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 7 | - Return a code object if the command is complete and valid |
| 8 | - Return None if the command is incomplete |
| 9 | - Raise SyntaxError, ValueError or OverflowError if the command is a |
| 10 | syntax error (OverflowError and ValueError can be produced by |
| 11 | malformed literals). |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 12 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 13 | Approach: |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 14 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 15 | First, check if the source consists entirely of blank lines and |
| 16 | comments; if so, replace it with 'pass', because the built-in |
| 17 | parser doesn't always do the right thing for these. |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 18 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 19 | Compile three times: as is, with \n, and with \n\n appended. If it |
| 20 | compiles as is, it's complete. If it compiles with one \n appended, |
| 21 | we expect more. If it doesn't compile either way, we compare the |
| 22 | error we get when compiling with \n or \n\n appended. If the errors |
| 23 | are the same, the code is broken. But if the errors are different, we |
| 24 | expect more. Not intuitive; not even guaranteed to hold in future |
| 25 | releases; but this matches the compiler's behavior from Python 1.4 |
| 26 | through 2.2, at least. |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 27 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 28 | Caveat: |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 29 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 30 | It is possible (but not likely) that the parser stops parsing with a |
| 31 | successful outcome before reaching the end of the source; in this |
| 32 | case, trailing symbols may be ignored instead of causing an error. |
| 33 | For example, a backslash followed by two newlines may be followed by |
| 34 | arbitrary garbage. This will be fixed once the API for the parser is |
| 35 | better. |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 36 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 37 | The two interfaces are: |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 38 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 39 | compile_command(source, filename, symbol): |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 40 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 41 | Compiles a single command in the manner described above. |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 42 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 43 | CommandCompiler(): |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 44 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 45 | Instances of this class have __call__ methods identical in |
| 46 | signature to compile_command; the difference is that if the |
| 47 | instance compiles program text containing a __future__ statement, |
| 48 | the instance 'remembers' and compiles all subsequent program texts |
| 49 | with the statement in force. |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 50 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 51 | The module also provides another class: |
| 52 | |
| 53 | Compile(): |
| 54 | |
| 55 | Instances of this class act like the built-in function compile, |
| 56 | but with 'memory' in the sense described above. |
| 57 | """ |
| 58 | |
| 59 | import __future__ |
| 60 | |
| 61 | _features = [getattr(__future__, fname) |
| 62 | for fname in __future__.all_feature_names] |
| 63 | |
| 64 | __all__ = ["compile_command", "Compile", "CommandCompiler"] |
| 65 | |
Guido van Rossum | 4b499dd3 | 2003-02-13 22:07:59 +0000 | [diff] [blame] | 66 | PyCF_DONT_IMPLY_DEDENT = 0x200 # Matches pythonrun.h |
| 67 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 68 | def _maybe_compile(compiler, source, filename, symbol): |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 69 | # Check for source consisting of only blank lines and comments |
Eric S. Raymond | 6b71e74 | 2001-02-09 08:56:30 +0000 | [diff] [blame] | 70 | for line in source.split("\n"): |
| 71 | line = line.strip() |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 72 | if line and line[0] != '#': |
| 73 | break # Leave it alone |
| 74 | else: |
| 75 | source = "pass" # Replace it with a 'pass' statement |
| 76 | |
| 77 | err = err1 = err2 = None |
| 78 | code = code1 = code2 = None |
| 79 | |
| 80 | try: |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 81 | code = compiler(source, filename, symbol) |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 82 | except SyntaxError, err: |
| 83 | pass |
| 84 | |
| 85 | try: |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 86 | code1 = compiler(source + "\n", filename, symbol) |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 87 | except SyntaxError, err1: |
| 88 | pass |
| 89 | |
| 90 | try: |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 91 | code2 = compiler(source + "\n\n", filename, symbol) |
Guido van Rossum | c41c1a9 | 1998-10-22 21:56:15 +0000 | [diff] [blame] | 92 | except SyntaxError, err2: |
| 93 | pass |
| 94 | |
| 95 | if code: |
| 96 | return code |
| 97 | try: |
| 98 | e1 = err1.__dict__ |
| 99 | except AttributeError: |
| 100 | e1 = err1 |
| 101 | try: |
| 102 | e2 = err2.__dict__ |
| 103 | except AttributeError: |
| 104 | e2 = err2 |
| 105 | if not code1 and e1 == e2: |
| 106 | raise SyntaxError, err1 |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 107 | |
Guido van Rossum | 4b499dd3 | 2003-02-13 22:07:59 +0000 | [diff] [blame] | 108 | def _compile(source, filename, symbol): |
| 109 | return compile(source, filename, symbol, PyCF_DONT_IMPLY_DEDENT) |
| 110 | |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 111 | def compile_command(source, filename="<input>", symbol="single"): |
| 112 | r"""Compile a command and determine whether it is incomplete. |
| 113 | |
| 114 | Arguments: |
| 115 | |
| 116 | source -- the source string; may contain \n characters |
| 117 | filename -- optional filename from which source was read; default |
| 118 | "<input>" |
| 119 | symbol -- optional grammar start symbol; "single" (default) or "eval" |
| 120 | |
| 121 | Return value / exceptions raised: |
| 122 | |
| 123 | - Return a code object if the command is complete and valid |
| 124 | - Return None if the command is incomplete |
| 125 | - Raise SyntaxError, ValueError or OverflowError if the command is a |
| 126 | syntax error (OverflowError and ValueError can be produced by |
| 127 | malformed literals). |
| 128 | """ |
Guido van Rossum | 4b499dd3 | 2003-02-13 22:07:59 +0000 | [diff] [blame] | 129 | return _maybe_compile(_compile, source, filename, symbol) |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 130 | |
| 131 | class Compile: |
| 132 | """Instances of this class behave much like the built-in compile |
| 133 | function, but if one is used to compile text containing a future |
| 134 | statement, it "remembers" and compiles all subsequent program texts |
| 135 | with the statement in force.""" |
| 136 | def __init__(self): |
Guido van Rossum | 4b499dd3 | 2003-02-13 22:07:59 +0000 | [diff] [blame] | 137 | self.flags = PyCF_DONT_IMPLY_DEDENT |
Tim Peters | 6cd6a82 | 2001-08-17 22:11:27 +0000 | [diff] [blame] | 138 | |
| 139 | def __call__(self, source, filename, symbol): |
| 140 | codeob = compile(source, filename, symbol, self.flags, 1) |
| 141 | for feature in _features: |
| 142 | if codeob.co_flags & feature.compiler_flag: |
| 143 | self.flags |= feature.compiler_flag |
| 144 | return codeob |
| 145 | |
| 146 | class CommandCompiler: |
| 147 | """Instances of this class have __call__ methods identical in |
| 148 | signature to compile_command; the difference is that if the |
| 149 | instance compiles program text containing a __future__ statement, |
| 150 | the instance 'remembers' and compiles all subsequent program texts |
| 151 | with the statement in force.""" |
| 152 | |
| 153 | def __init__(self,): |
| 154 | self.compiler = Compile() |
| 155 | |
| 156 | def __call__(self, source, filename="<input>", symbol="single"): |
| 157 | r"""Compile a command and determine whether it is incomplete. |
| 158 | |
| 159 | Arguments: |
| 160 | |
| 161 | source -- the source string; may contain \n characters |
| 162 | filename -- optional filename from which source was read; |
| 163 | default "<input>" |
| 164 | symbol -- optional grammar start symbol; "single" (default) or |
| 165 | "eval" |
| 166 | |
| 167 | Return value / exceptions raised: |
| 168 | |
| 169 | - Return a code object if the command is complete and valid |
| 170 | - Return None if the command is incomplete |
| 171 | - Raise SyntaxError, ValueError or OverflowError if the command is a |
| 172 | syntax error (OverflowError and ValueError can be produced by |
| 173 | malformed literals). |
| 174 | """ |
| 175 | return _maybe_compile(self.compiler, source, filename, symbol) |