bpo-17611. Move unwinding of stack for "pseudo exceptions" from interpreter to compiler. (GH-5006)
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Antoine Pitrou <antoine@python.org>
diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst
index 535b36e..aa66128 100644
--- a/Doc/library/dis.rst
+++ b/Doc/library/dis.rst
@@ -335,6 +335,14 @@
three.
+.. opcode:: ROT_FOUR
+
+ Lifts second, third and forth stack items one position up, moves top down
+ to position four.
+
+ .. versionadded:: 3.8
+
+
.. opcode:: DUP_TOP
Duplicates the reference on top of the stack.
@@ -605,17 +613,6 @@
is terminated with :opcode:`POP_TOP`.
-.. opcode:: BREAK_LOOP
-
- Terminates a loop due to a :keyword:`break` statement.
-
-
-.. opcode:: CONTINUE_LOOP (target)
-
- Continues a loop due to a :keyword:`continue` statement. *target* is the
- address to jump to (which should be a :opcode:`FOR_ITER` instruction).
-
-
.. opcode:: SET_ADD (i)
Calls ``set.add(TOS1[-i], TOS)``. Used to implement set comprehensions.
@@ -676,7 +673,7 @@
.. opcode:: POP_BLOCK
Removes one block from the block stack. Per frame, there is a stack of
- blocks, denoting nested loops, try statements, and such.
+ blocks, denoting :keyword:`try` statements, and such.
.. opcode:: POP_EXCEPT
@@ -687,11 +684,50 @@
popped values are used to restore the exception state.
+.. opcode:: POP_FINALLY (preserve_tos)
+
+ Cleans up the value stack and the block stack. If *preserve_tos* is not
+ ``0`` TOS first is popped from the stack and pushed on the stack after
+ perfoming other stack operations:
+
+ * If TOS is ``NULL`` or an integer (pushed by :opcode:`BEGIN_FINALLY`
+ or :opcode:`CALL_FINALLY`) it is popped from the stack.
+ * If TOS is an exception type (pushed when an exception has been raised)
+ 6 values are popped from the stack, the last three popped values are
+ used to restore the exception state. An exception handler block is
+ removed from the block stack.
+
+ It is similar to :opcode:`END_FINALLY`, but doesn't change the bytecode
+ counter nor raise an exception. Used for implementing :keyword:`break`
+ and :keyword:`return` in the :keyword:`finally` block.
+
+ .. versionadded:: 3.8
+
+
+.. opcode:: BEGIN_FINALLY
+
+ Pushes ``NULL`` onto the stack for using it in :opcode:`END_FINALLY`,
+ :opcode:`POP_FINALLY`, :opcode:`WITH_CLEANUP_START` and
+ :opcode:`WITH_CLEANUP_FINISH`. Starts the :keyword:`finally` block.
+
+ .. versionadded:: 3.8
+
+
.. opcode:: END_FINALLY
Terminates a :keyword:`finally` clause. The interpreter recalls whether the
- exception has to be re-raised, or whether the function returns, and continues
- with the outer-next block.
+ exception has to be re-raised or execution has to be continued depending on
+ the value of TOS.
+
+ * If TOS is ``NULL`` (pushed by :opcode:`BEGIN_FINALLY`) continue from
+ the next instruction. TOS is popped.
+ * If TOS is an integer (pushed by :opcode:`CALL_FINALLY`), sets the
+ bytecode counter to TOS. TOS is popped.
+ * If TOS is an exception type (pushed when an exception has been raised)
+ 6 values are popped from the stack, the first three popped values are
+ used to re-raise the exception and the last three popped values are used
+ to restore the exception state. An exception handler block is removed
+ from the block stack.
.. opcode:: LOAD_BUILD_CLASS
@@ -704,9 +740,9 @@
This opcode performs several operations before a with block starts. First,
it loads :meth:`~object.__exit__` from the context manager and pushes it onto
- the stack for later use by :opcode:`WITH_CLEANUP`. Then,
+ the stack for later use by :opcode:`WITH_CLEANUP_START`. Then,
:meth:`~object.__enter__` is called, and a finally block pointing to *delta*
- is pushed. Finally, the result of calling the enter method is pushed onto
+ is pushed. Finally, the result of calling the ``__enter__()`` method is pushed onto
the stack. The next opcode will either ignore it (:opcode:`POP_TOP`), or
store it in (a) variable(s) (:opcode:`STORE_FAST`, :opcode:`STORE_NAME`, or
:opcode:`UNPACK_SEQUENCE`).
@@ -716,30 +752,31 @@
.. opcode:: WITH_CLEANUP_START
- Cleans up the stack when a :keyword:`with` statement block exits. TOS is the
- context manager's :meth:`__exit__` bound method. Below TOS are 1--3 values
- indicating how/why the finally clause was entered:
+ Starts cleaning up the stack when a :keyword:`with` statement block exits.
- * SECOND = ``None``
- * (SECOND, THIRD) = (``WHY_{RETURN,CONTINUE}``), retval
- * SECOND = ``WHY_*``; no retval below it
- * (SECOND, THIRD, FOURTH) = exc_info()
+ At the top of the stack are either ``NULL`` (pushed by
+ :opcode:`BEGIN_FINALLY`) or 6 values pushed if an exception has been
+ raised in the with block. Below is the context manager's
+ :meth:`~object.__exit__` or :meth:`~object.__aexit__` bound method.
- In the last case, ``TOS(SECOND, THIRD, FOURTH)`` is called, otherwise
- ``TOS(None, None, None)``. Pushes SECOND and result of the call
- to the stack.
+ If TOS is ``NULL``, calls ``SECOND(None, None, None)``,
+ removes the function from the stack, leaving TOS, and pushes ``None``
+ to the stack. Otherwise calls ``SEVENTH(TOP, SECOND, THIRD)``,
+ shifts the bottom 3 values of the stack down, replaces the empty spot
+ with ``NULL`` and pushes TOS. Finally pushes the result of the call.
.. opcode:: WITH_CLEANUP_FINISH
- Pops exception type and result of 'exit' function call from the stack.
+ Finishes cleaning up the stack when a :keyword:`with` statement block exits.
- If the stack represents an exception, *and* the function call returns a
- 'true' value, this information is "zapped" and replaced with a single
- ``WHY_SILENCED`` to prevent :opcode:`END_FINALLY` from re-raising the
- exception. (But non-local gotos will still be resumed.)
+ TOS is result of ``__exit__()`` or ``__aexit__()`` function call pushed
+ by :opcode:`WITH_CLEANUP_START`. SECOND is ``None`` or an exception type
+ (pushed when an exception has been raised).
- .. XXX explain the WHY stuff!
+ Pops two values from the stack. If SECOND is not None and TOS is true
+ unwinds the EXCEPT_HANDLER block which was created when the exception
+ was caught and pushes ``NULL`` to the stack.
All of the following opcodes use their arguments.
@@ -987,22 +1024,19 @@
Loads the global named ``co_names[namei]`` onto the stack.
-.. opcode:: SETUP_LOOP (delta)
-
- Pushes a block for a loop onto the block stack. The block spans from the
- current instruction with a size of *delta* bytes.
-
-
-.. opcode:: SETUP_EXCEPT (delta)
-
- Pushes a try block from a try-except clause onto the block stack. *delta*
- points to the first except block.
-
-
.. opcode:: SETUP_FINALLY (delta)
- Pushes a try block from a try-except clause onto the block stack. *delta*
- points to the finally block.
+ Pushes a try block from a try-finally or try-except clause onto the block
+ stack. *delta* points to the finally block or the first except block.
+
+
+.. opcode:: CALL_FINALLY (delta)
+
+ Pushes the address of the next instruction onto the stack and increments
+ bytecode counter by *delta*. Used for calling the finally block as a
+ "subroutine".
+
+ .. versionadded:: 3.8
.. opcode:: LOAD_FAST (var_num)
diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst
index 4181981..1d722ef 100644
--- a/Doc/whatsnew/3.8.rst
+++ b/Doc/whatsnew/3.8.rst
@@ -137,3 +137,21 @@
:func:`dbm.dumb.open` with flags ``'r'`` and ``'w'`` no longer creates
a database if it does not exist.
(Contributed by Serhiy Storchaka in :issue:`32749`.)
+
+
+CPython bytecode changes
+------------------------
+
+* The interpreter loop has been simplified by moving the logic of unrolling
+ the stack of blocks into the compiler. The compiler emits now explicit
+ instructions for adjusting the stack of values and calling the cleaning
+ up code for :keyword:`break`, :keyword:`continue` and :keyword:`return`.
+
+ Removed opcodes :opcode:`BREAK_LOOP`, :opcode:`CONTINUE_LOOP`,
+ :opcode:`SETUP_LOOP` and :opcode:`SETUP_EXCEPT`. Added new opcodes
+ :opcode:`ROT_FOUR`, :opcode:`BEGIN_FINALLY`, :opcode:`CALL_FINALLY` and
+ :opcode:`POP_FINALLY`. Changed the behavior of :opcode:`END_FINALLY`
+ and :opcode:`WITH_CLEANUP_START`.
+
+ (Contributed by Mark Shannon, Antoine Pitrou and Serhiy Storchaka in
+ :issue:`17611`.)