Continuations and Stackless Python Where Do You Want To Jump Today?

Stacklessssshhh Python... And the clueless implementor...

Why Stackless? Recursion depth limits –Platform dependant Generators Coroutines Micro-Threads  Persistent State  –Pickle the running program –Mobile Agents Continuations Modify running Program 

Fact Sheet Minimum C Stack resources Unlimited recursion 100% binary compatible to Standard Python –Except for left bugs 5% faster executable on Win32faster Full support for first-class continuations Continuations included as extension module As portable as Standard Python Breaks 4K stack limit on PalmIII

What Is a Continuation x = 2; y = x + 1; z = x * 2 The continuation ofIs this! The current continuation at any point in the execution of a program is an abstraction of the rest of the program

Continuation Objects Tiny interface objects to Python frames Continuations are callable objects Calling brings us immediately back to where we left off Can pass and return a value

Implementing a Loop def looptest(n): this = continuation.current() k = this.update(n) if k: this(k-1) else: del this.link Loosing frame reference Saved assignment Jump back

Building a Generator (1)

Building a Generator (2) Catch the current caller Jump to saved frame

Easy? Almost, but... The given principle can implement coroutines We can switch to any other disjoint frame chain Trouble when more than one continuation owns the target frame!

a-frame with refcount > 1 b-frame still referenced c-frame wants to return Critical Situation

Frame Ownership In order to jump to a frame, we must own it We own a frame iff its refcount is 1* The program must ensure this by building clone frames (solution, first half). We cannot just make a copy, but keep the current fram in place... (solution, second half) *Note: This rule is strong enough to build a complete object system!

a-frame as continuation b-frame still referenced c-frame wants to return new frame with contents of a-frame Push-back Frame

3 rd Half of Solution Upto version 0.3: –Managing growing chains of pushed-back frames becomes tedious. Continuation frames pile up. Runtime isn‘t O(1) Insert an extra node object that always walks with the real frame does the trick Runtime is O(1) now

a-frame as continuation b-frame still referenced c-frame wants to return new frame with contents of a-frame PyCoNode Node Object

a-frame as continuation b-frame still referenced c-frame wants to return new frame with contents of a-frame PyCoNode Final View Rc=1 Rc=3 Rc=1 (exec) About to die Rc=1

def a(x):... b(x+1)... def b(x):... c(x*x)... def c(x):... print "x=",x... a(42) interactive frame a-frame b-frame c-frame Frame StackC Stack (simplified) PyRun_InteractiveOne PyEval_EvalCode eval_code2 call_function eval_code2... eval_code2... eval_code2... Standard „stackfull“ Python

c-frame PyEval_Frame_Dispatch this *can* still happen Dispatcher Object eval_code2_loop Bad case: an internal function does a call via the old interface interactive frame a-frame b-frame PyRun_InteractiveOne PyEval_EvalCode eval_code2_loop PyEval_Frame_Dispatch always calls from here Dispatcher Object Good case: always the same dispatcher Stackless Python

About Optimization Moved opcode/oparg into the instructions Shortened the big switch Mapped Error handling to pseudo-opcodes Shortened lifetime of state variables Inc/decref not macro but inline (win32) Would gain about 15% for standard Python –(but this is not what I want )