Continuations and Stackless Python Where Do You Want To Jump Today?

Stacklessssshhh Python... And the clueless implementor...

Applications of Stackless Python FAU – Florida Atlantic University *) –Soft real time control system for autonomous underwater vehicles. –Concurrent hierarchy of hierarchical state machines that can be loaded dynamically. Language Python. –Real thread implementation is expensive and operates at 1 Hz Resolution. Microthreads with Stackless Python supposed to run at 12 Hz or faster, also supporting multiple simultaneous behaviors. My current project Gordon McMillan: Highly parallelized FTP-Server (not open Source yet) Sam Rushing: Medusa on top of coroutines *?) MMPOG: Eve! Massively Multi-Player Online Game.  *) *) sponsored work *?) may become sponsored, soon

Moving Targets Early version: (1999) –Try to minimize changes in order to get into the core. Guido reejcts it but accepts the paper for IPC8. New version: (2000) –Still only a small number of changes –Massive changes to ceval.c for speed –Gave up on integration –Make it attractive even without stackless features –This is hard to maintain! Long delay for SLP 2.0. Version 2.0: (2000/2001) „The Kiel Edition“ Maintainability was re-gained –Minimized code-reorderings, all systematic optimization done by script

Stackless Python 2.2? No! Integration into Standard New implementation –Massive changes to core –Avoid *any* recursion if possible. –Coroutines, Generatrors, Microthreads –Not to be done by me (!!reasons: Gordo does it) –No continuations at all! Splitting paths? Stackless python will –do all of the above –Continue to support continuations Educational purposes Studying expense –Try to be faster

How Does It Work? Stackful Python: –... –Interpreter interprets opcodes –CALL_FUNCTION (Python function) –Builds new frame –Calls new interpreter, waiting... Interpreter interprets opcodes... –Continues with result

How Does It Work? Trampoline Style Stackless Python: –Dispatcher: Starts interpreter with topmost frame + result Interpreter interprets opcodes –CALL_FUNCTION (Python function) –Builds new frame –Returns to Dispatcher with unwind flag –Dispatcher: Starts interpreter with topmost frame + result Interpreter interprets opcodes –... and so on

Why Continuations? Appears to be the simplest possible, most powerful structure –Can model *any* kind of control flow (also any control flaw) –Can build Generators, Coroutines and Microthreads in Python –Can do much more! (But is it needed?) –Scheme does it, so Python too? –Can build very fast class-method like functions (see example)

Why Continuations - Example Def very_fast_with_state(*init_params): –… –# do complicated initialization –# prepare local state. Locals are persisting –args = continuation.return_current() –… –# do the cheap calculation –Return result fast_func = very_fast_with_state(“Hi”, 4) fast_func(1) # use it many times

Why Not Continuations? Generators, Coroutines and Microthreads can be implemented directly in C. –Will be much faster. –Safe encapsulation of internal continuations Full continuation support is cheap today –But may become expensive in the future (Stack optimization, native compiler) –Can build very fast class-method like functions with generators as well: (see example)

Why Not Continuations - Example Def very_fast_no_continuation(*init_params): –… –# do complicated initialization –# prepare local state. Locals are persisting –args = generator.initialize() –… –While 1: # do the cheap calculation Args = generator.suspend(result) fast_func = very_fast_no_continuation(“Hi”, 4) fast_func(1) # use it many times

Generator vs. Continuation Continuation is immutable Generator is mutable Continuation takes control over order of execution Generator does not introduce new control structure. Frame jumps from alone by opcode.

First Class vs. One-Shot Continuations are everywhere: –Program counter / return stack. –Every today‘s program consists of a series of one-shot continuations. One-shot: –The continuation does not survive its execution First-class continuation: –Does survive execution, can be executed infinite times –State of frame can always be reset First class is most powerful One-Shot is sufficient for Microthreads, Coroutines and Generators.

New Project: Cheetah Stackless from the ground First: Quick system-independent extension language Later: Fast system-specific extension language Forth related/inspired engine Parsed from restricted Python code Able to extend regular Python with new primitives Able to replace the whole Python implementation Sponsors are most welcome