Transcript Continuations and Stackless Python

Continuations and Stackless
Python
Where Do You Want To Jump Today?
Stacklessssshhh Python 
... And the clueless implementor ...
Applications of Stackless Python
• FAU – Florida Atlantic University *)
– Soft real time control system for autonomous underwater vehicles.
– Concurrent hierarchy of hierarchical state machines that can be
loaded dynamically. Language Python.
– Real thread implementation is expensive and operates at 1 Hz
Resolution. Microthreads with Stackless Python supposed to run at
12 Hz or faster, also supporting multiple simultaneous behaviors. My
current project 
• Gordon McMillan: Highly parallelized FTP-Server (not open
Source yet)
• Sam Rushing: Medusa on top of coroutines *?)
• MMPOG: Eve! Massively Multi-Player Online Game. *)
•
*) sponsored work *?) may become sponsored, soon
Moving Targets
• Early version: (1999)
– Try to minimize changes in order to get into the core. Guido
reejcts it but accepts the paper for IPC8.
• New version: (2000)
–
–
–
–
–
Still only a small number of changes
Massive changes to ceval.c for speed
Gave up on integration
Make it attractive even without stackless features
This is hard to maintain! Long delay for SLP 2.0.
• Version 2.0: (2000/2001) „The Kiel Edition“
• Maintainability was re-gained
– Minimized code-reorderings, all systematic optimization done by
script
Stackless Python 2.2? No!
• Integration into Standard
• New implementation
–
–
–
–
–
Massive changes to core
Avoid *any* recursion if possible.
Coroutines, Generatrors, Microthreads
Not to be done by me (!!reasons: Gordo does it)
No continuations at all!
• Splitting paths?
• Stackless python will
– do all of the above
– Continue to support continuations
• Educational purposes
• Studying expense
– Try to be faster
How Does It Work?
• Stackful Python:
–
–
–
–
–
...
Interpreter interprets opcodes
CALL_FUNCTION (Python function)
Builds new frame
Calls new interpreter, waiting
• ...
• Interpreter interprets opcodes
• ...
– Continues with result
How Does It Work? Trampoline Style
• Stackless Python:
– Dispatcher:
• Starts interpreter with topmost frame + result
• Interpreter interprets opcodes
– CALL_FUNCTION (Python function)
– Builds new frame
– Returns to Dispatcher with unwind flag
– Dispatcher:
• Starts interpreter with topmost frame + result
• Interpreter interprets opcodes
– ... and so on
Why Continuations?
• Appears to be the simplest possible, most
powerful structure
– Can model *any* kind of control flow
• (also any control flaw)
– Can build Generators, Coroutines and Microthreads in
Python
– Can do much more! (But is it needed?)
– Scheme does it, so Python too?
– Can build very fast class-method like functions
• (see example)
Why Continuations - Example
•
Def very_fast_with_state(*init_params):
–
–
–
–
–
–
–
…
# do complicated initialization
# prepare local state. Locals are persisting
args = continuation.return_current()
…
# do the cheap calculation
Return result
fast_func = very_fast_with_state(“Hi”, 4)
fast_func(1) # use it many times
Why Not Continuations?
• Generators, Coroutines and Microthreads can be
implemented directly in C.
– Will be much faster.
– Safe encapsulation of internal continuations
• Full continuation support is cheap today
– But may become expensive in the future (Stack
optimization, native compiler)
– Can build very fast class-method like functions with
generators as well:
• (see example)
Why Not Continuations - Example
•
Def very_fast_no_continuation(*init_params):
–
–
–
–
–
–
…
# do complicated initialization
# prepare local state. Locals are persisting
args = generator.initialize()
…
While 1:
• # do the cheap calculation
• Args = generator.suspend(result)
fast_func = very_fast_no_continuation(“Hi”, 4)
fast_func(1) # use it many times
Generator vs. Continuation
•
•
•
•
Continuation is immutable
Generator is mutable
Continuation takes control over order of execution
Generator does not introduce new control structure. Frame
jumps from alone by opcode.
First Class vs. One-Shot
• Continuations are everywhere:
– Program counter / return stack.
– Every today‘s program consists of a series of one-shot
continuations.
• One-shot:
– The continuation does not survive its execution
• First-class continuation:
– Does survive execution, can be executed infinite times
– State of frame can always be reset
• First class is most powerful
• One-Shot is sufficient for Microthreads, Coroutines and
Generators.
New Project: Cheetah
•
•
•
•
•
•
•
Stackless from the ground
First: Quick system-independent extension language
Later: Fast system-specific extension language
Forth related/inspired engine
Parsed from restricted Python code
Able to extend regular Python with new primitives
Able to replace the whole Python implementation
• Sponsors are most welcome