Transcript ParaSail Parallel Specification and Implementation Language: A Pointer-Free Path to Secure Object-Oriented Parallel Programming S.

ParaSail
Parallel Specification and
Implementation Language:
A Pointer-Free Path to
Secure Object-Oriented Parallel Programming
S. Tucker Taft
FOOL Workshop, Tuscon, AZ
October 2012
Goals of ParaSail
A simple, safe, pervasively parallel, yet familiar,
language for the multi/many-core world
Easier to write parallel algorithms than
sequential ones
 Parallel by default, yet race-free
 Have to work harder to get sequential execution
Not an academic exercise
 More of a human engineering exercise
 Presumes advanced static analysis at compile-time
AdaCore © 2012
2
Why a new language?
Computers have stopped getting faster
Courtesy IEEE
Computer,
January 2011,
page 33.
AdaCore © 2012
3
A Simplified Approach to
Secure Parallel Programming
 Simplify/Unify

Smaller number of concepts, uniformly applied, all features available
to user-defined types -- generally a good thing


Simplify to make conceptual room for parallelism and formalism
Simplify to make pervasive parallelism easy to verify

Remove all the hard problems!
 Parallelize


Parallel by default, every expression is parallelizable
Have to work harder to force sequential execution
 Formalize
4

Assertions, Invariants, Preconditions, Postconditions integrated into
the syntax


Compiler complains if it can’t prove the assertion
No run-time exceptions, no run-time exception handling
AdaCore © 2012
ParaSail Simplifications
 No pointers (the “goto” of data structuring)
 No global variables
 No global, garbage-collected heap
 No parameter aliasing
 No special syntax reserved for built-in types
 No run-time exception handling
 No explicit threads, lock/unlock, wait/signal
 No race conditions
 No algebraic data types -- use OO polymorphism
 No boxing/run-time-type overhead unless explicitly
polymorphic
5
AdaCore © 2012
ParaSail Type and Value Model
OO/functional core
Types:
 T ::= Tid | Tid+ | Mid <T1, T2, ...> | optional T | new T
 FT ::= (Oid1 : T1; Oid2 : T2; ... func Fid1 FT1; ...) -> T3
Values:
 V ::= N | L | (V) | FN (V1, V2, ... FV1, FV2, ...)
V ::= (if V1 then V2 else V3) | V is null | V not null
 N ::= Oid | N.Oid | Tid :: N
// names
 L ::= 42 | 3.14159 | “a string” | ‘c’ | #green
// literals
L ::= null | (Oid1 => V1, Oid2 => V2, ...)
 FV ::= FN | lambda FT is (V)
// function values
6 FN ::= Fid | Tid :: FN
// function names
AdaCore © 2012
ParaSail Declarations and Modules
OO/functional core
Declarations:
 D ::= type Tid is T;
D ::= func Fid FT [is (V)];
D ::= const Oid [ : T ] [ := V ];
D ::= M
Modules:
 M ::= [abstract] interface Mid < Tid1 is Mid1<>, ... >
[extends Mid2] [implements Mid3<...>, ...]
is D1; D2; ... end interface Mid;
M ::= class Mid is D11; ... exports D21; ... end class Mid;
AdaCore © 2012
7
ParaSail Syntactic Sugar
Sugared Syntax
Expanded Syntax
X bin-op Y
“bin-op”(X, Y)
un-op X
“un-op”(X)
Obj.F(X, Y)
F(Obj, X, Y)
A[I]
“indexing”(A, I)
A[I..J]
“slicing”(A, “..”(I, J))
[]
“[]”()
[A, B, ...]
[] | A | B | ...
#green
#green or “from_univ”(#green)
AdaCore © 2012
8
Example: Countable_Set
interface Countable_Set <Element_Type is Countable<>> is
op "[]"() -> Countable_Set;
func Singleton(Elem : Element_Type) -> Countable_Set;
op “in”(Element_Type; Countable_Set) -> Boolean;
op “|”(Left, Right : Countable_Set) -> Countable_Set;
...
end interface Countable_Set;
class Countable_Set is
type Elem_Interval is Closed_Interval<Element_Type>;
const Items : optional AA_Tree<Elem_Interval>;
exports
op "[]"() -> Countable_Set is ((Items => []));
func Singleton(Elem : Element_Type) -> Countable_Set
is ((Items => [(Low => Elem, High => Elem)] ));
...
end class Countable_Set;
AdaCore © 2012
9
How to add Mutability?
Why Pointer Free?
Consider F(X) + G(Y) ...
 We want to be able to safely evaluate F(X) and G(Y) in parallel
without looking inside of F or G

Presume X and/or Y might be incoming var (in-out) parameters to the
enclosing operation
 No global variables is clearly pretty helpful

Otherwise F and G might be stepping on same object
 No parameter aliasing is important, so we know X and Y do not refer
to the same object; use hand-off semantics
 What do we do if X and Y are pointers?
10

Without more information, we must presume that from X and Y you
could reach a common object Z

Parameter modes (in-out vs. in, var vs. non-var) don’t help with objects
accessible via pointers; need a more elaborate “permission” system
AdaCore © 2012
ParaSail types, values, declarations
with mutable objects
Types and Values with mutable objects:
 T ::= ... [as before]
 FT ::= ( [locked | queued] [var] Oid1 : T1; ... func Fid1 FT1; ...)
[ -> [Oid3 : ] T3 ]
 V ::= ... [as before]
Declarations and Modules with mutable objects:
 D ::= type Tid is T;
D ::= func Fid FT [is (V) | is S];
D ::= (const | var) Oid [ : T ] [ := V ];
D ::= M
 M ::= [abstract] [concurrent] interface ... [as before]
M ::= [concurrent] class ... [as before]
11
AdaCore © 2012
ParaSail imperative statements
preserve value semantics
 S ::= D
// declarations interspersed
S ::= N := V | N1 <== N2 | N1 <=> N2 // copy, move, swap
S ::= FN (V1, V2, ... FV1, FV2, ...) // call for effects on params
S ::= if V then S1 else S2 end if
S ::= block S end block
S ::= [while V] loop S end loop
S ::= exit (loop | block)
S ::= return [ V ]
// or may assign to named output
S ::= S1 ; S2
// sequential, but OK to parallelize
S ::= S1 || S2
// parallel; error if data dependence
S ::= S1 then S2
// force sequential
12
AdaCore © 2012
Expandable Mutable Objects
Instead of Pointers
 All types have additional null value; objects can be
declared optional (i.e.null is OK) and can grow and shrink


Eliminates many of the common uses for pointers, e.g. trees
Assignment (“:=“) is by copy

Move (“<==“) and swap (“<=>”) operators also provided
 Generalized indexing into containers replaces pointers for
cyclic structures

for each N in Directed_Graph[I].Successors loop ...
 Region-Based Storage Mgmt can replace Global Heap


All objects are local with growth/shrinkage using local heap
null value carries indication of region to use on growth
 Short-lived references to existing objects are permitted
13

Returned by user-defined indexing functions, for example
AdaCore © 2012
Mutable Pointer-Free Trees
interface Tree_Node<Payload_Type is Assignable> is
var Payload : Payload_Type;
var Left : optional Tree_Node := null; // defaults to null
var Right : optional Tree_Node := null; // defaults to null
end interface Tree_Node;
var Root : Tree_Node<Univ_String> :=
(Payload => “Top”, Left => null, Right => null);
Root.Left := (Payload => “L”, Right => (Payload => “LR”));
Root.Right <== Root.Left.Right; // Root.Left.Right now null
AdaCore © 2012
14
Example: Mutable Countable_Set
interface Countable_Set <Element_Type is Countable<>> is
op "[]"() -> Countable_Set;
op “|=“(var S : Countable_Set; Elem : Element_Type);
...
end interface Countable_Set;
class Countable_Set is
type Elem_Interval is Closed_Interval<Element_Type>;
var Items : optional AA_Tree<Elem_Interval>;
exports
op "[]"() -> Countable_Set is
return (Items => []);
end op "[]";
op "|="(var S : Countable_Set; Elem : Element_Type) is
const IV : Elem_Interval := (Low => Elem, High => Elem);
if not Overlapping(S.Items, IV) then
S.Items |= IV;
end if;
end op "|=";
AdaCore © 2012
15 ...
Region-Based Storage Management
 Pioneered by Tofte and Talpin

Refined in Cyclone language
 Region (i.e. local heap) in each scope
 Space allocated from, and returned to, associated region
when expanding and shrinking object


null value identifies region from which to allocate
No garbage accumulates; no asynchronous collector
 Region reclaimed in full on exiting scope

effectively => stack-based heap management
 Move (“<==“) and swap (“<=>”) very cheap when staying
within region (analogous to “mv” in Unix)

Can declare object and give “hint” to where it will be moved:
var X for Root : Payload_Type := ...
16
Root.Left.Payload <== X;
AdaCore © 2012
Stack of Regions
with Region Chunks
Regions
Region Chunks
Object
handles
(no chunks)
AdaCore © 2012
17
Regions in a Parallel Programming
Language
 Global garbage-collected heap bad news in parallel
language
 Global lock on allocation is serious bottleneck
 Concurrent threads allocate unrelated objects in neighboring locations
 Individual object has storage spread all over memory
Worst of both worlds (well, all 3 really)
 Region provides attractive alternative
 Each region can have its own lock
 Concurrent threads are allocating in different regions
 Individual objects are concentrated in a single region
Better in all dimensions
AdaCore © 2012
18
Overall ParaSail Model
 ParaSail has four basic concepts:

Module




Type






is an instance of a Type
var Obj : T := Create(...);
mutable value semantics
Operation


19
is an instance of a Module
type T is [new] M <Actual>;
“T+” is polymorphic type for types inheriting from T’s interface
Object


has an Interface, and Classes that implement it
interface M <Formal is Int<>> is ...
Supports inheritance of interface and code

is defined in a Module, and
operates on one or more Objects of specified Types.
are visible automatically based on types of parameters/result
AdaCore © 2012
Expression and Statement
Parallelism
 Within an expression, parameters/operands to an
operation are evaluated in parallel

Max ( F(X), G(Y) * H(Z) )

F(X), G(Y), H(Z) evaluated concurrently
 Programmer can force parallelism across statements

P(X) || Q(Y)
 Programmer can force sequentiality

P(X) then Q(Y)
 Default is run in parallel if no dependences


A := P(X); B := Q(Y) -- can run in parallel
Y := P(X); B := Q(Y) -- cannot run in parallel
 Work stealing used to schedule parallel computations
AdaCore © 2012
20
Example: Recursive Parallel
Word Count
func Word_Count
(S : Univ_String; Separators : Countable_Set<Univ_Character> := [' '])
-> Univ_Integer is
// Return count of words separated by given set of separators
case Length(S) of
[0] => return 0; // Empty string
[1] =>
if S[1] in Separators then
return 0; // A single separator
else
return 1; // A single non-separator
end if;
[..] =>
// Multi-character string; divide and conquer
const Half_Len := Length(S)/2;
const Sum := Word_Count(S[1 .. Half_Len], Separators) +
Word_Count(S[Half_Len <.. Length(S)], Separators);
if S[Half_Len] in Separators
or else S[Half_Len+1] in Separators then
return Sum;
// At least one separator at border
else
return Sum-1; // Combine words at border
end if;
end case;
end func Word_Count;
AdaCore © 2012
21
More Examples of ParaSail
Parallelism
for X => Root then X.Left || X.Right while X not null
concurrent loop
Process(X.Data);
// Process called on each node in parallel
end loop;
concurrent interface Box<Element is Assignable<>> is
func Create() -> Box;
// Creates an empty box
func Put(queued var B : Box; E : Element); // waits til empty
func Get(queued var B : Box) -> Element;
// waits til full
func Get_Now(locked B : Box) -> optional Element;
end interface Box;
type Item_Box is Box<Item>;
var My_Box : Item_Box := Create();
AdaCore © 2012
22
Synchronizing ParaSail Parallelism
concurrent class Box <Element is Assignable<>> is
var Content : optional Element; // starts out null
exports
func Create() -> Box is // Creates an empty box
return (Content => null);
end func Create;
func Put(queued var B : Box; E : Element) is
queued until B.Content is null then
B.Content := E;
end func Put;
// waits until empty
func Get(queued var B : Box) -> Result : Element is // waits until full
queued until B.Content not null then
Result <== B.Content; // “move” sets B.Content to null as side-effect
end func Get;
func Get_Now(locked B : Box) -> optional Element is
return B.Content;
end func Get_Now;
end class Box;
AdaCore © 2012
23
ParaSail Yin and Yang
Race-Free Parallel Programming
Mutable Value Semantics
Stack-Based Heap Management
Compile-Time Exception Handling
AdaCore © 2012
24
Conclusions and Ongoing Work
 Simplified, pointer-free type model can still be flexible

safe by construction
 Can support new capabilities


pervasive parallelism
integrated annotations checked at compile-time
 Ongoing ParaSail work




Finalizing language, interpreter, and work-stealing scheduler
Integrating language with IDE
Building backend to generate compilable C, Ada, LLVM Assembler
Working With Microsoft Research: ParaSail + Dafny
 Read the blog and download the prototype ...
http://parasail-programming-language.blogspot.com
25
AdaCore © 2012
AdaCore
24 Muzzey Street
Lexington, MA 02421
Tucker Taft
[email protected]
http://parasail-programming-language.blogspot.com
+1 (781) 750-8068 x220
AdaCore © 2012
26