Transcript An Introduction to Erlang

Static Detection of Race Conditions
in Erlang
Maria Christakis
National Technical University of Athens, Greece
Joint work with Kostis Sagonas
Concurrency

A method to better structure programs

A means to speed up their execution

A necessity in order to take advantage of multicore machines which are ubiquitous these days
The catch:

Concurrent programming is harder
and more error-prone than its
sequential counterpart
Maria Christakis
Static Race Detection in Erlang
Erlang



Erlang is a strict, dynamically typed
functional programming language
The main implementation of the
language is the Erlang/OTP system
from Ericsson
Its concurrency model is based on
user-level processes that communicate
using asynchronous message passing
Maria Christakis
Static Race Detection in Erlang
Erlang is not immune to Heisenbugs



Its implementation in Erlang/OTP allows for many
kinds of race conditions in programs
But Erlang is often advertized as supporting a
shared nothing concurrency model
If there is nothing shared between processes, how
can there be race conditions?
System built-ins allow processes to share data
Erlang currently provides no atomicity constructs
Maria Christakis
Static Race Detection in Erlang
Data races can happen
When a process reads some
variable, it then decides to
take some write action based
on the value of that variable
If it is possible for another process to succeed in
changing the value stored on that variable in
between the read and the action in such a way that
the action about to be taken is no longer
appropriate, then we say that our program has a
race condition
Maria Christakis
Static Race Detection in Erlang
Data races in the process registry
proc_reg(Name) ->
...
case whereis(Name) of
undefined ->
Pid = spawn(...),
register(Name, Pid);
Pid ->
% already
ok
% registered
end,
...
Maria Christakis
Static Race Detection in Erlang
Data races in the process registry
Maria Christakis
Static Race Detection in Erlang
Data races in ETS
run() ->
Tab = ets:new(some_tab_name, [public]),
Inc = compute_inc(),
Fun = fun () -> ets_inc(Tab, Inc) end,
spawn_some_processes(Fun).
ets_inc(Tab, Inc) ->
case ets:lookup(Tab, some_key) of
[] ->
ets:insert(Tab, {some_key, Inc});
[{some_key, OldValue}] ->
NewValue = OldValue + Inc,
ets:insert(Tab, {some_key, NewValue})
end.
Maria Christakis
Static Race Detection in Erlang
Data races in mnesia
-export([table_func/2]).
table_func(...) ->
create_time_stamp_table(), ...
create_time_stamp_table() ->
Props = [{type, set}, ...],
create_table(time_stamp, Props, ram_copies, false),
NRef =
case mnesia:dirty_read(time_stamp, ref_count) of
[] -> 1;
[#time_stamp{data = Ref}] -> Ref + 1
end,
mnesia:dirty_write(#time_stamp{data = NRef}).
Maria Christakis
Static Race Detection in Erlang
Single-threaded Erlang




A single scheduler picks up processes
from a single ready queue
The selected process gets assigned a
number of reductions to execute
Each time the process does a function
call, a reduction is consumed
A process gets suspended when the
number of remaining reductions
reaches zero, or when it gets stuck
Maria Christakis
Static Race Detection in Erlang
Single-threaded Erlang
proc_reg(Name) ->
...
case whereis(Name) of
undefined ->
Pid = spawn(...),
register(Name, Pid);
Pid ->
% already
ok
% registered
end,
...
Being struck by a lightning seems more likely!
Maria Christakis
Static Race Detection in Erlang
Multi-threaded Erlang



Since May 2006, a multi-threaded version of the
system has been released, which is the default on
multi-core architectures
There are multiple schedulers, each having its
own ready queue
Since March 2009, the runtime
system employs a redistribution
scheme based on work stealing
when some scheduler’s run
queue becomes empty
Maria Christakis
Static Race Detection in Erlang
Dialyzer



A DIscrepancy AnaLYZer for ERlang programs
Dialyzer has been used by the Erlang community
since 2007, as part of the Erlang/OTP distribution
A lightweight static analysis tool for finding
discrepancies in Erlang programs

Type errors

Exception-raising code

Unsatisfiable conditions

Redundancies such as unreachable code, etc.
Maria Christakis
Static Race Detection in Erlang
Dialyzer
Characteristics of dialyzer (in sequential programs):

Sound for defect detection – not for correctness!

Push-button technology, completely automatic

Fast and scalable
Why dialyzer?

Inter-modular call graph and escape analysis

Control-flow graphs

Type information
Maria Christakis
Static Race Detection in Erlang
The analysis
Characteristics:

Sound for either correctness or defect
detection!

As precise as possible

Completely automatic

Fast and scalable

Smoothly integrated into dialyzer
Maria Christakis
Static Race Detection in Erlang
The analysis: a three-step process
1. Collecting information

Control-flow graphs of functions
and closures

Escape analysis

Inter-modular call graph

Sharing/alias analysis

Fine-grained type information
(singleton types)
Maria Christakis
Static Race Detection in Erlang
The analysis: a three-step process
2. Determining all code points with possible race
conditions

Find the root nodes in the inter-modular call graph

Traverse their CFGs using depth-first search

Special cases:

Statically known function or closure calls

Unknown higher-order calls

Recursion
Maria Christakis
Static Race Detection in Erlang
The analysis: a three-step process
3. Filtering false alarms

Variable sharing

Type information

Characteristics of race conditions
foo(Fun, N, M) ->
...
case whereis(N) of
undefined ->
...,
Fun(M);
Pid -> ...
end,
...
Maria Christakis
Static Race Detection in Erlang
Some optimizations



Control-flow graph minimization
Avoiding repeated traversals and benefiting from
temporal locality
Making unknown function calls less unknown
Maria Christakis
Static Race Detection in Erlang
Detecting data races
mod.erl
1 : proc_reg(Name) ->
2 :
...
3 :
case whereis(Name) of
4 :
undefined ->
5 :
Pid = spawn(...),
6 :
register(Name, Pid);
7 :
Pid ->
% already
8 :
ok
% registered
9 :
end,
10:
...
mod.erl:6:The call erlang:register(Name::atom(),Pid::pid())
might fail due to a possible race condition caused by its combination with
the erlang:whereis(Name::atom()) in mod.erl on line 3
Maria Christakis
Static Race Detection in Erlang
Performance
Maria Christakis
Static Race Detection in Erlang
Since September…

Our analysis has been publicly released as part of the
latest Erlang/OTP distribution (November 2009)
From: Bernard Duggan (Erlang developer)
Sent to us on 27 November 2009
“Our Erlang codebase comprises 5 applications
and a few little ancillary bits and pieces on the side
– it's about 40k lines. So far it's turned up three
race conditions. … Thanks for a brilliant tool.”
Maria Christakis
Static Race Detection in Erlang
Race Detection in Erlang (ICFP’09)
QuickCheck:



A property-based testing tool
PULSE is a ProTest User Level Scheduler for
Erlang that randomly schedules the test case
processes and records a detailed trace
A race condition is a possibility of nondeterministic execution that can make a program
fail to meet its specification
Maria Christakis
Static Race Detection in Erlang
Concluding remarks


Future work:

Detection of more kinds of race conditions

Detection of more types of concurrency errors

Addition of an atomic construct to the language
Besides providing a tool that statically detects data
races, we believe that we have also contributed to
raising the awareness of the Erlang community on
race conditions
Maria Christakis
Static Race Detection in Erlang