Transcript Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld Chapter 9 Consensus Version: June 2014 Chapter 9 Synchronization Algorithms and Concurrent Programming Gadi Taubenfeld © 2014

Synchronization Algorithms
and Concurrent Programming
Gadi Taubenfeld
Chapter 9
Consensus
Version: June 2014
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
1
Synchronization Algorithms
and Concurrent Programming
ISBN: 0131972596, 1st edition
A note on the use of these ppt slides:
I am making these slides freely available to all (faculty, students, readers).
They are in PowerPoint (2003) form so you can add, modify, and delete slides
and slide content to suit your needs. They obviously represent a lot of work on
my part. In return for use, I only ask the following:
 That you mention their source, after all, I would like people to use my book!
 That you note that they are adapted from (or perhaps identical to)
my slides, and note my copyright of this material.
Thanks and enjoy!
Gadi Taubenfeld
All material copyright 2014
Gadi Taubenfeld, All Rights Reserved
To get the most updated version of these slides go to:
http://www.faculty.idc.ac.il/gadi/book.htm
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
2
Chapter 9
Consensus
9.1
9.2
9.3
9.4
9.5
The Problem
Three Simple Consensus Algorithms
Consensus without Memory Initialization
Reaching Consensus Using a Shared Queue
Impossibility of Consensus with One Faulty
Process
9.6 The Relative Power of Synchronization Primitives
9.7 The Universality of Consensus
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
3
Section 9.1
Consensus
1
0
1
1
0
1
p1
p2
p3
p4
p5
p6
1
1
1
1
1
1
0
…
pn
1
Each process pi has input value ini
 agreement: All non-faulty processes eventually
decides on the same value v.
 validity: v  {in1, in2, …, inn }.
-- binary consensus ; multi-valued consensus
-- we assume crash (fail-stop) failures
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
4
Consensus
1
0
1
1
0
1
p1
p2
p3
p4
p5
p6
1
1
1
1
1
1
0
…
pn
1
There is a trivial solution when there are no faults,
but what happens when there are faults?
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
5
Impossibility of Consensus with
One Faulty Process
Theorem: There is no (asynchronous) algorithm that
solves the consensus problem using atomic read/write
registers in the presence of a single faulty process.
We give a detailed proof of this important result in
Chapter 9.5 of the book.
 Same result holds also for message passing systems
What can be done?
 Strong primitives.
 Timing assumptions.
 Randomization.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
6
Three Simple Consensus Algorithms
Section 9.2
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
7
RMW: n processes one 3-valued register
(trivial)
3-valued RMW
register
z

The Algorithm
 The first process to access z sets z to its input.
 All the processes adopt this value as the decision value.
The algorithm is wait-free (i.e., can tolerate any
number of failures)
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
8
RMW: Two processes three bits (trivial)
atomic
r/w bit
atomic
r/w bit
r1
r2
RMW
bit
z
0
The Algorithm
1.
Each process pi uses one bit ri to announce its input.
2.
Then, each process tries to set z from 0 to 1.
3.
The decision value is the input of the process that
succeeds in step 2.
 The algorithm is wait-free
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
9
RMW: Two processes two bits (not trivial)
RMW
bit
x
RMW
bit
0
The algorithm is wait-free
Chapter 9
y
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
10
RMW: Two processes two bits (not trivial)
RMW
bit
x
RMW
bit
0
y
0
Algorithm for process pi with input ini
1.
If x =1 then pi decides 1 and halts, otherwise pi sets x to ini
and continues.
2.
If y =0 then pi sets y to 1, decides ini and halts, otherwise pi
continues.
3.
If x =0 then pi decides 0 and halts, otherwise pi decides 1-ini
.
The algorithm is wait-free
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
11
RMW: Two processes two bits (not trivial)
RMW
bit
x
RMW
bit
0
y
0
NO
Is the following modified version of the algorithm correct?
Algorithm for process pi with input ini
1.
If x =1 then pi decides 1 and halts, otherwise pi sets x to ini
if ini =1 than pi decides 1 and halts else pi continues.
2.
If y =0 then pi sets y to 1, decides ini and halts, otherwise pi
continues.
3.
If x =0 then pi decides 0 and halts, otherwise pi decides 1-ini
.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
12
RMW: Three processes two bits
RMW
bit
x
RMW
bit
0
y
0
IMPOSSIBLE!
There is NO wait-free algorithm!!!
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
13
Three processes many RMW bits and
atomic register
RMW
bits
atomic
read/write
registers
0
0
0
0
0
0
0
0
0
0
0
0
IMPOSSIBLE!
There is NO wait-free algorithm!!!
Theorem: There is NO consensus algorithm for n
processes that can tolerate 2 faults, for any n.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
14
Two processes two atomic read/write
registers
atomic
register
x
atomic
register
0
y
0
IMPOSSIBLE!
There is NO wait-free algorithm!!!
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
15
Two processes many atomic register
atomic
read/write
registers
0
0
0
0
0
0
IMPOSSIBLE!
There is NO wait-free algorithm!!!
Theorem: There is NO consensus algorithm for n
processes that can tolerate one fault, for any n.
We will prove this very interesting theorem later.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
16
RMW: many processes, four bits,
at most one fault
RMW
bits
Chapter 9
0
0
0
0
x
y
decision
finish
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
17
RMW: n processes, four bits,
at most one fault
RMW
bits
0
0
0
0
x
y
decision
finish
The Algorithm
1.
Process 1 and process 2 execute the algorithm for two
processes using x and y.
2.
Then, they write the decision value into the decision bit
and set finish to 1.
3.
Each other process busy waits until finish=1, and then
decides on the value of the decision bit.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
18
RMW: n processes, four bits,
at most one fault
RMW
bits
0
0
0
0
x
y
decision
finish
Is it important that all the four bits are initially 0 ?
No, the initial value of the decision bit is immaterial.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
19
Important Conclusion
RMW
bits
read/write
registers
Theorem: There is a consensus algorithm for n
processes that can tolerate one fault, for any n.
Theorem: There is NO consensus algorithm for n
processes that can tolerate one fault, for any n.
It is not possible to implement a single RMW bit for
two processes or more, from atomic read/write
registers, that can tolerate one fault.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
20
Question
Crash-safe semaphores
Why is it not possible to implement a semaphore S for
three processes or more, from RMW bits (or test-andset bits) and atomic read/write registers (no kernel
calls), which supports the usual up(S) and down(S)
operations, and in addition, it is possible to infer which
process holds the semaphore S ?
(Semaphores are defined in Section 4.6, page 176.)
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
21
Consensus without Memory Initialization
Section 9.3
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
Consensus without Memory Initialization
RMW register
(not bit)
?
Q: How big?
A: 3n values.
Solve the consensus problem in the following model:

There are n processes that communicate via a single
“large enough” RMW register.

The initial value of the register is not a priori known.

As always, processes are asynchronous.

Processes may crash.
How
would we
do it
in a
With
a known
initial
value,
fault-free
it can
be donemodel?
with a 3valued register.
Chapter 9
We assume that at most n/2 1 processes may crash.
Impossible if n/2
processes may crash!
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
23
Consensus without Memory Initialization
A single RMW register with 3n values
bit
?
b
n
Chapter 9
1.5n -1
?
c
0
0
circle
b=0
circle
b=1
n/2
B = {0,1}
C = {0, 1.5n -2
n
n/2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
24
Consensus without Memory Initialization
Example: n = 10
n
10
Chapter 9
0
4
b
c
n = 10
B = {0,1}
C = {0,…,13}
0
0
circle
b=0
circle
b=1
n
n/2
5
n/2
10
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
5
25
Consensus without Memory Initialization
n
0
0
circle
b=0
circle
b=1
n
n/2
10
5
10
n =10
n/2
5
0
4
5
11
b
c
The Algorithm:

Each process reads the value of the shared register,
stores it, and increments c by one (modulo |C|).

A process becomes the master if it learns (by
inspecting c) that more than half of the processes
have waked up and the value of b has not changed.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
0
4
26
Consensus without Memory Initialization
10
Example:
v=0
0
0
n =10
0-interval
0-interval
Example:
0 v = 11
1
1-interval
5
10
5
1-interval
b
c
The master:

The master decides on its own input value, say v ;

if v = 0 then c

if b = 0 then b  1 else b  0 ;
Chapter 9

0 else c
 n/2
;
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
one
atomic
action
27
Consensus without Memory Initialization
10
0
0
0-interval
0-interval
1-interval
5
10
1-interval
n =10
5
1
5
b
c
The master:

continues forever setting c to the beginning of the
interval corresponding to the decision.
(i.e., if v = 0 then c
Chapter 9

0 else c  n/2 )
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
28
Consensus without Memory Initialization
10
0
0
0-interval
0-interval
1-interval
5
10
1-interval
n =10
5
1
5
b
c
Every other process:


Chapter 9
find outs that a decision has been made:

b changed value, or

c has been incremented by more than n (modulo |C|) ;
makes a decision according to the interval in which c lies ;
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
29
Consensus without Memory Initialization
10
0
0
0-interval
0-interval
1-interval
5
10
1-interval
n =10
5
1
5
b
c
Every process:

Chapter 9
after making its decision, continues forever
setting c to the beginning of the interval
corresponding to its decision.
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
30
Consensus without Memory Initialization
Comments:
Chapter 9

There is a fault-free solution that uses only 5
values. (Based on the See-Saw Barrier, page 215.)

The known lower bound (when there are faults) is
Ω(n0.63) values.
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
31
Reaching Consensus using a Shared Queue
Section 9.4
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
Reaching Consensus using a Shared Queue
Requirements:

Solution must be wait-free.

Can use many queues & atomic registers
Observations:

With a peek operation – trivial for many processes.

Three processes – Impossible! (without peek)

Two processes with initialized queue – trivial (without peek)

Two processes with empty queue – not trivial (without peek)
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
33
Reaching Consensus using a Shared Queue
Program for process pi with input ini
if ini = 0 then
enqueue(Q,0)
if R=0 then decide(0)
else
if dequeue(Q) = empty
then decide(0)
else decide (1) fi
fi
else
R:=1
if dequeue(Q) = empty
then decide(1)
else decide (0) fi
fi
Chapter 9
Queue,
initially empty
Q
Atomic bit
R
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
0
34
Impossibility of Consensus with One Faulty
Process
Section 9.5
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
Section 9.5
Impossibility of Consensus with
One Faulty Process
Theorem: There is no (asynchronous) algorithm that
solves the consensus problem using atomic read/write
registers in the presence of a single faulty process.
Same result holds also for message passing systems.
Why?
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
36
Section 9.5
Impossibility of Consensus with
One Faulty Process
Notions:

run
run, empty run,
state
event
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
37
bivalent & univalent
bivalent
step by p
time
bivalent
bivalent
0
step by q
1
bivalent
bivalent
0-valent
0
0
0
1
1-valent
1
1
final decision values
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
38
Lemma 1
(simple)
Any consensus algorithm that can tolerate one crash failure
has a bivalent empty run (i.e., bivalent initial state).
bivalent
0
Chapter 9
1
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
39
Lemma 1
Proof
Any consensus algorithm that can tolerate one crash failure
has a bivalent empty run.
Chapter 9
0000
0001
0
0
0011
0
1
0111
1111
1
1
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
40
Proof of the theorem
Assume to the contrary that CONS is a consensus algorithm
that can tolerate one crash failure ...
step by p1
bivalent
other processes may
take steps, but p1 must
take at least one step
step by p2
Since CONS is correct,
this run can not be
extended forever.
step by pn
step by p1
x
step by p2
step by pn
Chapter 9
 for some process p,
for every extension y of
x the run yp is univalent.
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
41
Proof of the theorem
for every extension y of
x the run yp is univalent
single step
by p
x
no steps by p
the shortest
extension of x
which is
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
42
Proof of the theorem
single step
by p
x
no steps by p
single step
by p
Chapter 9
option 1:
single step
by p
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
43
Proof of the theorem
single step
by p
x
no steps by p
option 2:
not a step
by p
single step
by p
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
44
Proof of the theorem
So, in both option 1 and option 2
we get the following picture
Four cases
p
q
read read
write read
single step
by p
read write
x
no steps by p
p
write write
q
p
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
45
Proof of the theorem
Four cases
p
q
read read
write read
read write
write write
p
q
p
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
46
Proof of the theorem
Four cases
p
p
q
p
q
read read
write read
read write
write write
q does not
take steps
decide 0
A contradiction
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
47
Proof of the theorem
Four cases
p
p
q
q
p
q
read read
write read
read write
write write
p does not
take steps
decide 0
A contradiction
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
48
Proof of the theorem
Four cases
p
p
read read
q
q
write read
read write
p
All processes
take steps
q
write write
• different registers
• same register
decide 0
A contradiction
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
49
Proof of the theorem
Four cases
p
p
q
read read
q
write read
p
read write
write write
q does not
take steps
• different registers
• same register
decide 0
A contradiction
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
50
Proof of the theorem
QED
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
51
The Relative Power of Synchronization
Primitives
Section 9.6
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
The Relative Power of Synchronization Primitives
Definition: The consensus number of an object of type o,
denoted CN(o), is the largest n for which it is possible to
solve consensus for n processes using any number of objects
of type o and any number of atomic registers. If no largest n
exists, the consensus number of o is infinite.
Examples:
• CN(atomic-register) = 1
• CN(read-modify-write bit) = 2
• CN(queue) = 2
• CN(3-valued read-modify-write register) = 
• CN(compare-and-swap) = 
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
53
Section 9.6
The Relative Power of Synchronization Primitives
Consensus
Number
object
1
atomic-register, atomic-snapshot, safem, regularm
2
test-and-set, fetch-and-increment, fetch-and-add,
swap, queue, stack, read-modify-write bit,
fetch-and-incrementm, fetch-and-addm
(m)
swapm
2m-2
m-register assignment, atomic-registerm (m>1)

compare-and-swap, LL/SC, sticky-bit, queue2,
3-valued read-modify-write, augmented-queue
Om is an object in which a process is allowed to atomically
access m objects of type O
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
54
Section 9.6
The Relative Power of Synchronization Primitives
Consensus
Number
object
1
atomic-register, atomic-snapshot, safem, regularm
2
test-and-set, fetch-and-increment, fetch-and-add,
swap, queue, stack, read-modify-write bit,
fetch-and-incrementm, fetch-and-addm
(m)
swapm
2m-2
m-register assignment, atomic-registerm (m>1)

compare-and-swap, LL/SC, sticky-bit, queue2,
3-valued read-modify-write, augmented-queue
Theorem: Let o1 and o2 be two objects such that CN(o1) < CN(o2).
Then, in a system with CN(o2) processes,

There is no wait-free implementation of an object of type o2
from objects of type o1 and atomic registers;

Chapter 9
There is a wait-free implementation of an object of type o1
from objects of type o2 and atomic registers.
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
55
Section 9.6
The Relative Power of Synchronization Primitives
Consensus
Number
object
1
atomic-register, atomic-snapshot, safem, regularm
2
test-and-set, fetch-and-increment, fetch-and-add,
swap, queue, stack, read-modify-write bit,
fetch-and-incrementm, fetch-and-addm
(m)
swapm
2m-2
m-register assignment, atomic-registerm (m>1)

compare-and-swap, LL/SC, sticky-bit, queue2,
3-valued read-modify-write, augmented-queue
Theorem: Let o1 and o2 be two objects with consensus number n.
Then, using any number of additional atomic registers,

The objects o1 and o2 can wait-free implement each other
when the number of processes is less than or equal to n;

Chapter 9
The objects o1 and o2 do not necessarily wait-free implement
each other when the number of processes is more than n.
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
56
Open Question
Consensus
Number
object
1
atomic-register
2
test-and-set, fetch-and-increment, fetch-and-add,
swap, queue, stack, read-modify-write bit
(m)
2m-2

Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
57
The Universality of Consensus
Section 9.7
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
Section 9.7
Universality
Definition: An object o is universal for n processes if

any object that has sequential specification has

wait-free

linearizable implementation

using atomic registers and objects of type o

in a system with n processes.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
59
Section 9.7
The Universality of Consensus
Definition: An object o is universal for n processes if

any object that has sequential specification has

wait-free

linearizable implementation

using atomic registers and objects of type o

in a system with n processes.
Theorem: A consensus object for n processes is universal
in a system with n processes, for any positive n.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
60
Section 9.6
Consensus Numbers & Universality
Consensus
Number
object
1
atomic-register, atomic-snapshot, consensus(1)
2
test-and-set, fetch-and-increment, fetch-and-add,
swap, queue, stack,
consensus(2)
fetch-and-incrementm, fetch-and-addm
(m)
swapm
consensus(m)
2m-2
m-register assignment,
consensus(2m-2)

compare-and-swap, LL/SC, sticky-bit, queue2,
3-valued read-modify-write,
consensus()
Theorem: A consensus object for n processes is universal
in a system with n processes, for any positive n.
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
61
Section 9.7
The Universality of Consensus: Proof
A detailed proof appears in Section 9.7
Chapter 9
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
62