Transcript www.faculty.idc.ac.il

Synchronization Algorithms
and Concurrent Programming
Gadi Taubenfeld
Chapter 2
Mutual Exclusion using atomic registers:
Basic Topics
Version: June 2014
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
1
Synchronization Algorithms
and Concurrent Programming
ISBN: 0131972596, 1st edition
A note on the use of these power-point slides:
I am making these slides freely available to all (faculty, students, readers).
They are in PowerPoint form so you can add, modify, and delete slides and slide
content to suit your needs. They obviously represent a lot of work on my part.
In return for use, I only ask the following:
 That you mention their source, after all, I would like people to use my book!
 That you note that they are adapted from (or perhaps identical to)
my slides, and note my copyright of this material.
Thanks and enjoy!
Gadi Taubenfeld
All material copyright 2014
Gadi Taubenfeld, All Rights Reserved
To get the most updated version of these slides go to:
http://www.faculty.idc.ac.il/gadi/book.htm
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
2
Chapter 2
Mutual Exclusion using atomic
registers: Basic Topics
2.1 Algorithms for Two Processes
2.2 Tournament Algorithms
2.3 A Fast Mutual Exclusion
Algorithms
2.4 Starvation-free Algorithms
2.5 Tight Space bounds
2.6 Automatic Discovery of
algorithms
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
3
The Mutual Exclusion Problem
Basic definitions
Chapter 1
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
4
The mutual exclusion problem
remainder code
entry code
critical section
exit code
The problem is to design the entry and exit code in a way that
guarantees that the mutual exclusion and deadlock-freedom
properties are satisfied.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
5
The mutual exclusion problem
 Mutual Exclusion: No two processes are in
their critical sections at the same time.
 Deadlock-freedom: If a process is trying
to enter its critical section, then some
process, not necessarily the same one,
eventually enters its critical section.
 Starvation-freedom: If a process is
trying to enter its critical section, then
this process must eventually enter its
critical section.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
remainder
entry code
critical
section
exit code
6
Question: true or false ?
Algorithm A
Algorithm B
Algorithm C
remainder
entry code of A
remainder
entry code of B
entry code
critical
section
critical
section
exit code of B
exit code
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
exit code of A
7
Question: true or false ?
Algorithm C
 A and B are deadlock-free  C is
deadlock-free.
 A and B are starvation-free  C is
starvation-free.
 A or B satisfies mutual exclusion 
C satisfies mutual exclusion.
 A is deadlock-free and B is
starvation-free  C is starvationfree.
 A is starvation-free and B is
deadlock-free  C is starvationfree.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
remainder
entry code of A
entry code of B
critical
section
exit code of B
exit code of A
8
Atomic read/write registers
Proposed solution 1
entry
code
exit
code
Thread 0
while (true} {
remainder code
while (turn = 1) {skip}
Thread 1
while (true} {
remainder code
while (turn = 0) {skip}
turn = 1
}
turn = 0
}
critical section
critical section
turn
0/1
mutual exclusion
 deadlock-freedom
Does it work?
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
9
Convention
Thread 0
while (true} {
remainder code
entry code
entry code
critical section
critical section
exit code
exit code
}
Chapter 2
Thread 1
while (true} {
remainder code
}
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
10
Convention
Thread 0
Chapter 2
Thread 1
entry code
entry code
critical section
critical section
exit code
exit code
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
11
Proposed solution 2
Thread 0
Thread 1
flag[0] = true
flag[1] = true
while (flag[1]) {skip} while (flag[0]) {skip}
critical section
critical section
flag[0] = false
flag[1] = false
flag
0 false
1 false
mutual exclusion
 deadlock-freedom
Does it work?
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
12
Proposed solution 3
Thread 0
Thread 1
while (flag[1]) {skip}
while (flag[0]) {skip}
flag[0] = true
flag[1] = true
flag[0] = false
flag[1] = false
critical section
critical section
flag
0 false
1 false
 mutual exclusion
Deadlock-freedom
Does it work?
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
13
Algorithms for Two Processes
Section 2.1
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
14
Section 2.1.1
Peterson’s algorithm
Thread 0
flag[0] = true
turn = 1
while (flag[1] and turn = 1)
{skip}
Thread 1
flag[1] = true
turn = 0
while (flag[0] and turn = 0)
{skip}
flag[0] = false
flag[1] = false
critical section
critical section
flag
0 false
1 false
turn 0/1
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
15
A variant of Peterson’s algorithm
Is it correct ?
Thread 0
turn = 1
flag[0] = true
while (flag[1] and turn = 1)
{skip}
Thread 1
turn = 0
flag[1] = true
while (flag[0] and turn = 0)
{skip}
flag[0] = false
flag[1] = false
critical section
critical section
flag
true
0 false
1 false
true
turn 0/1
0
1
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
16
Schematic for Peterson’s algorithm
Indicate contending
b[i] := true
Barrier
turn := i
Contention?
b[i] = true ?
no / maybe
yes
First to cross
the barrier?
turn = j ?
no
critical section
yes
exit code
b[i] = false ?
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
17
Properties of Peterson’s Solution
 Satisfies mutual exclusion and starvation-freedom
 Memory accesses are assumed to be atomic
 Solution for two processes only
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
18
Section 2.1.2
Question
Using only single-writer registers
Replace the multi-writer register
turn with two single-writer registers.
What is new algorithm?
Hint: use Solution #4 for the
too-much-milk problem.
Answer (Kessels’ Alg.)
turn = 0  turn[0] = turn[1]
turn = 1  turn[0]  turn[1]
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
19
Algorithms for Many Processes
Section 2.2
How can we use two-process
algorithm to construct an
algorithm for many processes?
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
20
Section 2.2
Tournament Algorithms
Assume that the base twoYes
process algorithm satisfies
starvation-freedom. Does the
tournament algorithm also
satisfies starvation-freedom ?
1
Chapter 2
2
3
4
5
6
7
8
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
21
Models, Properties & Complexity
Basic definitions and notations
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
22
Shared Memory Models
P
. . .
P
P
. . .
C
M
P
C
. . .
M
P
M
M
Simple
Shared
Memory
Chapter 2
P
Coherent
Caching
Distributed
Shared
Memory
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
23
Properties & complexity
 Time complexity
Fast
 Adaptive
Fairness
 FIFO, ...
Fault-tolerance
Local spinning
Space complexity
Communication primitives






Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
24
Complexity Measures
 Counting steps
contention-free time complexity
 process time complexity
 process step complexity
 Counting time units
 system response time
 process response time
 Counting communications
 distributed shared memory
 coherent caching
 Space complexity

Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
25
Contention-free time complexity
What is the contention-free time complexity?
Thread 0
flag[0] = true
turn = 1
while (flag[1] and turn = 1)
{skip}
Thread 1
flag[1] = true
turn = 0
while (flag[0] and turn = 0)
{skip}
flag[0] = false
flag[1] = false
critical section
critical section
4
(3 in the entry + 1 in the exit)
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
26
Process time complexity
 Informal: The longest black interval,
P1
P2
counting steps.
 Formal: The max # of (shared memory)
steps a winning process need to take in
its entry/exit sections since the last
time some process released its critical
section.
time
 Process step complexity: # of steps,
since it has started…
Given an algorithm divide it steps …
• A – entry/exit steps;
• B – critical section steps;
• C – other steps (remainder).
Chapter 2
A single
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
run
27
Process time complexity
What is the process time/step complexity?
Thread 0
flag[0] = true
turn = 1
while (flag[1] and turn = 1)
{skip}
Thread 1
flag[1] = true
turn = 0
while (flag[0] and turn = 0)
{skip}
flag[0] = false
flag[1] = false
critical section
critical section
Theorem: There is no two-process mutual exclusion with an
upper bound on the process time/step complexity. [1992]
∞
Proof: see Section 3.2.5 .
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
28
System response time
P1
P2
one
time
unit
time
A single run
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
29
System response time
 Informal: The longest black interval,
P1
P2
counting time units.
 Formal: The longest time interval
where some process is in its entry code
while no process is in its CS, assuming
an upper bound of one time unit for
step time, and no lower bound.
time
Given an algorithm divide it steps …
• A – entry/exit steps;
• B – critical section steps;
• C – other steps (remainder).
Chapter 2
A single
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
run
30
How many time units
P1
P2
1
2
3
time
4
5
6
7
A single run
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
31
How many time units
P1
P2
1
2
3
4
5
6
time
7
8
9
10
A single run
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
32
What is the system response time?
Show a run of Peterson’s algorithm where the
system response time is 5 time units.
Thread 0
flag[0] = true
turn = 1
while (flag[1] and turn = 1)
{skip}
Thread 1
flag[1] = true
turn = 0
while (flag[0] and turn = 0)
{skip}
flag[0] = false
flag[1] = false
critical section
1
Chapter 2
2
3
4
P1
P2
1
2
3
4
5
critical section
5
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
33
A Fast Mutual exclusion Algorithm
Section 2.3
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
34
A Fast Mutual exclusion Algorithm
 Mutual exclusion and deadlock-freedom
 Starvation of individual processes is possible
 fast access
in the absence of contention, only 7
accesses to shared memory are needed
 With contention
 Even if only 2 processes contend, the
winner may need to check all the 0(n)
shared registers
 System response time is of order n time
units
 n+2 shared registers are used

Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
35
Splitter
 At most
n-1 can move right
 At most
n-1 can move down
 At most 1 can
1
2
3
4
5
n
win
win
 In solo run  1 win
n-1
1
right
n-1
down
x=i
if y  0 then go right fi
y=1
if x  i then go down fi
win
Chapter 2
x
y
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
36
Notation
await (condition) == while (not condition) do skip od
Example:
await (x=5) == wait until x=5
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
37
Fast Mutual exclusion Algorithm
code of process i ,
x
y
0
b
i  {1 ,..., n}
1
false
2
false
n
false
false
start: b[i] = true
x=i
right
if y  0 then b[i] = false
await y = 0
goto start fi
y=i
if x  i then b[i] = false
for j = 1 to n do await b[j] = false od
win
if y  i then await y = 0
goto start fi fi
critical section
down
y=0
b[i] = false
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
38
Can we switch the order of the
last two statements?
What is the maximum
number of processes that
can be in their CS at the
same time?
start: b[i] = true
x=i
if y  0 then b[i] = false
await y = 0
goto start fi
y=i
if x  i then b[i] = false
for j = 1 to n do await b[j] = false od
if y  i then await y = 0
goto start fi fi
critical section
b[i] = false
y=0
x
Chapter 2
y
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
39
Fast Mutual exclusion Algorithm
code of process i ,
x
y
0
b
i  {1 ,..., n}
1
false
2
false
n
false
false
start: b[i] = true
x=i
Can we replace the
if y  0 then b[i] = false
order of these two
await y = 0
statements ?
goto start fi
y=i
if x  i then b[i] = false
for j = 1 to n do await b[j] = false od
if y  i then await y = 0
goto start fi fi
critical section
y=0
No. Mutual exclusion
is not satisfied.
b[i] = false
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
40
Can we replace y=0 in line 12 with ... ?
start: b[i] = true
x=i
if y  0 then b[i] = false
await y = 0
goto start fi
y=i
if x  i then b[i] = false
for j = 1 to n do await b[j] = false od
if y  i then await y = 0
goto start fi fi
critical section
y = 0 replace with: if y = i then y = 0 fi
b[i] = false
Yes
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
41
Can we remove await y=0 in line 4 ?
start: b[i] = true
x=i
if y  0 then b[i] = false
remove ?
await y = 0
goto start fi
y=i
if x  i then b[i] = false
for j = 1 to n do await b[j] = false od
if y  i then await y = 0
goto start fi fi
critical section
y=0
b[i] = false
No. Deadlock is possible
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
42
Can we remove await y=0 in line 9 ?
start: b[i] = true
x=i
if y  0 then b[i] = false
await y = 0
goto start fi
y=i
if x  i then b[i] = false
for j = 1 to n do await b[j] = false od
if y  i then await y = 0
remove ?
goto start fi fi
critical section
y=0
b[i] = false
Yes
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
43
Would this work ?
add
start: b[i] = true
and
x=i
if y  0 and y =i then b[i] = false
await y = 0
remove
goto start fi
y=i
if x  i then b[i] = false
for j = 1 to n do await b[j] = false od
if y  i then await y = 0
goto start fi fi
critical section
y=0
No. Mutual exclusion
b[i] = false
is not satisfied.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
44
Schematic for the fast algorithm
Indicate contending
b[i] := true
Contention?
y=0?
yes
Wait until CS is released
no
The last to cross
the barrier!
Barrier
turn := i
Contention?
xi?
yes
Continue only after it is
guaranteed that no one can
cross the barrier
yes
Last to cross the barrier?
y=i?
no
critical section
no
exit code
Chapter 2
Wait until CS is released
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
45
Starvation-free Algorithms
Section 2.4
2.4.1 Basic Definitions (i.e., FIFO)
2.4.2 The Bakery Algorithm
2.4.3 The Black-White Bakery Algorithm
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
46
The mutual exclusion problem
 Mutual Exclusion
 Deadlock-freedom
 Starvation-freedom
remainder
doorway
waiting
entry code
critical
section
 FIFO
exit code
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
47
Section 2.4.2
The Bakery Algorithm
entry
remainder
1
2
3
4
5
n
0
0
0
0
0
0
doorway
1
2
3
2
4
waiting
1
2
3
2
4
CS
1
2
2
exit
1
2
2
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
time
48
Implementation 1
code of process i ,
i  {1 ,..., n}
number[i] = 1 + max {number[j] | (1  j  n)}
for j = 1 to n {
await (number[j] = 0)  (number[j] > number[i])
}
critical section
number[i] = 0
number
1
2
3
4
0
0
0
0
n
0
0
integer
Answer: No! can deadlock
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
49
Implementation 1: deadlock
entry
remainder
1
2
3
4
5
n
0
0
0
0
0
0
doorway
1
2
2
waiting
1
2
2
CS
1
deadlock
exit
1
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
time
50
Implementation 1
code of process i ,
i  {1 ,..., n}
number[i] = 1 + max {number[j] | (1  j  n)}
for j = 1 to n {
await (number[j] = 0)  (number[j] > number[i])
}
critical section
What if we replace > with  ?
number[i] = 0
number
1
2
3
4
0
0
0
0
n
0
0
integer
Answer: does not satisfy mutual exclusion
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
51
Implementation 2
code of process i ,
i  {1 ,..., n}
number[i] = 1 + max {number[j] | (1  j  n)}
for j = 1 to n {
await (number[j] = 0)  (number[j],j)  number[i],i)
// lexicographical order
}
critical section
number[i] = 0
number
1
2
3
4
0
0
0
0
n
0
0
integer
Answer: does not satisfy mutual exclusion
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
52
Implementation 2: no mutual exclusion
remainder
1
2
3
4
5
n
0
0
0
0
0
0
entry
2
doorway
0
1
0
2
0
2
waiting
1
2
2
CS
1
2
2
exit
1
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
time
53
The Bakery Algorithm
code of process i ,
i  {1 ,..., n}
choosing[i] = true
number[i] = 1 + max {number[j] | (1  j  n)}
choosing[i] = false
for j = 1 to n {
await choosing[j] = false
await (number[j] = 0)  (number[j],j)  (number[i],i)
}
critical section
number[i] = 0
choosing
number
Chapter 2
1
2
3
4
n
false
false
false
false
false
false
0
0
0
0
0
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
bits
integer
54
Properties of the Bakery Algorithm
 Satisfies mutex & FIFO.
 The size of number[i] is unbounded.
 Safe registers: reads which are concurrent with
writes may return arbitrary value.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
55
Is the following version correct?
code of process i ,
i  {1 ,..., n}
choosing[i] = true
number[i] = 1 + max {number[j] | (1  j  n)}
choosing[i] = false
for j = 1 to n {
if i > j { await choosing[j] = false }
await (number[j] = 0)  (number[j],j)  (number[i],i)
}
critical section
number[i] = 0
choosing
number
Chapter 2
1
2
3
4
n
false
false
false
false
false
false
0
0
0
0
0
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
bits
integer
57
No
entry
remainder
1
2
3
4
5
n
0
0
0
0
0
0
doorway
0
1
0
2
0
1
waiting
1
2
1
CS
1
2
1
exit
1
Chapter 2
1
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
time
58
Is the following version correct?
code of process i ,
i  {1 ,..., n}
Yes
number[i] = -1
number[i] = 1 + max {number[j] | (1  j  n) , 0}
for j = 1 to n {
await number[j]  -1
await (number[j]  0)  (number[j],j)  (number[i],i)
}
critical section
Can we replace  with = ?
number[i] = 0
choosing
number
Chapter 2
1
2
3
4
n
false
false
false
false
false
false
0
0
0
0
0
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
bits
integer
59
Computing the Maximum #1
code of process i , i  {1 ,..., n}
Correct implementation
number[i] = 1 + max {number[j] | (1  j  n)}
local1 = 0
for local2 = 1 to n do
local3 = number[local2]
if local1 < local3 then local1 = local3 fi
od
number[i] = 1 + local1
number
Chapter 2
1
0
2
0
3
0
4
0
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
n
0
integer
60
Computing the Maximum #2
code of process i , i  {1 ,..., n}
Is the following implementation correct ?
number[i] = 1 + max {number[j] | (1  j  n)}
local1 = i
for local2 = 1 to n do
if number[local1] < number[local2] then local1 = local2 fi
od
number[i] = 1 + number[local1]
number
Chapter 2
1
0
2
0
3
0
4
0
0
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
n
0
integer
61
local1
No
2
?
2
3
4
5
n
remainder
0
0
0
0
0
0
doorway
?1
1
1
waiting
1
1
1
CS
1
1
1
entry
1
exit
Chapter 2
Passed process 1
Waiting for process 2
time
1
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
62
Computing the Maximum #3
code of process i , i  {1 ,..., n}
Is the following implementation correct ?
A difficult question !!!
local1 = i
for local2 = 1 to n do
if number[local1]  number[local2] then local1 = local2 fi
od
number[i] = 1 + number[local1]
This is Problem 2.39
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
63
Section 2.4.3
The Black-White Bakery Algorithm
Bounding the space of the Bakery Algorithm
Bakery (FIFO, unbounded)
The Black-White Bakery Algorithm
FIFO
Bounded space
+ one bit
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
64
The Black-White Bakery Algorithm
color bit
entry
remainder
1
2
3
4
5
n
0
0
0
0
0
0
doorway
0
1
0
2
1
0
2
waiting
1
2
1
2
CS
1
2
1
2
exit
1
2
1
2
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
time
65
The Black-White Bakery Algorithm
Data Structures
1
2
3
4
n
choosing
bits
mycolor
number
bits
{0,1,...,n}
color bit
Chapter 2
{black,white}
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
66
The Black-White Bakery Algorithm
code of process i ,
i  {1 ,..., n}
choosing[i] = true
mycolor[i] = color
number[i] = 1 + max{number[j] | (1  j  n)  (mycolor[j] = mycolor[i])}
choosing[i] = false
for j = 0 to n do
await choosing[j] = false
if mycolor[j] = mycolor[i]
then await (number[j] = 0)  (number[j],j)  (number[i],i) 
(mycolor[j]  mycolor[i])
else await (number[j] = 0)  (mycolor[i]  color) 
(mycolor[j] = mycolor[i]) fi od
critical section
if mycolor[i] = black then color = white else color = black fi
number[i] = 0
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
67
Question
What happens if in the algorithm each process
chooses its identifier instead of choosing 1 + max ?
Would the algorithm be correct? What properties
would it satisfy?
Answer: Incorrect.
Run process i first until it enters its CS; Now run a
process with a smaller id until it also enters its CS.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
68
Question
Does it matter if we change the order of the last two
statements (in the exit code) of the Black-White
Bakery Algorithm?
Answer: Yes, it matters.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
69
Question
Replace the multi-writer bit color
with n single-writer bits.
Bakery
Black-White Bakery
Use single-writer bits only!
Answer: See page 57.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
70
Tight Space Bounds
Section 2.5
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
71
A Simple Observation
A very simple Lower Bound
Theorem: Any deadlock-free mutual exclusion
algorithm for n processes using only SWMR
registers must use at least n such registers.
Proof: Before entering its critical section a process
must write at least once …
Question: Can we do better using MWMR registers ?
Answer: No.
(SWMR == Single Writer Multi Reader)
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
72
Tight Space Bounds
Section 2.5.1
A Lower Bound (very difficult to prove!)
Theorem: Any deadlock-free mutual exclusion
algorithm for n processes must use at least n
shared registers.
Proof:  (see next few slides)
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
73
Definitions
 run – a sequence of events
 run x looks like run y to process p
 process p is hidden in run x
 process p covers register r in run x
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
74
Example
run x looks like run y to process p
•
•
•
•
•
•
run x
p reads 5 from r
q writes 6 to r
p writes 7 to r
q writes 8 to r
p reads 8 from r
q write 6 to r1
r 68
•
•
•
•
•
•
run y
p reads 5 from r
p writes 7 to r
q writes 6 to r
q reads 6 from r
q writes 8 to r
p reads 8 from r
r
8
The values of the shared registers must also be the same
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
75
Example
process p is hidden in run x
•
•
•
•
•
•
p reads 5 from r
q reads 5 from r
p writes 7 to r
q writes 8 to r
p reads 8 from r
q writes 6 from r
r
writes must be overwritten before any other process has
read the value written
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
76
Example
process p covers register r in run x
•
•
•
•
•
•
•
p reads 5 from r
p reads 5 from r
q reads 5 from r
p writes 7 to r
q writes 8 to r
p reads 8 from r
p writes 2 to r
p covers r at
these three
points
r
writes must be overwritten before any other process has
read the value written
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
77
Lemma 2.17
(very simple)
run x looks like run y to
processes in P
P events
only
x
y
then, this is
also a run
z
Proof: By induction on the number of events in (z-x) …
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
78
Lemma 2.18
(very simple)
If p is in its critical section in run z then p is
not hidden in z.
p is in its
critical
section
then, p is
not hidden
z
Proof: Assume to the contrary … by Lemma 2.17 …
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
79
Lemma 2.19
Then, for any p there
exists y such that
x looks like y to p
all the
processes
are hidden
y
x
all processes,
except maybe p,
are in their
remainders
Proof: By induction on the number of steps of
processes other than p in x ...
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
80
Lemma 2.19: Example
5
r
•
•
•
•
•
•
•
•
•
•
•
Chapter 2
w is in its
remainder
w reads 5
q is in its
p reads 5
remainder
q reads 5
p writes 7
q writes 8
q reads 8
w writes 3
w reads 3
p write 9
p in its critical section
p is in its
p exits
 w is hidden  remove w
 important: q is still hidden !!!
 q is hidden  remove q
Formal proof: By induction on the
number of steps of processes
other than p in x ...
remainder
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
81
Lemma 2.20
(simple)
all the
processes
are
hidden
x
Then, for every
p there exists z
only p takes
steps
z
Chapter 2
p covers unique
register
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
82
Lemma 2.20: Proof
By Lemma 2.19: there exists y …
By the deadlock-freedom assumption …
By Lemma 2.17 …
… p is hidden … imp. by Lemma 2.18
x looks like y to p
all processes, except
maybe p, are in their
remainders
all the
processes
are hidden
x
p covers unique
register
z
p is in its critical
section
p is in its critical
section
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
83
Lemma 2.21
(main lemma)
all the
processes
are in their
remainder
x
Then, for every set of
processes P there exists z
only P take
steps
z
P are hidden and
cover |P| distinct
registers
Proof: By induction on k the size of P.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
84
Lemma 2.21
(main lemma)
Proof: By induction on k the size of P.
 Base: k = 0 is trivial.
 Induction hypothesis: Assume that it holds for k  0.
 Step: We prove that it holds for k+1.



let P be a set of processes where |P|= k;
let p be a process not in P.
We show that:
only P+p
take steps
z
Chapter 2
P+p are hidden and
cover |P+p|
distinct registers
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
85
Lemma 2.21
each process in PBy
takes
the induction hypothesis:
one step first; all in
their remainders ; ...
only P take
steps
x
W1
W2
Wi
y1
y2
... yi
only p takes
steps
yj
...
P are hidden and P are hidden? Yes
cover |P| distinct P cover W1 ? Yes
zregisters
z2 p covers w1?zYes
1
i
p is hidden? Maybe
p covers a unique
register w1 W1
(Lemma 2.20) This
Chapter 2
Wj
w1
w2
wi
completes the lower bound
...
equals
P are hidden? Yes
P cover Wj ? Yes
proof
p covers wj? Yes
p is hidden? Yes!
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
...
zj
wj
86
Question
What breaks in the lower bound proof if we change the
definition of “process p is hidden in run x” as follows:
•
•
•
•
•
•
p reads 5 from r
q reads 5 to r
p writes 7 to r
q writes 8 to r
p reads 8 from r
p writes 8 to r
r
Chapter 2
this write does
not change the
value of r!
8
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
87
Tight Space Bounds
Section 2.5.2
An Upper Bound
Theorem: There is a deadlock-free mutual
exclusion algorithm for n processes that uses
n shared bits.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
88
The One-Bit Algorithm
code of process i ,
i  {1 ,..., n}
p3 writes
false
true
1
b
false
2
false
3
false
true
pn writes
false
true
4
false
true
n
false
false
true
bits
critical section
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
89
The One-Bit Algorithm
code of process i ,
i  {1 ,..., n}
repeat
b[i] = true; j = 1;
while (b[i] = true) and (j < i) do
if b[j] = true then b[i] = false; await b[j] =false fi
j = j+1
od
until b[i] = true
for j = i+1 to n do await b[j] = false od
critical section
b[i] = false
b
Chapter 2
1
2
3
4
false
false
false
false
n
false
false
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
bits
90
Question
Does the correctness of the One-bit algorithm
depend on the order in which the bits are read
within the loops ?
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
91
Properties of the One-Bit Algorithm
•
•
•
•
•
Satisfies mutual exclusion and deadlock-freedom
Starvation is possible
It is not fast
It is not symmetric
It uses only n shared bits and hence it is space optimal
• Would it work with safe bits?
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
92
Section 2.5.3
Question
Computing with Infinitely Many Process
Does it follow from the lower bound that there is no
solution, using atomic registers, when the number of
processes is not known in advance or when the number
is infinite ?
No. It follows that in such a case infinitely many
registers are needed. A simple algorithm for infinitely
many processes is presented in Section 2.5.3.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
93
Section 2.5.3
Computing with Infinitely Many Process: The algorithm
1
2
3
4
5
6
7
8
9
10
owner
0
0
0
1
0
1
0
0
1
0
0
0
0
…
other
0
1
10
10
0
1
0
0
0
0
0
0
…
loser
0
0
0
1
0
0
0
0
0
0
0
…
winner
0
0
0
0
1
0
0
1
0
0
0
0
…
process 4 runs alone …
process 4 is the winner
process 3 runs ...
process 3 lost
process 6 runs ...
process 3 lost
This is just one game, we need infinitely many …
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
94
Automatic Discovery of Algorithms
Section 2.6
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
95
Section 2.6
Automatic Discovery of Algorithms
How do we invent algorithms?
eureka
This is one way 
See next slide for another way ...
(maybe they are the same ...)
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
96
Automatic Discovery
Algorithm
Generator
Chapter 2
algorithms
Algorithm
Verifier
correct
algorithms
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
97
System Overview
Algorithm
Generator
algorithms
verification
results
Algorithm
Verifier
correct
algorithms
User Interface
parameters
(# of: processes, bits, lines of code)
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
98
Results
User-defined parameters
Results
Shared Entry Exit Complex Starvation
bits
conditions freedom
2
2
3
3
4
4
4
5
6
6
7
4
6
4
5
6
5
5
1
1
1
1
1
1
1
1
1
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Tested
algorithms
7,196,536,269
846,712,059
25,221,389
1,838,128,995
129,542,873
129,190,403
*900,000,000
*22,000,000
*70,000,000
Correct
alg.
0
66
105
10
480
56
80
106
96
appx.
running
hours
216
39
0.4
47
1
1
12
0.4
1
* This run was stopped after few solutions were found. Not all algorithms were tested.
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
99
Next Chapter
In Chapter 3 we will consider more
advanced solutions to the mutual
exclusion problem using atomic
registers.
-- Gadi
Chapter 2
Synchronization Algorithms and Concurrent Programming
Gadi Taubenfeld © 2014
10
0