Transcript Gray& Reuter: Locking Concurrency Control Jim Gray Microsoft, Gray @ Microsoft.com Andreas Reuter International University, [email protected] Mon Tue Wed Thur Fri 9:00 Overview TP mons Log Files &Buffers B-tree 11:00 Faults Lock Theory ResMgr COM+ Access Paths 1:30 Tolerance Lock Techniq CICS & Inet Corba Groupware 3:30 T Models Queues Adv TM Replication Benchmark 7:00 Party Workflow Cyberbrick Party 6-7:

Gray& Reuter: Locking
Concurrency Control
Jim Gray
Microsoft, Gray @ Microsoft.com
Andreas Reuter
International University, [email protected]
Mon
Tue
Wed
Thur
Fri
9:00
Overview
TP mons
Log
Files &Buffers
B-tree
11:00
Faults
Lock Theory
ResMgr
COM+
Access Paths
1:30
Tolerance
Lock Techniq
CICS & Inet
Corba
Groupware
3:30
T Models
Queues
Adv TM
Replication
Benchmark
7:00
Party
Workflow
Cyberbrick
Party
6-7: 1
Gray& Reuter: Locking
Concurrency Control:
Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 2
Gray& Reuter: Locking
Why Lock?
• Need isolation (the "I" of ACID):
• Give each transaction the illusion that
there are no concurrent updates
• Hide concurrency anomalies.
• Do it automatically
– (system does not know transaction semantics)
• Goal:
– Although there is concurrency in system
execution is equivalent to some serial execution
of the system
– Not deterministic outcome, just a consistent
transformation
6-7: 3
Gray& Reuter: Locking
The Essentials
• Transactions Conflict if Reads And Writes overlap
OK: DISJOINT RANGE AND DOMAIN
BAD: T1 READS OUTPUTS OF T2
• More formally:
Transaction T has Read Set: R(T)
Write Set: W(T)
• T1 and T2 conflict IFF
W(T2) & (R(T1) U W(T1)) Ø
Or W(T1) & (R(T2) U W(T2)) Ø
• If they conflict, delay one until the other finishes
6-7: 4
Gray& Reuter: Locking
Laws Of Concurrency Control
• First Law of Concurrency Control
Concurrent execution should not cause
application programs to malfunction.
• Second Law of Concurrency Control
Concurrent execution should not have lower
throughput or much higher response times than
serial execution.
6-7: 5
Gray& Reuter: Locking
Concurrency Control: Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 6
Formal Model: Transactions and
Serial(izble) Histories
Gray& Reuter: Locking
• State is a set of name value pairs: DB = {<e,v>}
• Actions are defined on state:
<Ti, read, e> means Ti reads value of entity e
<Ti, write, e> means Ti writes value of entity e
• Each Transaction is a sequence of actions:
Ti = < <Ti, a, e> | i = 1,...,ni>
• Want to “run” a set of transactions: T = {Ti | i= 1,...,n}
• A History is any sequence S such that
each and every Ti is a subsequence of S
• A history is Serial if it is of the form:
Ti,Tj,.....,Tz (i.e. one transaction at a time).
• A history is serializable (isolated) if it is equivalent to a
serial history.
6-7: 7
Formal Mode: Execution History
Equivalence
Gray& Reuter: Locking
• Define the following permutation subgroup:
<Ti READ ei> commutes with <Tj READ ej> (for all i, j)
<Ti WRITE ei> commutes with < Tj {READ|WRITE} ej>
(iff i  j and ei ej)
• Si is Equivalent to Sj if it can be permuted to Sj
• Alternative Definition (inputs and outputs):
• Define the Dependency set of history S D(S):
{<Ti,e,Tj> | <Ti,ai,e> ,...,<Tj,aj,e> is a subsequence of S
and ai = write or aj = write }
(note: some initial and terminal dependencies are also needed)
• Two Schedules are equivalent iff they have same
dependencies
6-7: 8
Transaction Dependency Relations
Gray& Reuter: Locking
• Shows data flow among transactions
T1 READ e,1 T1 WRITE e,2 T1 WRITE e,2
T2 WRITE e,2 T2 READ e,2 T2 WRITE e,3
R
<e,1>
T1
W
<e,1>
T2
<e,1>
<e,2>
W
R
T1
T2
<e,2>
<e,2>
<e,1>
W
W
T1
T2
<e,2>
<e,3>
6-7: 9
The Three Bad
Transaction Dependencies
Gray& Reuter: Locking
Lost Update
T2 READ e,1
T1 WRITE e,2
T2 WRITE e,3
<e,1>
<e,1>
<e,1>
<e,2>
<e,2>
<e,2>
T2 WRITE e,2 T1 READ e,1
T1 READ e,2 T2 WRITE e,2
T2 WRITE e,3 T1 READ e,2
<e,1>
T2
T1
UnRepeatable
Dirty Read Read
T2
T1
<e,2>
<e,3>
<e,2>
<e,3>
<e,1>
<e,1>
<e, 1>
T1
OK
T2
T1
<e,1>
T2
<e,2>
<e,2>
Locks are often used to prevent these dependencies.
6-7: 10
Dependencies Show Dataflows
Among Transactions
Gray& Reuter: Locking
T3
T1
T3
T1
T5
T2
T4
T6
•
•
•
•
•
T5
T2
T4
T6
Two histories are equivalent iff they have the same dependencies
We want only histories equivalent to a serial history.
If T2 depends on data from T1, then T2 ran after T1.
If T4 depends on data from T3, then T4 ran after T3.
This is a wormhole (in time): T4 ran after T4.
T3-> T4 -> T6 ->T3 ->T4
6-7: 11
• Cycles in the dependency graph are bad.
Gray& Reuter: Locking
Locks Cover Actions
Introduce three new actions:
LOCK [READ | WRITE]
UNLOCK
Lock <Ti, LOCK a, e > Covers
<Ti, a', e>
t
t
If
<Ti, a', e> is at or after the lock step
AND
t
No intervening unlock
Not Covered
t
t
Not Covered
AND
t
a is WRITE OR a' IS READ
t
(write covers write and read)
t
Not Covered
t
t
slock
xlock
o1
o2
read
read
write
write
read
unlock
unlock
unlock
o1
o3
o1
o2
o2
o1
o2
6-7: 12
o3
Gray& Reuter: Locking
Well Formed and Two-Phased
Transactions
• Transaction T is well formed if:
All actions of T are covered by a lock
Lock
Unlock
Act
• Transaction T is 2-phase if:
No unlock precedes a lock in T.
(i.e. A T has a LOCK phase and an UNLOCK phase)
GROW
Shrink
• Basic rules:
– Lock everything you read/write
– Hold locks to end-oftransaction
6-7: 13
Gray& Reuter: Locking
Lock Commutativity
<Ti ,LOCK, ei>
commutes with
<Tj ,LOCK, ej>
(if i j, and (ei  ej))
<Ti ,LOCK READ, e> commutes with <Tj ,LOCK READ, e>
(if i  j)
MODE OF LOCK
EXCLUSIVE
SHARE
MODE OF
SHARE
COMP ATI BLE CONFLI CT
CONFLI CT
REQ UEST EXCLUSIVE
CONFLI CT
COMPATIB ILITY
6-7: 14
Gray& Reuter: Locking
LOCKS define LEGAL HISTORIES
History is legal if:
Don't grant incompatible locks to two at once.
If T1 covers e with a WRITE lock,
No T2 covers e at that point in the history (if T1  T2).
Legal & Serial
T1
T1
T1
T1
T1
T1
T2
T2
T2
T2
T2
T2
T2
T1
T1
T1
T1
T1
T1
SLOCK
XLOCK
READ
WRITE
UNLOCK
UNLOCK
SLOCK
READ
XLOCK
WRITE
WRITE
UNLOCK
UNLOCK
Begin
Slock A
Xlock B
Read A
Write B
Commit
Legal & Not Serial
A
B
A
B
A
B
A
A
B
B
B
A
B
T2
T1
T2
T2
T2
T2
T2
T2
T1
T1
T1
T1
T1
T2 Begin
T2 Slock A
T2 Read A
T2 Xlock B
T2 Write B
T2 Rollback
T1
T1
T1
T1
T1
T1
SLOCK
SLOCK
READ
XLOCK
WRITE
WRITE
UNLOCK
UNLOCK
XLOCK
READ
WRITE
UNLOCK
UNLOCK
Begin
Slock A
Xlock B
Read A
Write B
Commit
A
A
A
B
B
B
A
B
B
A
B
A
B
T2 Begin
T2 Slock A
T2 Read A
T2 Xlock B
T2 Write B
T2 Rollback
NotLegal & Not Serial
T1
T1
T2
T2
T2
T2
T2
T2
T2
T1
T1
T1
T1
T
T
T
T
T
T
SLOCK
XLOCK
SLOCK
READ
XLOCK
WRITE
WRITE
UNLOCK
UNLOCK
READ
WRITE
UNLOCK
UNLOCK
Begin
Slock A
Xlock B
Read A
Write B
Commit
A
B
A
A
B
B
B
A
B
A
B
A
B
T' Begin
T' Slock A
T' Read A
T' Xlock B
T' Write B
T' Rollback
6-7: 15
SERIALIZABILITY THEOREM
(Wormhole Theorem)
Gray& Reuter: Locking
1. If all transactions are well formed and 2
then all legal histories are serializable.
2. If T1 is not well formed or not 2
then there is a T2 such that
T1 and T2 have a legal
but not serializable history
except for trivial cases.
6-7: 16
Gray& Reuter: Locking
WF & 2 => No Wormholes
• Consider first unlock in schedule
<Ti, unlock, e>
• Claim: Ti is a “first”
– There is no Tj >> Ti for all j  I (Tj before Ti)
• Suppose not (suppose Tj >> Ti):
–
–
–
–
–
Then Tj accesses some e2
Then Tj unlocks e2 (Tj is WF)
Then Ti locks e2 (Ti is WF)
Then Ti reads or writes e2
So Tj unlock is before Ti unlock (2) =><=
• Contradiction proves Ti is a first.
6-7: 17
Proof of Wormhole Theorem
If no wormhole, then equivalent to serial
Gray& Reuter: Locking
Induction:
If no cycles, then there is a “last” transaction
Permute it to end of history (this will preserve dependencies).
So, equivalent history.
By induction remaining history is equivalent to a serial history.
T1
T2
T1
T2
T2
T2
T1
T2
T1
T3
T3
T3
T3
T3
T4
T4
T4
T4
T4
T3
T1
T3
T2
T1
T3
T2
T3
T2
T3
T2
T1
T2
T1
T4
T4
T4
T4
T4
T3
T1
T3
T2
T1
T3
T2
T3
T4
T2
T3
T2
T1
T4
T2
T4
T4
T4
T1
Original
T4 Moved
T1
T1
T1
T1
T2
T2
T2
T2
T2
T3
T3
T3
T3
T3
T4
T4
T4
T4
T4
T3 Moved
Serial
18
T2 Moved =6-7:
Gray& Reuter: Locking
Proof of serializability theorem
2.1. Not WF = > nonserial
T1 not well formed: <T1, a, e> not covered by a lock.
Construct WF & 2 T2 = <<T2,LOCK WRITE,e>
,<T2, WRITE,e>
,<T2,UNLOCK,e>>
Now the history H:
< ...
T2
e
,<T2,LOCK WRITE,e>
,<T1,a, e>
,<T2, WRITE,e>
,<T2,UNLOCK,e>
,...>
T1
e
e
Is legal, but not equivalent to serial
T1 is "after" T2 and also "before" T2.
6-7: 19
Gray& Reuter: Locking
Proof of serializability theorem
2.1. Not 2 = > nonserial
T1 NOT 2 means <T1,unlock, e1> ... <T1, LOCK, e2
Construct WF & 2 T2: <<T2,LOCK READ, e1>
,<T2, LOCK READ,e2>
,<T2, UNLOCK, e1>
,<T2,UNLOCK, e2>>
Now the history H:
< ...
,<T1,UNLOCK, e1>
,<T2,LOCK READ, e1>
,<T2, LOCK READ,e2>
,<T2, UNLOCK, e1>
,<T2,UNLOCK, e2>>
,<T1,LOCK, e2
,...>
Is legal, but not equivalent to a serial history
T1 is "after" T2 and also "before" T2
e1
e1
T1
T2 e2
e2
6-7: 20
Gray& Reuter: Locking
Restatement of serializability theorem
• Lock everything transaction accesses
• Do not lock after unlock.
• Backout may have to undo a unlock (= lock).
• So do not release locks prior to commit ø1
• Keep exclusive locks (write locks) to commit ø2
6-7: 21
Gray& Reuter: Locking
Serializability Theorems
• Wormhole Theorem:
A history is isolated if, and only if, it has no wormhole
transactions.
• Locking Theorem:
If all transactions are well-formed and two-phase, then any
legal history will be isolated.
• Locking Theorem (converse):
If a transaction is not well-formed or is not two-phase, then
it is possible to write another transaction, such that the
resulting pair is a wormhole.
• Rollback Theorem:
An update transaction that does an UNLOCK and then a
ROLLBACK is not two-phase.
6-7: 22
Gray& Reuter: Locking
Concurrency Control: Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 23
Gray& Reuter: Locking
Isolation Levels
= Degrees of { Isolation | Consistency }
0:
1:
2:
3:
transaction gets short xlocks for writes
(well formed writes not 2Ø, no read locks)
transaction gets no read locks
(well formed and 2Ø writes,)
transaction releases read locks right after read
(well formed with respect to reads
but not 2Ø with respect to reads)
well formed and 2Ø
(= Serializable by previous theorems!!)
Transaction backout prohibits 0.
6-7: 24
Gray& Reuter: Locking
What Do Systems Do?
Most non SQL systems support 2
Most SQL systems default to 3 and allow forms of 1, 2
e.g.: NonStop SQL:
1 = BROWSE
2 ~ STABLE READ
DB2:
2 ~ CURSOR STABILITY
3 ~ REPEATABLE READ
SQL Standard
1 = READ UNCOMMITTED
2 = READ COMMITTED
2.99 = REPEATABLE READ
3 = SERIALIZABLE
6-7: 25
Gray& Reuter: Locking
Isolation Levels Theorem
If others lock 1° or 2°
and I lock 3° then I get 3° (serializable).
Any other trans is before me or after me.
BUT
DB may be corrupted by them.
6-7: 26
Comparison of Isolation Levels
Gray& Reuter: Locking
Issue
Degree 0
Degree 1
Degree 2
Common name
Chaos
Browse
read uncommited
Cursor stability
read committed
Protection
Provided
Committed data
Dirty data
Lock protocol
Trans structure
Concurrency
1+
Lets others run
0 and
No dirty reads
at higher
no lost updates
Rollback supported
isolation
Same
Writes visible Writes visible at
immediately
eot
0, 1, and you
You don't
0 and others do
don't read dirty
overwrite dirty
not overwrite
data
data
your dirty data
1 and set short
Set short
Set long
share locks on
exclusive locks
exclusive locks
data you read
on data you
on data you
write
write
Well-formed
Well-formed
Well-formed wrt
And
Wrt writes
writes
Two-phase wrt
Two-phase wrt
writes
writes
Medium:
Greatest:
Great:
hold few read
only set short
only wait for
locks
write locks
write locks
Degree 3
Isolated
Serializable
Repeatable reads
2+
Repeatable
reads
Same
0,1,2 and others
don't dirty data
you read
1 and set long
share locks on
data you read
Well-formed
and
Two-phase
Lowest:
any data
touched
Locked to 6-7:
eot 27
Gray& Reuter: Locking
Comparison of Isolation Levels
Issue
Degree 0
Degree 1
Degree 2
Medium:
Set R&W
but short R
Degree 3
Overhead
Least:
Small:
short W locks Only write locks
Most:
Set long R&W
Rollback
UNDO may
cascade
Cant rollback
Can Undo
incomplete
transactions
same
Same
System Recovery
Dangerous,
Updates may
be lost and
violate 3°
Apply log in 1°
order
Same
same
Dependencies
None
WW
WW
WR
WW
WR
RW
6-7: 28
Gray& Reuter: Locking
Concurrency Control: Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 29
Gray& Reuter: Locking
The Phantom Detail
• If I try to read hair = "red" and eyes = "blue"
and get not found, what gets locked?
No records have been accessed so no records get locked
• If I delete a record, what gets locked?(the record is gone)
• These are cases of phantom records.
• Predicate locks solve this problem (see below)
• Page Locks (done right) can solve this problem
lock the red hair page and the blue eye page,
prevents others red hair and blue eye inserts & updates
• High volume TP systems use esoteric locking mechanisms:
Key Range Locks: to protect b-trees
Hole Locks: to protect space for uncommitted deletes
6-7: 30
Gray& Reuter: Locking
Predicate Locks
Read and write sets can be defined by predicates
(e.g. Where clauses in SQL statements)
When a transaction accesses a set for the first time,
1. Automatically capture the predicate
2. Do set intersection with predicates of others.
3. Delay this transaction if conflict with others.
Problems with predicate locks:
1. Set intersection = predicate satisfiability is NP complete
(slow).
2. Hard to capture predicates
3. Pessimistic: Jim locks
eye = blue
Andreas locks
hair=red
Predicate says conflict, but DB may not have blue eyed red haired
person.
6-7: 31
Gray& Reuter: Locking
Precision Locks: Lazy Predicate
Locks
Precision locks solve problems 1 & 3:
Check returned records against predicates on
each read/write
Example:
Andreas can't insert/read blue eyes
Jim can't insert/read red hair.
check records as they go by
6-7: 32
Granularity Of Locks
An Engineering Solution To Predicate Locks
Gray& Reuter: Locking
Idea:
Pick a fixed set of predicates
They form a lattice under and, or
This can be represented as a graph
Lock the nodes in this graph
DATABASE
Simple example:
FILE-1
FILE-2
Can lock whole DB, whole file, or just one key value.
Size of lock is called granule.
KEY-A
KEY-A
FILE-3
KEY-A
6-7: 33
Gray& Reuter: Locking
Lock Granularity
Batch wants to lock whole DB
Interactive wants to lock records
How can we allow both granularities?
Intention mode locks on coarse granules
Simple compatibility matrix
Mode
I
S
X
Compatibility Matrix
S hare
I ntent
+
+
-
e X clusive
6-7: 34
Gray& Reuter: Locking
Lock Granularity: refined intent modes
Intent mode locks say locks being set at finer granularity
If only reading at finer granularity then I compatible with S.
Introduce IS: intend to set fine S locks
IX: intend to set fine S or X locks
SIX: S + IX
IS
IX
S
SIX
X
Compatibility Matrix
S
SIX
IS
IX
+
+
+
+
+
+
+
+
+
-
X
- 6-7: 35
Gray& Reuter: Locking
Notes:
Granularity Example
T3 is waiting
T2 has all of file 3 locked shared
Most of file 2 locked shared (Fine granularity)
T1 has record locks in file-1 and file-2
DATABASE
FILE-1
T1:IX, T2:IS
T1:IX
T2:IST3:ST3:S
FILE-2
FILE-3
T1:IX, T2:IS
T1:IX
T2:IS
T1:IX
T1:IX
T2:S
T2:S
T2:S
T2:S
T2:S
T1:X
T2:S
Rules:
KEY-A
T1:X
Lock root to leaf
If set X,S below get IX or IS above
On a DAG (Directed Acyclic Graph)
Get ONE IS,IS,...,S path for reads
Get ALL IX,IX,...,X paths for a write
T1:X
KEY-A
KEY-A
T1:S, T2:S
T1:S
T2:S
6-7: 36
Update Mode Locks (minor tangent)
Gray& Reuter: Locking
Most common form of deadlock
T1 READ A
T2 READ A
T1 UPDATE A
T2 UPDATE A
(lock A shared)
(lock A shared)
(lock A exclusive, wait for T2)
(deadlock A exclusive, wait for T1)
So introduce update mode lock:
IS
IX
S
SIX
U
X
IS
+
+
+
+
-
Compatibility Matrix
SIX
IX
S
+
+
+
+
+
+
-
U
-
X
-
U compatible with S so updators do not hurt readers.
If certain to update record then get x mode lock right away.
6-7: 37
Gray& Reuter: Locking
Escalation
If transaction gets too many locks, system probably guessed
wrong about granularity
Convert fine grain locks to one coarse one
Example:
1000 record locks on table t becomes
1 file lock on table t.
In some contexts, DE-Escalation is best:
Get course grained locks.
Remember fine grain resource names.
On callback:
request fine-grained locks and
de-escalate coarse lock
6-7: 38
Gray& Reuter: Locking
Lock Conversion
If requested lock already held in one
mode,
new mode is: max (old,
requested)
X
SIX
U
IX
S
IS
6-7: 39
Gray& Reuter: Locking
Key Range Locking
(for Phantoms)
Suppose operations are:
Read(key);
/* return current value
Write(key, value); /* set key's value
Delete(key);
/* delete key + value
Read_Next(key); /* returns next key + val
*/
*/
*/
*/
Insert between X and Y must test to see that
No one else cares that [X,Y] was empty, but is
now full
no other concurrent trans did a Read_Next("X");).
6-7: 40
Gray& Reuter: Locking
Key Range Locking (a solution)
Prev-Key Key-Range locking.
Fixed ranges [A,B), [B,C),...., [Z,)
(this is easy)
Dynamic ranges:
If X, Y, Z are in list
....[X, Y), [Y,Z), [Z,)
are the lock ranges. List changes as list changes.
Ranges named by first key in range.
Lock a range when operating on the range
Insert and delete Lock 3 ranges [X. Y), [X, Z), [Y, Z)
Dual Is next key locking.
Best solution is not published.
6-7: 41
Gray& Reuter: Locking
DAG Locking
In general predicate locks and key-range locks form a DAG
not just a tree:
a lock can have many parents.
Blue-eye key range
Blonde-hair key range.
Hierarchical locks work for this.
Read locks any path
Writes lock all paths.
Famous People
Hair Index
Eye Index
T:X
Black
Blue
Brown
Green
Gray
Red
TT
: IX
T’ : IX
: IX, T'
Blonde
Hair
Marilyn
Monroe
Blue Eyes
Don
Red Hair
Auburn
Black
Blonde
Brunette
Platinum
Red
White
Yellow
T' : S
T:X
6-7: 42
Gray& Reuter: Locking
Concurrency Control: Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 43
Basic Synchronization Primitives
(hardware)
Gray& Reuter: Locking
boolean CompareAndSwap(long * cell, long old, long new)
{ if ( *cell == old) { *cell = *new; return TRUE:}
{ *old = * cell; return FALSE} }
value = Load Locked(long * cell);
other stuff;
boolean = StoreConditional(*cell, value);
/* store conditional fails if cell has changed */
Can use these operators to implement:
Shared storage managers
Queues
Semaphores (latches)
6-7: 44
Gray& Reuter: Locking
LOCK
Lock Interface (API)
(name,
,mode
,duration
,wait )
UNLOCK (name
,clear
or not.
- name of resource
- S, X, SIX, IX, IS, U
- instant, short, long
- no, timeout,yes
- name of resource
)- decrement count to zero
Locks must count: if lock twice and unlock once, lock kept
Lock state must be saved (somehow) at
transaction save points (allow rollback)
commit (allow restart to reacquire)
generally, save is implicit (recompute them)
6-7: 45
Gray& Reuter: Locking
Most locks are free most of the time
so only allocate space for a lock if busy
hash table points at lock names
HEADER
FROM
Lock queue:
LOCK NAME
HASH
TABLE
HASH LINK
SEMAPHORE
MODE
WAITS?
QUEUE
NEXT IN
HASH CHAIN
GRANTED GROUP
list of locks
of transaction
T (from Trans
control block)
NEXT IN QUEUE
HEADER
MODE HELD
MODE DESIRED
GRANTED?
DURATION
COUNT
TRANSACTION
next lock of T
WAITING GROUP
Notes: this is a very busy one, most queues = 1 holder
transaction lock queue helps at commit (want to free all locks of T)
semaphores cover hash chain, lock queue (not shown below)
6-7: 46
Gray& Reuter: Locking
LOCK Control Flow
Hash Name & Search for Name
Not Found: Allocate & Format
Header and Body
Exit
Requestor Already Granted To Requestor?
Yes: Requested Mode Compatible With Other Granted?
Yes: Grant, Bump Count, Exit
No: Bump Count, Set Desire, Wait
No: Allocate Queue Element at End
Anyone Waiting?
Yes: Mark Waiting, Wait
No: Compatible With Grantees?
Yes: Grant
No: Wait
Exit
6-7: 47
Gray& Reuter: Locking
UNLOCK Control Flow
Hash Name & Find Name
Find Requestor In Queue
Decrement Count
If Count > 0 Then Exit -- No Change
Dequeue Requestor
If Queue Empty Then
Deallocate Header
Exit
For Each Waiting Conversion
If Compatible With Granted Group
Then Mark Granted & Wakeup
If No More Conversions Waiting Then
For Each Waiter (in FIFO order)
If Compatible With Granted
Then Mark Granted & Wakeup
Else Exit
6-7: 48
Exit
Gray& Reuter: Locking
Concurrency Control: Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 49
Gray& Reuter: Locking
Optimistic Locking
Idea: hope no one else changes data
Timestamp: keep timestamp with each object
defer all updates
at ø1 commit:
lock objects
if timestamps original then abort
else release read locks
at ø2 commit: apply deferred updates
Value:
keep original value of each object
at ø1 commit:
lock objects
if values original then abort
else release read locks
at ø2 commit: apply deferred updates
6-7: 50
Gray& Reuter: Locking
Optimistic Locking: Field Calls
Field call: if predicate then update
predicates:
updates:
Assumption:
single variable query Examples:
SQL update clause
no predicate, just an update ,
(single variable update)
no contention:
update transactions
set count := count + 1
where branch =
:home_branch;
if predicate true, transform applies
good for hot spots
original idea in IMS/FP MSDBS.
at call:
test predicate
if false give diagnostic
else record call in log
typical predicate:
at commit ø1: lock
update inventory
test predicate
set quantity = quantity if false then abort (restart)
:delta
at commit ø2: apply update
where part_no = :part
unlock
and quantity > :delta; 6-7: 51
Gray& Reuter: Locking
Escrow Locking
Schemes that hold locks at end of Ø1 fail to solve hot spot
problem for distributed system since ø1 to Ø2 transition
may take time
Idea: leave value fuzzy,
store minimum and maximum values
Poly-values of (Warren Montgomery's MIT thesis)
Escrow locking of (Pat O’Neil)
Example: I order 100 widgets:
quantity was:
[1000,1000]
now is:
[900, 1000]
I commit now is:
[900, 900]
No locks are held
6-7: 52
Works only for commutative ops
Gray& Reuter: Locking
Versioning vs Locking
(Oracle’s Approach)
Records & Objects have values over time:
R: [T0..T1): V0, [T1…T2) V1, [T2….now) V3.
Value time Ti is transaction commit timestamp.
Writes set conventional locks and transaction sees its own updates
Transaction makes read request at time Tx, return
Value as of that time
Value as of time statement began (Oracle Consistent Read)
Value as of time transaction began (Snapshot Isolation)
(= Oracle Serializable)
There is a whole neat theory here waiting to be worked out:
see “A Critique of SQL Isolation Levels”, SIGMOD 95.
6-7: 53
Gray& Reuter: Locking
Tradeoffs
• Deferred updates are confusing for user (application).
• Value is good if object small (record)
• Timestamp is good if object large (file)
• Field calls more general than either
• Field calls great for SQL systems (automatic)
• Field call commutativity assumption is prone to error
6-7: 54
Gray& Reuter: Locking
Weird Lock Options
Available To Application
Read thru locks:
Read past locks:
wants to see all data,
caller wants to skip locked
even uncommitted updates
items
example: ad hoc query
e.g.,: get next free item in a
Adaptive locking:
queue,
holder of lock notified that
others wait
Notify locks:
holder releases/changes
caller waits for object to change
lock access
e.g.: wait for nonempty queue
or trigger on data change
Bounce locks:
caller never waits (timeout = 0)
example: want any free record.
6-7: 55
Gray& Reuter: Locking
Concurrency Control: Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 56
Gray& Reuter: Locking
Deadlock Detection
Deadlock: a cycle in the wait-for graph
Kinds of waits: database locks
terminal/device
storage
session
server
Correct detection must get complete graph
Not likely, so always fall back on timeout
Model of deadlock shows:
waits are rare
deadlocks are rare2 (very very rare)
virtually all cycles are length 2
so do depth-first search either as soon as you wait
6-7: 57
OR after a timeout
Gray& Reuter: Locking
How To Find a Deadlock
Call each object manager getting his part of
wait-for graph
Do depth-first search (transitive closure) of
graph
If find cycle: pick minimum cost victim
Reference: Obermarck and Beri
6-7: 58
Gray& Reuter: Locking
Model of Deadlock (1)
R:number of objects (locks)
r: objects locked per transaction
N+1: Concurrent Transactions
ASSUME
r
Transaction is: LOCK lock
steps, then commit
Uniform distribution
exclusive locks only
Nr << R
r
rN
Probability a request waits: PW (request )  2  R
2
r
Prob a transaction waits: PW (Transaction)   N
2 R
R
Nxr/2
6-7: 59
Gray& Reuter: Locking
Model of Deadlock (2)
R: number of objects (locks)
r: objects locked per transaction
 1
 2
2
3
N+1: Concurrent Transactions PW  N   PW  N  ...
Probability of cycle
r4 N
PW 2



length 2 + length 3 +...
4R2
N
4
2
Prob transaction deadlocks PD:
r
N
assumes all cycles of length 2

2
4R
System deadlock rate, N+1 times higher
Conclusions: control transaction size and duration
limit multiprogramming
6-7: 60
Gray& Reuter: Locking
Common Performance Bugs
Convoys on semaphores or high-traffic locks
Log semaphore is hotspot
Sequential insert is hotspot
Lock manager costs too much
A good number: 300 instructions for lock+unlock
(no wait case))
= 1500 clocks on a P6/200
file or page granularity locking causes hotspot for
small files
6-7: 61
Gray& Reuter: Locking
Concurrency Control: Outline
•
Why lock (isolation)
•
When to lock (2ø locking, 1°, 2°,3° isolation
•
What to lock (granularity)
•
How to lock
•
Exotics (optimistic, field calls, escrow)
•
Deadlock
•
Performance
6-7: 62