Transcript Increasing Concurrency in Databases using Program Analysis Roman Vitenberg, Kristian Kvilekval and Ambuj Singh University of California, Santa Barbara Ecoop June 15-18 Oslo, Norway.

Increasing Concurrency in
Databases using Program Analysis
Roman Vitenberg, Kristian
Kvilekval and Ambuj Singh
University of California, Santa Barbara
Ecoop June 15-18 Oslo,
Norway
1
Outline
 Motivation
 Shape graphs and prediction
 Predictive scheduler optimizations
 Results
 Conclusions
Ecoop June 15-18 Oslo, Norway
2
Transactional Schedulers
 Most fundamental problem of
concurrency control [Pap86][BHG87]…
 Little has been done on predictive
schedulers


Difficult to extract future usage
Difficult to use it (Optimal Scheduling
NP-complete)
Ecoop June 15-18 Oslo, Norway
3
Program analysis for better
concurrency
 OODB’s increasingly integrated (JDO)
 Shape analysis makes predicting data
accesses more feasible


Transactional code is known in advance
Rich type structure available to analysis
 Enhance schedulers with specific
optimizations based on the extracted info

Deadlock handling, early lock release, adaptive
preclaiming
Ecoop June 15-18 Oslo, Norway
4
Previous and Related Work
 Scheduling in object bases

[Graham&Barker95]
 Shape analysis (pointer analysis)




Escape analysis[Bogda99][Ruf00]
Parallelization[Corbera99]
Type Safety[Ghiya96] [Nurit98]
[Wilhem00]
Others...
Ecoop June 15-18 Oslo, Norway
5
Outline
 Motivation
 Shape graphs and prediction
 Predictive scheduler optimizations
 Results
 Conclusions
Ecoop June 15-18 Oslo, Norway
6
Overview of Prediction
 Predict locking based on program code


Extract program summary in shape graph
Provide runtime system with future accesses
 Advantages



Facilitates integration OODB
Supports complex pointer-based structures
Automatically derived from source code
Ecoop June 15-18 Oslo, Norway
7
Example shape
class Connector{ Part a,b; }
class Part {
Connector left,right,up,down;
Material m
Supplier s;
…
int volume();
}
part
right
part
connector
material
weight=0;
while (part) {
weight+=(part.material.density
*part.volume());
part=part.right.b;
}
Ecoop June 15-18 Oslo, Norway
8
Basic Shape Analysis
 Graph


Abstract locations (heap cells)
Edges labeled with with field names
 Abstract interpretation



Extend graph through field references
Combine graphs when heap location is shared
Merge shapes bottom up through static call
graph
Ecoop June 15-18 Oslo, Norway
9
Predicting with shape
graphs
 Compile Time

Generate shapes for method references



Self, arguments, and global variables
Label shape edges with earliest access
Annotate programs to pass visible references
and shapes to transaction scheduler
 Runtime

Interpret shape graph on the actual object
graph generating expected object set
Ecoop June 15-18 Oslo, Norway
10
Predicting with Shape
Graphs
Object Graph
N:5
a1
F:10
J:10
a2
G:5
a3
o1:0
a4
K:10
F
o2:10
K
N
o4:5
J
o5:15
N
o6:10
J
o3:20
a5
Shape Graph
(o1,a1) » (o4,a1) » (o2,a2) » (o6,a1) » (o5,a4) » (o3,a5)
Ecoop June 15-18 Oslo, Norway
11
Outline
 Motivation
 Shape graphs and prediction
 Predictive scheduler optimizations
 Results
 Conclusions
Ecoop June 15-18 Oslo, Norway
12
Deadlock Handling: Background
 Deadlock detection

Wait-for-graph (WFG)



Nodes are active transactions
Edge (ti,tj) indicates that ti waits for tj to release a lock
Maintaining throughout execution is expensive
 Deadlock prevention

Heuristic-based techniques (wound-wait)


Cheaper but causing unnecessary transaction aborts
Resource allocation graph (infeasible w/o future
knowledge)
Ecoop June 15-18 Oslo, Norway
13
Deadlock handling: our approach
 Shape analysis helps us prune parts of WFG and
other graphs in all graph-based algorithms
Ecoop June 15-18 Oslo, Norway
14
Early lock release: background
 Strict 2PL: all locks are held till the
end of transaction
 Non-strict 2PL: read locks are released
after the last access to the object
 Non-strict allows greater concurrency
but


Difficult to determine the last access
Allows non-serializable execution
Ecoop June 15-18 Oslo, Norway
15
Early lock release: nonsearializable execution
T1 :
Acq(o1)
T2:
read(o1)
Rel (o1)
Acq(o1)
write(o1)
Acq(o2)
write(o2)
Rel(o1,o2)
Acq(o2)
read(o2)
Rel(o2)
Ecoop June 15-18 Oslo, Norway
16
Early lock release: our approach
 Estimate the last access to an object in
order to release early
 Causality-aware scheduler

T1 causally precedes T2 (T1 c T2) if either




T2 is initiated after T1 by the same client, or
T2 acquires a lock that T1 has released, or
There is T3 such that T1 c T3 and T3 c T2
If T1 c T2 the scheduler blocks T2.A(O) if T1
may access O in the future.
Ecoop June 15-18 Oslo, Norway
17
Early lock release: Causalityaware scheduling
T1 :
Acq(o1)
T2:
read(o1)
Rel (o1)
Acq(o1)
write(o1)
Acq(o2)
<- Blocks
write(o2)
Rel(o1,o2)
Acq(o2)
read(o2)
Rel(o2)
Ecoop June 15-18 Oslo, Norway
18
Lock preclaiming: background
 Standard 2PL acquires locks gradually
 Conservative 2PL preclaims all the locks


Fewer aborts upon high contention & long
transactions
Shorter wait-for chains
T1.A(O2)
T2.A(O1)T2.A(O2)
T3.A(O1)

Requires apriori knowledge of locks
Ecoop June 15-18 Oslo, Norway
19
Lock preclaiming: our approach
 Predict the set of objects to be
preclaimed automatically
 Adaptive hybrid schemes


Preclaim when expected contention level
is high across executing transactions
Preclaim locks only for short transaction
Ecoop June 15-18 Oslo, Norway
20
Outline
 Motivation
 Shape graphs and prediction
 Predictive scheduler optimizations
 Results
 Conclusions
Ecoop June 15-18 Oslo, Norway
21
Overview of performance
measurements
 Two applications: OO7 & Prototype
reservation system
 Mix of transaction types (lock sequence,
short/long transactions)
 Transaction rates, and transaction
parameters
 Compare Standard 2PL with our enhanced
2PL based on scheduling delay time
Ecoop June 15-18 Oslo, Norway
22
Evaluation: OO7
•Short Read
•2ms
•600/min
•Short Update
Comp
•2ms
Parts
•10/min
•Long Reorg
•2000ms
•3/min
Ecoop June 15-18 Oslo, Norway
23
OO7 Results
Relative delay by scheduler
ms
100
update
50
query
0
reorg
S2PL
Early Release
Preclaim
Ecoop June 15-18 Oslo, Norway
Both
24
Delays in short query
Strict 2-Phase
Predictive
Ecoop June 15-18 Oslo, Norway
25
Conclusions & future directions
•Runtime use of shape graph can generate object use
prediction (even without prior statistics)
• Transaction scheduler improved by
•Better deadlock handling (smaller WFG)
•Early read lock release
•Lock preclaiming
•Future: lease prediction
Ecoop June 15-18 Oslo, Norway
26
End
Questions?
Ecoop June 15-18 Oslo, Norway
27
Evaluation: Car Reservation
1
2
3
User Rate
300/s
30/s
30/s
User dur
30ms
30ms
30ms
Trav Rate
1/min
1/min
1/min
Trav dur
560ms
560ms
1200ms
Maint rate
6min
2min
2min
Maint dur
6s
6s
6s
Ecoop June 15-18 Oslo, Norway
28
Evaluation: car reservation
Query
60
50
40
30
20
10
0
Traversal
Starved
3423
Starved
S2PL
Pr
5000
Op
0
Conf 1
Conf 2
Conf 3
Conf 1
Ecoop June 15-18 Oslo, Norway
Conf 2
Conf 3
Exec
29
Construction of Shape
Graphs
x
Ecoop June 15-18 Oslo, Norway
30
Construction of Shape
Graphs
x.f = s;
F
Ecoop June 15-18 Oslo, Norway
31
Construction of Shape
Graphs
x.f = s;
t = x.f.g;
F
G
Ecoop June 15-18 Oslo, Norway
32
Construction of Shape
Graphs
x.f = s;
x.f.g = t;
N
F
?
x = x.n;
G
Ecoop June 15-18 Oslo, Norway
33
Construction of Shape
Graphs
N
x.f =
x.f.g
if (x
x =
s;
= t;
!= null)
x.n;
F
G
Ecoop June 15-18 Oslo, Norway
34
Combining Shape Graphs
x = y;
N
F
F
G
K
J
Ecoop June 15-18 Oslo, Norway
35
Combining Shape Graphs
 Unify graphs recursively
N
F
F
G
K
J
Ecoop June 15-18 Oslo, Norway
36
Unification of Shape
Graphs
 Unify graphs recursively
N
N
F
F
G
K
J
F
G
J
K
Ecoop June 15-18 Oslo, Norway
37
Shape Analysis Algorithm
  methods Interpret basic blocks


Create shapes for basic blocks
Run until fixed-point is reached
 Propagate in static callgraph
Ecoop June 15-18 Oslo, Norway
38
Static Call Graphs
Static representation of calls
m3{
a.f = s;
o.m4(a)
}
Class C {
m4(F f) {
…
}
main
m2
m3
m4
Unify(a,f)
f
Ecoop June 15-18 Oslo, Norway
39
Call Graphs
 Propagate bottom up
 Merge polymorphic calls
m1
m2
m3
m4
B.m4
D1.m4
D2.m4
 Recursive Calls


Fixed point
Merge SCC[Ruf00]
m1
m2
m1
Ecoop June 15-18 Oslo, Norway
40
Overview of scheduler
enhancements
 Better Deadlock handling


Smaller WFG based on type information
Conservative Resource Allocation Graph
 Early Lock Release

Read locks released after last use
 Lock Preclaiming

Better throughput in high contention
Ecoop June 15-18 Oslo, Norway
41