Transcript Untitled [aitrc.kaist.ac.kr]

Query Optimization over
Web Services
Utkarsh Srivastava
Jennifer Widom
Kamesh Munagala
Rajeev Motwani
Performance Numbers
Student
Advisor
Percent Contribution
Relative Contribution to Research
100
80
This
Work
60
40
20
0
0
2
1
2
3
4
Time in Program (years)
5
Future Directions (sample)
• Web services with monetary cost
• Web services with unstable response times
(QoS guarantees?)
• Multiple web services for same data
• Caching web-service query results
• More expressive queries, also workflows
• Web service profiling and statistics-tracking
3
First Steps in Big Problem
New Query
Optimization
Problem
4
Our
contribution
Web Services
Standardized way of sharing data and
functionality
• Description and discovery
• Communication
Users/ WSDL,UDDI
Clients
SOAP
5
Data,
Web
Functionality
Services
Example Web Services
Stock symbol
Company info
WS1
Reuters
Stock symbol
Stock activity
WS2
NASDAQ
6
Querying Across Web Services
Get info about all
companies with
high-activity stock
User/
Client
• Easy
• Transparent
• Efficient
• Etc.
7
Stock symbol
Company info
Query
WS1
Reuters
Results
Stock symbol
Stock activity
WS2
NASDAQ
Same Basic Goal as Traditional DBMS
Declarative
Interface
User/
Client
Query
Results
• Easy
• Transparent
• Efficient
• Etc.
8
Database
Management
System
Data
Web Service Management System
User/
Client
• Easy
• Transparent
• Efficient
• Etc.
9
Query
WS1
Web Service
Reuters
Management
Results
System
WS2
NASDAQ
WSMS Architecture
Declarative Interface
WSMS
WS Invocations
Metadata Component
Schema
Web service
mapper
registration
Query +
input data
Client
Results
Query Processing Component
Plan
selection
Plan
execution
Profiling and Statistics Component
Responsetime profiler
10
Statistics
tracker
WS1
WS2
WSn
Running Example
Credit card company wants to send offers to
people with:
a) credit rating > 600, and
b) payment history = “good” on prior credit card
Company has at its disposal:
L : List of potential recipients (identified by SSN)
WS1 : SSN  credit rating
WS2 : SSN  cc number(s)
WS3 : cc number  payment history
11
Plan 1
SSN
WSMS
SSN
1
2
L(SSN)
Client
SSN
2
SSN,cr
WS2
SSNccn
SSN ccn
2 123
2 456
WS3
ccnph
ccn ph
123 bad
456 good
WS1
Filter on cr, keep SSN
Query
Plan
SSN,ccn
SSN,ccn,ph
Filter on ph, keep SSN
Note: Pipelined processing
12
SSNcr
SSN cr
1 500
2 700
Simple Representation of Plan 1
L
13
WS1
WS2
WS3
SSNcr
SSNccn
ccnph
Results
Plan 2
WSMS
SSN
1
2
SSN
L(SSN)
SSN
Client
SSN
2
SSN,cr
Filter on cr, keep SSN
Join
SSN
SSN
SSN,ccn
SSN,ccn,ph
Filter on ph, keep SSN
14
SSNcr
SSN cr
1 500
2 700
WS2
SSNccn
SSN ccn
2 123
2 456
WS3
ccnph
ccn ph
123 bad
456 good
WS1
Simple Representation of Plan 2
SSNcr
WS1
L
15
Results
WS2
WS3
SSNccn
ccnph
Quiz Which plan is better?
Plan 1
Plan 2
L
WS1
WS2
WS3
Results
WS1
L
Results
WS2
WS3
• Cost metric: steady-state throughput
• Assume join is “free”
Plan 1 is never worse
16
Query Optimization Primer
Possible query plans: P1, …, Pn
Data/access statistics: S
Execution cost metric: cost(Pi, S)
GOAL: Find least-cost plan
17
Query Optimization Primer
Possible query plans: P1, …, Pn
Data/access statistics: S
Execution cost metric: cost(Pi, S)
GOAL: Find least-cost plan
18
Queries and Plans
“Select-Project-Join” queries over input data L
and set of web services WS1, …, WSn
Precedence constraints
Output of WSi may be needed as input for WSj
Ex: WS2: SSN  ccn and WS3: ccn  ph
Precedence DAG defines space of query plans
19
Query Optimization Primer
Possible query plans: P1, …, Pn
Data/access statistics: S
Execution cost metric: cost(Pi, S)
GOAL: Find least-cost plan
20
Statistics
1) Web service response times
2) Web service selectivities
New Query
Optimization
Problem
21
Our
contribution
Statistics: Response Times
ri: per-tuple response time of WSi from client
SSN
Client
cr
WS1 SSNcr
r1
• ri ≈ 1/throughput, can be reduced
batching parallel calls
by batching,
(see paper)
New Query
Optimization
Problem
22
• Assume independent response
times within query plans
Our
contribution
Statistics: Selectivities
si: selectivity of WSi
Average # output tuples per input tuple to WSi
including post-filtering in query plan
WS1: SSN  cr, filter cr > 600
If 90% of SSNs have cr > 600 then s1 = 0.9
WS2: SSN  ccn
If on average each SSN has 2 credit cards then s2 = 2.0
New Query
Optimization
Problem
23
• Assume independent
selectivities within query plans
Our
contribution
Query Optimization Primer
Possible query plans: P1, …, Pn
Data/access statistics: S
Execution cost metric: cost(Pi, S)
GOAL: Find least-cost plan
24
Bottleneck Cost Metric
New Query
Optimization
Problem
25
Our
contribution
Bottleneck Cost Metric
Conference Lunch Buffet
Dish 1
Dish 2
Dish 3
Dish 4
Average per-tuple processing time =
response time of slowest (bottleneck) stage in pipeline
Note: selectivities=1 in this example
26
Cost Equation for Plan P
Ri(P): Predecessors of WSi in plan P
• Fraction of input tuples seen by WSi =
• WSi response time per input tuple =
Πj∈Ri(P) sj
(Πj∈Ri(P) sj)•ri
• Bottleneck cost metric:
cost(P) = max1≤i≤n(
(Πj∈Ri(P) sj)•ri )
(assumes WSMS processing is not the bottleneck)
27
Contrast with Sum Cost Metric
cost(P) = ∑1≤i≤n(
(Πj∈Ri(P) sj)•ri )
• Stream filter ordering
• Expensive predicate placement
“Polite” Lunch Buffet
Dish 1
28
Dish 2
Dish 3
Dish 4
Problem Statement
Input:
• Web services WS1, …, WSn
• Response times r1, …, rn
• Selectivities s1, …, sn
• Precedence constraints among web services
Output:
• Web services arranged into a plan P
• P respects all precedence constraints
• cost(P) is minimized
29
No Precedence Constraints
All selectivities ≤ 1
Theorem: Optimal to order linearly by ri
(selectivities irrelevant)
General case
(optimal):
“proliferative”
web services
“selective” web services
ordered by response-time
…
30
join
at
WSMS
Results
With Precedence Constraints
cost(P) = max1≤i≤n(
31
(Πj∈Ri(P) sj)•ri )
With Precedence Constraints
cost(P) = ∑1≤i≤n(
(Πj∈Ri(P) sj)•ri )
Sum cost metric
Student
Advisor
Percent Contribution
• Hard to even obtain a factor O(n) of optimal
100
80
60
40
20
0
0
1
2
3
4
Time in Program (years)
32
5
With Precedence Constraints
cost(P) = max1≤i≤n(
(Πj∈Ri(P) sj)•ri )
Bottleneck (max) cost metric
• Surprisingly, optimal solution in polynomial time
Student
Advisor
Percent Contribution
• O(n5) algorithm in paper
– Add one WS at a time to the plan
– WS chosen by solving a linear program
100
80
60
40
20
0
0
33
1
2
3
4
Time in Program (years)
5
Example Revisited
Plan 1
L
WS1
WS2
WS3
SSNcr
SSNccn
ccnph
SSNcr
Plan 2
WS1
max1≤i≤n(
L
Results
(Πj∈Ri(P) sj)•ri )
Results
WS2
WS3
SSNccn
ccnph
Selective
Proliferative
34
WS2
WS3 Precedence constraint
Implementation
Built prototype WSMS query processor
• Optimizer and execution engine
• Assumes schema issues resolved, statistics provided
• Written in Java and uses Apache Axis
(open-source SOAP implementation)
• Experiments (see paper) validate analytical results
35
Isn’t Problem the Same as … ?
Web Service composition
• Targeted for workflow-oriented applications
• No provably optimal strategies
Parallel/distributed query optimization
• Freedom to place query operators
• Much larger space of execution plans
Data integration, mediators
• For general sources of data
• Optimization of total resource consumption
36
Future Directions (sample)
• Web services with monetary cost
• Web services with unstable response times
(QoS guarantees?)
• Multiple web services for same data
• Caching web-service query results
• More expressive queries, also workflows
• Web service profiling and statistics-tracking
37
Conclusion
New Query
Optimization
Problem
38
Our
contribution
Conclusion
New Query
Optimization
Problem
39
Our
contribution
Student
Advisor
Percent Contribution
Questions?
100
80
60
40
20
0
0
40
1
2
3
4
Time in Program (years)
5