Transcript - A Context Interchange for Dynamic Services daptability, extensibility, scalability analysis

Context Interchange for Dynamic Services
- Adaptability, extensibility, scalability analysis
Hongwei (Harry) Zhu
Stuart Madnick
MIT Sloan School of Management
{mrzhu, smadnick}@mit.edu
http://interchange.mit.edu/coin
WEB ‘04, December 11, 2004, Washington, D.C.
Characteristics of services
Large number of sources
– Online travel services
– Comparison shopping services
Diverse user needs
– Increasing usability with personalization
Cannot establish a single data standard
Must get semantics right
– Adaptability, extensibility, scalability
2
Motivating example
Online comparison shopping
– 400 vendor sources in different countries; 270 potential contexts
– Different semantic assumptions in data
– Compare prices in the context of any source chosen by the user
Semantic aspect Number of distinctions
Currency
10 different currencies
Scale factor
3 different scale factors: 1, 1K, 1M
Price definition
3 different definitions: base, base+tax, base+tax+SH
Date format
3 different formats, mm/dd/yyyy, dd-mm-yyyy, yyyy-mm-dd
Need many conversions - 159,600 of them!
3
Desired properties
Adaptability
– Capability of accommodating changes in sources
Extensibility
– Easy to add/remove sources
Scalability
– Effort of enabling interoperation wrt the number of
sources and the size of ontology
– Performance wrt number of sources and the size of
each source (query optimization issue)
Flexibility = Adaptability + Extensibility
4
Interoperate: hard-wired approaches
(a) BFS approach
Brute-force between pair-wise sources
1
(c) Internal standard approach
Adopting a standard
2
3
6
5
Internal
standard
6
5
4
(b) BFC approach
Brute-force between contexts
2
1
1
3
4
2
context_a
currency: ‘KRW’; scaleFactor:1000
kind: base; format: yyyy-mm-dd
5
6
context_b
currency: ‘TRL’; scaleFactor:1e6
kind:base+tax; format: dd-mm-yyyy
3
4
context_c
currency: ‘USD’; scaleFactor:1
kind:base+tax+SH; format: mm/dd/yyyy
5
Interoperate: COIN Approach
Concept:
Length
Meters
f()
meters
feet
Feet
Shared
Ontologies
Source
Context
Conversion
Libraries
Context
Mediator
Receiver
Context
part
length
Context
Transformation
17
Source
Context Management
Administrator
Select partlength
From catalog
Where partno=“12AY”
Select partlength/.3048
From catalog
Where partno=“12AY”
55.79
Auto-composition
of conversions
Receiver
6
Ontology and conversion function
format
temporalEntity
scaleFactor
currency
monetaryValue
kind
price
Legend
is_a relationship
attribute
modifier
basic
context_a
currency: ‘KRW’; scaleFactor:1000
kind: base; format: yyyy.mm.dd
context_b
currency: ‘TRL’; scaleFactor:1e6
kind:base+tax; format: dd-mm-yyyy
taxRate
organization
context_c
currency: ‘USD’; scaleFactor:1
kind:base+tax+SH; format: mm/dd/yyyy
context_d is_a context_b
scaleFactor:1e3
context_e is_a context_d
Format: yyyy-mm-dd
context_f is_a context_c
Kind: base+tax
Example source: src_turkey(Poduct, Vendor, QuoteDate, Price)
x : monetaryValue |
x[cvt (currency, C 2) @ C1, u  v] 
x[currency(C1)  C f ]  x[currency(C 2)  Ct ]  x[tempAttr  T ] 
C2
C2
C2
olsen_( A, B, R, D)  C f  A  Ct  B  T  D 
R[value(C 2)  r ]  v  u * r.
7
Demo – same context
No semantic differences
Meaningful data returned
8
Compose only relevant conversions
(b  e)
(a) Select Vendor, Price From src_turkey Where Product=“Samsung SyncMaster 173P”;
Conversion for scale factor
(b) Select Vendor, QuoteDate, Price From src_turkey Where Product=“Samsung SyncMaster 173P”;
Conversion for scale factor
Conversion for date format
9
Auto-reconciliation for auxiliary source
(b  f)
Introduced because
of context difference
in auxiliary source
10
Detection and explication (ba)
11
Mediated query (b  a)
Date format for receiver
Price definition – remove tax
Currency
Scale factor
Date format for auxiliary source olsen
12
Flexibility and Scalability
Need to update/add many conversion programs
Not
flexible
Approach General case
BFS
N(N-1), N:= number of sources and
receivers
C
BF
n(n-1), n:= number of unique contexts
ETL/GS
2N, N:= number of sources and receivers
1) Worst case:  n (n  1) , ni:= number of
COIN
unique values of ith modifier, m := number
of modifiers in ontology
2)  (n  1) , when equational relationships
exist
3) m, if all conversions can be
parameterized
m
i 1
i
i
m
i 1
Flexible
i
In the example
159,600
72,630
800
1) worst: 108
2) actual number: 5 (3
general
conversions
plus 2 for price)
Update the declarative knowledge base.
 Why other approaches cannot fully benefit from general
purpose conversion?
– the decision whether to invoke the conversion is in the
conversion program
13
How COIN scales
Component conversions are defined for
each modifier
Overall conversions are automatically
composed by abductive reasoning engine
Composition via symbolic equation solver
and a shortest path algorithm
Inheritance enabled
14
Conclusion
Semantic differences cannot be
standardized away
Must be flexible and scalable
COIN is a good solution
– Modularization, declarativeness
– Automatic composition of necessary
conversions
15