Adaptive Content-based

MIDDLEWARE SYSTEMS RESEARCH GROUP

Routing In General Overlay Topologies

Guoli Li, Vinod Muthusamy Hans-Arno Jacobsen Middleware Systems Research Group University of Toronto http://padres.msrg.toronto.edu

MIDDLEWARE SYSTEMS RESEARCH GROUP

Distributed Publish/Subscribe

Advertisement Publisher Publication Subscriber Subscriber Subscription  Applications  Business process execution e.g., BPEL     Business activity monitoring Service discovery and integration …  An acyclic overlay is sensitive to:  Congestion  Broker failures Benefits of a general overlay:   Routing around congestion and failures Handling imbalanced workloads Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Challenges With General Overlays

S X     Subscriptions are routed in loops Brokers receive duplicate subscriptions Subscription copies exacerbate the problem Same problem for publications

Adv 1

1 S 3 5 2 4 6 S

Adv 2

Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Agenda

 Content-based routing protocol for general overlays   Atomic and composite subscriptions Optimal publication routing  Evaluation   Dynamic publication routing Adaptive composite subscription routing Middleware 2008 @ Leuven, Belgium

TID-based Approach

S  Each advertisement is assigned to a unique tree identifier ( TID )

Adv 1

 Each subscription has a TID predicate with a variable X 1 3 5

MIDDLEWARE SYSTEMS RESEARCH GROUP

2 4 6

Adv 2

Middleware 2008 @ Leuven, Belgium

Subscription Routing

S

S: [

class=stock][symbol=*] [TID=$Z ] X

MIDDLEWARE SYSTEMS RESEARCH GROUP At Broker 1:

Adv1: [class=stock][symbol=IBM] [TID=Adv1] Adv2: [class=stock][symbol=HP] [TID=Adv2]

Adv 1

S A1 S A2 1 3 2 4 S matching Adv1:

[

class=stock][symbol=*][TID=Adv1] S matching Adv2: [class=stock][symbol=*] [TID=Adv2] Middleware 2008 @ Leuven, Belgium 5 6

Adv 2

MIDDLEWARE SYSTEMS RESEARCH GROUP

Publication Routing

 Each publication is assigned the TID of its matching advertisement  e.g., p [class, stock][symbol,HP][TID, adv_msg_id ]  Publications are routed:   Fixed TID routing : a publication is routed to subscribers along its advertisement tree.

Dynamic publication routing : a publication may be routed to subscribers across advertisement trees.

Middleware 2008 @ Leuven, Belgium

Fixed TID Routing

 Property  No broker receives duplicate publication messages

Adv 2

P

MIDDLEWARE SYSTEMS RESEARCH GROUP Adv 1

P 1 2 3 4 5

Sub

Middleware 2008 @ Leuven, Belgium X 6

MIDDLEWARE SYSTEMS RESEARCH GROUP

Dynamic Publication Routing

Adv 1

 Publication’s TID is changeable  Routing heuristic  Util = R output / R sending  Property  Changing a publication’s TID while in transit will not change the set of notified subscribers.

Adv 2

P 1 3 5 2 4 6

Sub

X Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Advantages

 Retains the publish/subscribe client interface  Speeds up subscription and publication matching  Avoids duplicate subscriptions and publications  Routes publications dynamically across multiple alternatives  Enables routing around failures, congestion and load imbalances Middleware 2008 @ Leuven, Belgium

Composite Subscription

MIDDLEWARE SYSTEMS RESEARCH GROUP

 A composite subscription consists of atomic subscriptions linked by logical operators (e.g., AND, OR).

AND e.g., CS= {[class=stock][symbol=YHOO][price>12]} AND {[class=stock][symbol=MSFT][price<20]} S1 S2  Composite subscription routing   Topology-based routing Adaptive routing Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Topology-based CS Routing

Adv 3 Adv 2

1 3 2 S2

A2

S1

A1

4 7 CS’ 8 S3

A3

Broker 4 and 8 are the joint point brokers

Adv 1

5 6 9 Middleware 2008 @ Leuven, Belgium CS CS={{S1 AND S2} AND S3} CS’ ={S1 AND S2}

MIDDLEWARE SYSTEMS RESEARCH GROUP

Adaptive CS Routing

 CS’s joint points are determined according to potential publication traffic, bandwidth, latency, etc.

Adv 2 Adv 2

2 2

Adv 1

1 3 CS={S1 AND S2} 1

Adv 1

CS={S1 AND S2} Middleware 2008 @ Leuven, Belgium 3

MIDDLEWARE SYSTEMS RESEARCH GROUP

Cost Model

 Routing cost of CS  RC(CS)) = + +  Subscription cardinality  |P(S)| : The number of matching publications per unit of time.

 |P(S)| =  |P(CS)| = |P(S l )| + |P(S r )| if op = or Broker input queue Matching Engine + Routing Table

subscription dest

symbol=IBM B1 symbol=HP B2

output output

B1 B2

Middleware 2008 @ Leuven, Belgium

Adaptive CS Routing

MIDDLEWARE SYSTEMS RESEARCH GROUP Adv 3 Adv 2

1 2 7 3 CS’ 4 CS’ 8 S3

A3 Adv 1

S2

A2

5 S1

A1

6 9 Middleware 2008 @ Leuven, Belgium CS CS={{S1 AND S2} AND S3} CS’ ={S1 AND S2}

MIDDLEWARE SYSTEMS RESEARCH GROUP

Evaluation

 Setup   Overlays of 32 brokers with different connection degrees Cluster (each node:1.86GHz, 4G) and PlanetLab  Workloads:   Yahoo!Finance stock quote traces http://research.msrg.utoronto.ca/Padres/DataSets  Metrics   End to end notification delay Network traffic Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Dense vs. Sparser Topologies

20% Note: The benefit is not proportional to the connection degree.

Middleware 2008 @ Leuven, Belgium 4%

Higher Publication Rate

MIDDLEWARE SYSTEMS RESEARCH GROUP

stabilized Middleware 2008 @ Leuven, Belgium

Publication Burst

Burst

MIDDLEWARE SYSTEMS RESEARCH GROUP

Middleware 2008 @ Leuven, Belgium

With Broker Failures

MIDDLEWARE SYSTEMS RESEARCH GROUP

2 nd failure 1 st failure Middleware 2008 @ Leuven, Belgium

CS Routing Traffic

MIDDLEWARE SYSTEMS RESEARCH GROUP

Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Conclusions

 Enables routing around failures, congestion and load imbalances  Allows publications routing across alternative paths  Improves the notification delay by 20%  Enables flexible CS routing  Reduces 80% publication traffic  Improves the notification delay by 55%  Simplifies solutions for failure recovery and load balancing Middleware 2008 @ Leuven, Belgium

Questions?

MIDDLEWARE SYSTEMS RESEARCH GROUP

http://padres.msrg.utoronto.ca

P A D R E S

Middleware 2008 @ Leuven, Belgium

More Publishers

MIDDLEWARE SYSTEMS RESEARCH GROUP

Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Effect of Subscriber Distance

Distance Fixed(ms) Dynamic(ms) Improvement 6 Hops 10 Hops 12 Hops Max Diff 47.202

64.477

74.416

57.65% 47.568

52.895

60.598

27.39% -0.78% 17.96% 18.57% Middleware 2008 @ Leuven, Belgium

On PlanetLab

MIDDLEWARE SYSTEMS RESEARCH GROUP

Middleware 2008 @ Leuven, Belgium

CS Delay

MIDDLEWARE SYSTEMS RESEARCH GROUP

Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Faster Matching with TIDs

 Subscriptions are augmented with TIDs only once at the first broker.

 Other brokers can route the subscription based on the TID alone.

 Similar argument applies to publication routing.

Middleware 2008 @ Leuven, Belgium

Advertisement Routing

MIDDLEWARE SYSTEMS RESEARCH GROUP

 Each advertisement forms a spanning advertisement tree  Duplicated advertisements are discarded by brokers  Each advertisement is assigned a unique tree identifier ( TID )  e.g., a [class,eq,stock]……[TID,eq, adv_msg_id ]  Subscription Routing Table (SRT)  A set of [advertisement, last hop] Middleware 2008 @ Leuven, Belgium

MIDDLEWARE SYSTEMS RESEARCH GROUP

Subscription Routing

 Each subscription has a TID predicate with a variable.

 e.g., s [class,eq,stock]……[TID,eq, $X ]  The variable is bound to the TID of a matching advertisement  Publication Routing Table (PRT)  A set of [subscription, {TID, last hop of subscription }] Middleware 2008 @ Leuven, Belgium