Transcript Brief Announcement: Practical Summation via Gossip Wesley W. Terpstra, Christof Leng, Alejandro P.

Brief Announcement:
Practical Summation via Gossip
Wesley W. Terpstra, Christof Leng, Alejandro P. Buchmann
Databases and Distributed Systems Group
Technische Universität Darmstadt
Germany
www.dvs1.informatik.tu-darmstadt.de
Input:
every peer has a value x p
Output:
(at least) one peer knows
x
p
p P

Useful in computing many global statistics:
 Network size
 Average utilization

 Load balance (standard deviation)
 Churn (rate of peer replacement)
 Size of stored data
For our system, BubbleStorm, we compute  degi(p)
2
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Sum calculation in peer-to-peer
 Approaches can be compared by
 Message rounds (latency)
 Total messages (bandwidth)
 Parameters: system size (n), accuracy ()
Rounds
Push-Sum (2003, FOCS)
logn  log
Sample&Collide (2006)
logn 
1

Messages

1
nlogn  log 

 
1
n logn
1

n

Random Tour (2006) 
1
n  2
Comp&Spread (2006)
1
1


1


log2
n
2

2
2
n
n log2 n
 algorithm for practical use
 We improve the Push-Sum

3

DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Build on an existing solution
4
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Analogy: Measuring a lake’s volume
5
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Push-Sum visualized
6
Equilibrium: edges carry the same water and fish in both directions
peers have water and fish proportional to degree and clock
Perturbations of equilibrium do not affect water/fish ratio
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Stationary Distribution (Steady State)
7
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Improvement: Big Fish eat smaller fish
8
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Fish eating in the Network
9
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Stationary Distribution (Steady State)
 Round switching
 Once the result is accurate “enough”, restart
 Provides a running estimate on network statistics
 Compensate for message loss
 Prevent adding two of the most aggressive fish
 Save bandwidth for multiple measurements
10
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Other improvements
 Kempe et al. prove correctness with synchronous
model, but conjecture that it works asynchronously
 We validate this claim by simulation
 1 million peers, 5s gossip interval, find network size:
60
Max im um
St d dev .
Minimum
Logarithmic size estimate
50
40
30
20
10
0
27: 00
11
29: 00
31: 00
Time (mm :s s )
33: 00
35: 00
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Synchrony
 Push-Sum is very vulnerable to attack
 Any peer can completely change the result
 This is largely due to the problem statement (sum!)
 Simplistic prevention (bounds) easily defeated
 Introduce too few of the largest fish type  too large
 Switch rounds prematurely  too small & unstable
 What is a useful adversary model for summation?
12
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Open Problem
Thanks for listening!
Questions
www.dvs1.informatik.tu-darmstadt.de