Brief Announcement: Practical Summation via Gossip Wesley W. Terpstra, Christof Leng, Alejandro P.
Download
Report
Transcript Brief Announcement: Practical Summation via Gossip Wesley W. Terpstra, Christof Leng, Alejandro P.
Brief Announcement:
Practical Summation via Gossip
Wesley W. Terpstra, Christof Leng, Alejandro P. Buchmann
Databases and Distributed Systems Group
Technische Universität Darmstadt
Germany
www.dvs1.informatik.tu-darmstadt.de
Input:
every peer has a value x p
Output:
(at least) one peer knows
x
p
p P
Useful in computing many global statistics:
Network size
Average utilization
Load balance (standard deviation)
Churn (rate of peer replacement)
Size of stored data
For our system, BubbleStorm, we compute degi(p)
2
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Sum calculation in peer-to-peer
Approaches can be compared by
Message rounds (latency)
Total messages (bandwidth)
Parameters: system size (n), accuracy ()
Rounds
Push-Sum (2003, FOCS)
logn log
Sample&Collide (2006)
logn
1
Messages
1
nlogn log
1
n logn
1
n
Random Tour (2006)
1
n 2
Comp&Spread (2006)
1
1
1
log2
n
2
2
2
n
n log2 n
algorithm for practical use
We improve the Push-Sum
3
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Build on an existing solution
4
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Analogy: Measuring a lake’s volume
5
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Push-Sum visualized
6
Equilibrium: edges carry the same water and fish in both directions
peers have water and fish proportional to degree and clock
Perturbations of equilibrium do not affect water/fish ratio
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Stationary Distribution (Steady State)
7
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Improvement: Big Fish eat smaller fish
8
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Fish eating in the Network
9
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Stationary Distribution (Steady State)
Round switching
Once the result is accurate “enough”, restart
Provides a running estimate on network statistics
Compensate for message loss
Prevent adding two of the most aggressive fish
Save bandwidth for multiple measurements
10
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Other improvements
Kempe et al. prove correctness with synchronous
model, but conjecture that it works asynchronously
We validate this claim by simulation
1 million peers, 5s gossip interval, find network size:
60
Max im um
St d dev .
Minimum
Logarithmic size estimate
50
40
30
20
10
0
27: 00
11
29: 00
31: 00
Time (mm :s s )
33: 00
35: 00
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Synchrony
Push-Sum is very vulnerable to attack
Any peer can completely change the result
This is largely due to the problem statement (sum!)
Simplistic prevention (bounds) easily defeated
Introduce too few of the largest fish type too large
Switch rounds prematurely too small & unstable
What is a useful adversary model for summation?
12
DATABASES AND DISTRIBUTED SYSTEMS
TECHNISCHE UNIVERSITÄT DARMSTADT
Open Problem
Thanks for listening!
Questions
www.dvs1.informatik.tu-darmstadt.de