Transcript presentation.odp

Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
A Measurement Study of
Piece Population in BitTorrent
Cameron Dale and Jiangchuan Liu
Simon Fraser University
Burnaby, BC, Canada
[email protected]
Globecom, November 29th, 2007, Washington, D.C.
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Overview
● background on some BitTorrent features
● describe the experiment
● results for piece population snapshots and
the evolution of the piece population for
some real BitTorrent swarms*
● PlanetLab simulations of BitTorrent swarms
● discussion and future work
*swarm: a group of peers connected to one another via the
BitTorrent file distribution protocol
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Piece Population
● BitTorrent splits a large file into many pieces
● each piece will have a certain number of
copies in all the peers in the swarm
● ideally, all pieces should have approximately
the same number of copies
● local knowledge limitations prevent the ideal
and introduce an imbalance
● too much variation can lead to inefficiencies
and even starvation
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Rarest-First Policy
● responsible for choosing the next piece that
a peer will download
● each client chooses the piece that it believes
is the rarest one
● choices are made based only on local
knowledge of neighboring peers
● goal is to maintain an even distribution of
pieces throughout the swarm
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Piece Population Distribution
● expect the population to form a distribution
around a mean value
● the mean will be determined solely by the
macro-characteristics: peer arrival rate,
download time, departure rate, etc.
● the width of the distribution will indicate the
effectiveness of the rarest-first policy
– the narrower, the better
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Real Swarm Snapshots
● used a modified BitTorrent client to gather
data from real swarms
● client constantly requests peers from the
tracker* and connects to all returned peers
● client connects to most (90%) peers in a
swarm and collects their piece information
● collection occurs very quickly and is
terminated, usually after less than an hour
*tracker: a server that coordinates communication between
peers attempting to download a file
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Real Swarm Evolution
● collection was done as in the snapshots
● collection continued for many hours, or even
days
● observe changes in the Piece Population as
the swarm evolves
● focussed on the early stages of a swarm
when there are a large number of leechers*
*leecher: a user who does not yet have a complete copy of the
file, and so is currently downloading (and uploading)
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Swarm Characteristics
Torrent Name
KNOPPIX
FreeBSD
Pieces Size (MB) Leechers Clients*
4125
4325
169
10
5699
1494
34
10
mandriva
openSUSE
feisty
openSUSE-2
2803
14805
1387
14977
735
3881
727
3926
89
398
65-120
100-150
9
9
20
18
PlanetLab
PlanetLab-2
1497
1497
784
784
0-340
0-390
340
390
*clients: the number of clients administered by us
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Piece Population Plots
● x-axis shows the number of copies within the
downloaders in the swarm
● y-axis shows the number of pieces that have
that number of copies
● x-axis is normalized to range from 0 to 1, by
the total number of downloaders
● y-axis can be normalized by the number of
pieces, or so that the area under the graph is
1 (to facilitate comparisons)
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Real Internet Swarm Snapshots
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: feisty
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: feisty
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: feisty
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: feisty
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: openSUSE-2
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: openSUSE-2
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: openSUSE-2
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Evolution: openSUSE-2
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulated Swarms
● run on PlanetLab research network testbed
● all peers are controlled by us
● experiment starts with a single seed* and
peers join randomly over first 4 hours
● PlanetLab: identical peers, no peers join
after 4 hours
● PlanetLab-2: distribution of peers, new peers
arriving continuously
*seed: a peer that has the complete file, and is only uploading
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab
● all peers are identical
● peers stay in the system for about 9 hours,
then leave forever
● each peer's maximum number of
connections is limited to 40 to enhance the
local effect
● it takes 9 hours for the initial seed to upload
a single copy of the file
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab-2
● peers have a distribution of download and
upload speeds
● peers download the file, seed for a random
time, then leave and rejoin as a new peer
● arrivals and departures are grouped together
to increase the amount of churn*
● each peer's maximum number of
connections is limited to 80 (default)
*churn: the arrival and departure of peers in a P2P system
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab-2
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab-2
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Simulation: PlanetLab-2
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Rarest-first Policy
● is effective
– results in normal distributions that are narrow
● improves on random piece selection by 3-4x
– recovers quickly (exponentially) from events that
cause a widened distribution
● could be improved
– larger swarms show tails on the high end due to
the limited local knowledge of peers
– churn causes increases in the width of the
population
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Suggested Improvement
● peers knowledge of pieces could be
extended through gossiping
– share all piece information with neighbors
● increased communication cost
– share information on which pieces are the rarest
● piece with the most votes gets downloaded next
● should reduce the size of the tails for the
larger swarms
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
Future Work
● use these measurements to evaluate the
effect of new piece selection strategies
● further analysis of swarms in the presence of
churn to determine the cause of the
increased population width
● work is already under way to create an
analytical piece-level model describing the
piece population of a swarm
– complicated due to the complex interactions
between peers
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent
Introduction
BitTorrent
Experiment
Results
Simulation
Discussion
A Measurement Study of
Piece Population in BitTorrent
Cameron Dale and Jiangchuan Liu
Simon Fraser University
Burnaby, BC, Canada
[email protected]
Globecom, November 29th, 2007, Washington, D.C.
Cameron Dale and Jiangchuan Liu
A Measurement Study of Piece Population in BitTorrent