Transcript Taming the torrent - Northwestern University

Taming the Torrent
(Can’t P2P and ISPs just get along?)
David R. Choffnes
Fabián E. Bustamante
Northwestern University
SIGCOMM 2008
Peer-to-Peer Systems
Enable a range of important services
– End-system multicast, file sharing,
content distribution, …
Leverage everybody’s resources
– Disk, processing power, bandwidth
Natural scalability, inherent robustness,
high flexibility …
David Choffnes, Taming the Torrent, SIGCOMM 2008
2
The BitTorrent protocol
Over 66% of P2P users & growing
Distribute files through piece
exchanges between collaborating peers
– A file for download is described by a torrent
– Peers sharing content are connected to the same
torrent
Which peers? Trackers provide a
random subset of peers in the torrent
– For a peer’s initial set of connections
– For new connections, as transfer progresses and other
connections are dropped
David Choffnes, Taming the Torrent, SIGCOMM 2008
3
The ISP Perspective
P2P performance - key factor for service
upgrade & selection by users
A major engineering challenge for ISPs
– ≈70% of the Internet traffic
– Random peer connections → growing ISP operation
costs
Internet Video to TV
Internet Video to PC
Users pay flat rate, but ISPs see
increasing, expensive traffic volumes
VoIP
Video
Gaming
P2P
Web/E-mail
Source: Cisco
David Choffnes, Taming the Torrent, SIGCOMM 2008
4
All fun and games until someone gets a subpoena
ISPs shape traffic directed to standard ports
– P2Ps move to dynamic, non-standard ports
ISPs turn to deep-packet inspection to identify
& shape P2P flows
– P2Ps encrypt their connections
ISPs place caches and/or spoofs TCP RST msgs
– Lawsuits + bad publicity
First lesson
– An effective solution needs users to buy in
David Choffnes, Taming the Torrent, SIGCOMM 2008
5
A straightforward approach
ISPs guide peers when choosing neighbors
– Help reduce cross-ISP traffic
First proposed by Bindal et al. (ICDCS 2006) and
Aggarwal et al. (CCR 2007)
More recently – Xie et al. P4P (SIGCOMM 2008)
Two remaining issues
– Assumes P2P users & ISPs trust each other
– Misses incentive for user adoption
6
An alternative, practical approach
CDNs are already collecting the necessary info
Various CDNs, e.g. Akamai, Limelight
– Serve popular sites e.g. NYTimes, Apple, Yahoo …
– Move load from providers to edge of network
How do they do it?
– Using thousands of replica servers worldwide
– Extensive network measurements to inform
replica server selection
– Redirecting web-clients to web content replicas
Reuse CDNs’ net views for peer selection
– How? By simply comparing DNS redirections
7
Reusing CDNs’ network views
Client’s request redirected to “nearby” server
– Client gets web site’s DNS CNAME entry with domain name in CDN
network
– Hierarchy of CDN’s DNS servers direct client to nearby servers
Multiple redirections to find
nearby edge servers
Internet
Hierarchy of CDN
DNS servers
Customer DNS
servers
Web replica servers
(3) servers
Clients and replica
are “nearby”
Client
is given [SIGCOMM
2 web replica’06]
Another web client
(4)
servers (fault tolerance)
Client gets CNAME entry
(2)
with domain name in Akamai
Client requests
translation for Yahoo
LDNS
(5)
(6)
(1)
Web client
David Choffnes, Taming the Torrent, SIGCOMM 2008
8
Reusing CDNs’ network views (Cont)
Hypotheses
– Links between “nearby” hosts cross few ISPs
– If two hosts are close to the same CDN replica
servers, they are close to each other
Advantages
– Requires no additional infrastructure
– Needs no topological information
– Avoids the trust issue between ISPs and P2P
– Reduces cross-ISP traffic …
– while improving users’ transfer performance
David Choffnes, Taming the Torrent, SIGCOMM 2008
9
Deployment
Available as Ono, an extension to a
popular BitTorrent client
First released April 2007
Currently > 195,000 users worldwide
… in hundreds of countries
… observing > 2 million peers per day
… with flows traversing ~ 10,000 ASs
… collecting ~15GB of data per day
David Choffnes, Taming the Torrent, SIGCOMM 2008
10
Coverage
David Choffnes, Taming the Torrent, SIGCOMM 2008
11
Implementation
Ono
– Extension to Azureus/Vuze BitTorrent client (GPL)
– Now also part of a tracker
CDN-based peer selection
– Uses multiple CDN customers (DNS names)
• Only DNS resolution, no content download needed
• Adaptive lookup rates on CDN names
– Exchanges ratio information w/ other Ono peers
– Sends Ono information to supporting trackers
Over 12,000 SLOC (it’s Java)
– 3,000 for data collection, 3,000 for GUI
12
Experimental dataset
Types of data
– Per-connection sampled transfer rates
(> 100 million per day)
– Ping RTTs (> 100 million per day)
– Traceroutes (> 2 million per day)
Figures based on a 2-week study in
December, 2007
– Each data point is the average for a peer
over a 6-hour interval
13
Reducing cross-ISP traffic
Average number of AS hops to reach Onorecommended/random peers
> 30% of paths to Ono-recommended
peers do not leave the AS of origin
Note BT curve includes all peers, either
Ono-recommended or randomly selected
David Choffnes, Taming the Torrent, SIGCOMM 2008
14
Finding nearby peers
Two orders of magnitude difference
And, on average, 31% lower loss rates!
David Choffnes, Taming the Torrent, SIGCOMM 2008
15
Improving transfer performance
Heavy
– Average
performance
by 31%
DSL inTail
England
-- 4/8Mbps
down, improves
only 768Kbps
up
ISP bandwidth allocation policy
brings
bottleneck
todifference
the access
link
~2KB/s
Even when Median
Ono doesn’t
help, is
it allows
BT to naturally select faster peers
David Choffnes, Taming the Torrent, SIGCOMM 2008
16
… with the right bandwidth allocation policy
Romania: 50 Mb/s in metro-area, 4 Mb/s
outside
883% median improvement
David Choffnes, Taming the Torrent, SIGCOMM 2008
17
Helpful ISPs can help themselves
David Choffnes, Taming the Torrent, SIGCOMM 2008
18
Discussion
Scalability/Overhead
– No path probing to locate nearby peers
– Requires only periodic DNS lookups
• Max: 18KB up and 36KB down per day
Practical
– No new infrastructure required
• Ready to use now, even a year ago
– No need for trust between ISPs & P2P
– Performance improvement is the best
incentive for adoption!
Cross-ISP traffic
David Choffnes, Taming the Torrent, SIGCOMM 2008
19
That’s all fine and good, but why not …?
Get rid of peers that are “far away” (?)
– Latency is weakly correlated with path distance
– Requires everyone to measure
Use AS numbers
– AS-level info can be too …
• Fine-grained - Cross-AS != Cross-ISP
• Coarse - Intra-AS could be coast-to-coast
Ask your ISP
– Decisions guided by aggregated traffic policies
– But what about adoption/deployment
cost/operation …?
David Choffnes, Taming the Torrent, SIGCOMM 2008
20
Summary
Ono – A practical & effective way to reduce
P2P networking costs
– Reuses CDNs’ network views
An alternative approach for ISPs?
– Recommend Ono to your P2P users
– Change bandwidth allocation to favor innetwork connections
Part of 3R project - Reduce, Reuse, Recycle
– Reuse/recycle the views of long-running
services in building large distributed systems
David Choffnes, Taming the Torrent, SIGCOMM 2008
21
Extra, Extra…
Network Early Warning System (NEWS)
– Reuse P2P systems’ aggregated views and
natural traffic
– Detect network anomalies locally
– Use corroboration to confirm them
globally
Join the 4,207 users who have already done so!
http://aqualab.cs.northwestern.edu/projects/NEWS.html
David Choffnes, Taming the Torrent, SIGCOMM 2008
22
Fin
David Choffnes, Taming the Torrent, SIGCOMM 2008
23