Transcript Grid Computing in a Commodity World

P2P: Grid-Oriented
Distribution Systems
KCCMG Fall Impact 2005
Lorin Olsen, Sprint Nextel
History of File Sharing
• 1980’s: Corporate file sharing (anyone
remember IBM MVS/BDT???) and initial IPbased file transfer protocols for DoD use
• 1990’s: FTP used for point-to-point file transfers
• 1999: Shawn Fanning releases Napster
• March 2000: Justin Frankel releases Gnutella;
"See? AOL can bring you good things!"
• March 2001: Niklas Zennström and Janus Friis
introduce KaZaA
• Defcon 2001 / CodeCon 2002: Bram Cohen
shows off BitTorrent
Client-Server File Sharing
Napster
•
•
•
•
•
You started the Napster software on your
machine. Your machine became a small
server able to make files available to
other Napster users.
Your machine connected to Napster's
central servers. It told the central servers
which files were available on your
machine. So the Napster central servers
had a complete list of every shared song
available on every hard disk connected
to Napster at that time.
You typed in a query for a song. Let's say
you were looking for the song "Roxanne"
by The Police. Napster's central servers
listed all of the machines storing that
song.
You picked a version of the song from
the list.
Your machine connected to the user's
machine that had that song, and
downloaded the song directly from that
machine.
“Napster’s Architecture,
http://computer.howstuffworks.com/file-sharing1.htm
P2P Philosophy
“Many hands make light work” - Proverb
“Single servants are less powerful then a
single server but the collective of many
servants is more powerful then any single
server” - Daniel Stephen Rule.
P2P Evolution
“For the people that aim to stop P2P, they
have turned a centralized system like
Napster – easily controlled, easily monitored
– into a fully decentralized system in the
form of Kazaa, as well as a fragmented
ecosystem of thousands of centralized
servers through BitTorrent. This was
probably a bad decision.”
Monkey Methods Research Group
P2P Architecture
P2P Dark Side
“Too many cooks spoil the broth” - Proverb
“Arggghhhhh” – The sound Lorin makes
when trying to coordinate the efforts of his
four children
Gnutella
• Distributed query +
lightweight content
distribution
• Clients
–
–
–
–
–
–
BearShare
Gnucleus
LimeWire
Morpheus
WinMX
XoloX
BitTorrent
•
You open a Web page and click on a link for
the file you want.
•
BitTorrent client software communicates with
a tracker to find other computers running
BitTorrent that have the complete file.
•
The tracker identifies the swarm.
•
The tracker helps the client software trade
pieces of the file you want with other
computers in the swarm.
•
If you continue to run the BitTorrent client
software after your download is complete,
others can receive .torrent files from your
computer; your future download rates
improve because you are ranked higher in
the "tit-for-tat" system.
“What BitTorrent Does”; How Stuff Works;
http://computer.howstuffworks.com/bittorrent3.htm
BitTorrent Architecture
•
A single source file within a group
of BitTorrent users, called a
swarm, spreads around pieces of
a film or videogame or TV show
so that everyone has a chunk to
share.
•
After the initial downloading, those
pieces are then uploaded to other
needy users in the swarm.
•
Before long, the swarm has
shared all the pieces, and
everyone has their own complete
source.
Thompson, Clive; “The BitTorent Effect”; Wired Magazine;
January 2005
BitTorrent Terms
•
Peer: A peer is one instance of a BitTorrent client running on a computer on the
Internet that you connect to and transfer data. Usually a peer does not have the
complete file, but only parts of it.
•
Seed: A seed is a peer that has a complete copy of the torrent and still offers it for
upload. The more seeds there are, the better the chances are for completion of the
file.
•
Leech A leech is a peer that does not have a complete copy of the torrent yet. When
downloading is complete, it may stay around and seed the file as a seed so that
others can complete their download. The term leech is also used for peers that have
very poor upload/download ratios or leave the swarm immediately after their
downloads are complete. The leeches usually contribute a majority of the bandwidth
in a swarm.
•
Swarm: Together, all peers sharing a torrent are called a swarm. Six leeches and two
seeds makes a swarm of eight.
•
Tracker: A tracker is a broker service that mediates contacts between peers. The
tracker is not directly involved in the data transfer and does not have a copy of the
file.
The Role of the Tracker
Azureus: A Client in Action
File Distribution
BitTorrent Traffic Patterns
• BitTorrent has taken
an enormous share of
Internet bandwidth
• BitTorrent usage
increases during “offprime” hours (5PM6AM)
Legal Uses of BitTorrent
• The demo of the flight sim X-Plane is offered via
BitTorrent, as well as the World of Warcraft beta and its
patches.
• PlaneShift is a free open-source MMORPG, which uses
BitTorrent for its primary method of distribution.
• The fan-film Star Wars: Revelations is distributing two
DVD images as well as the film by itself via BitTorrent.
• The NetBSD operating system version 1.6.2 and later as
well as most major Linux distributions use BitTorrent as
an alternative way of distributing ISO images of their
releases.
“BitTorrent”; http://en.wikipedia.org/wiki/Bittorrent
References
Cohen, Bram; “Incentives Build Robustness in BitTorrent”; 5/22/2003
Ewing, J & Lowry,T (2004); “It Seemed Like A Good Idea”; Business Week, New York;
Aug 2, 2004, Iss. 3894; pg. 60
Pouwelse, Johan; “The BitTorrent P2P file-sharing system”; 12/18/2004;
http://www.theregister.co.uk/2004/12/18/bittorrent_measurements_analysis/
Zihui Ge, Daniel R. Figueiredo, Sharad Jaiswal, Jim Kurose, Don Towsley; “Modeling
Peer-Peer File Sharing Systems”; IEEE Infocom 2003;