Transcript Peer to Peer Networks including JXTA Euresco Conference at Castelvecchio Pascoli Italy June 16-21 2001 EuroConference on Problem-Solving Environments for Numerical Mathematics, Science and.

Peer to Peer Networks
including JXTA
Euresco Conference at Castelvecchio Pascoli Italy June 16-21 2001
EuroConference on Problem-Solving Environments for Numerical
Mathematics, Science and Engineering Applications
Geoffrey Fox
IPCRES Laboratory for Grid Technology
Computer Science, Informatics, Physics
Indiana University
Bloomington IN
[email protected]
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
1
P2P: Peer-to-Peer Networks
• Peer-to-Peer Networks are considered to be the next "killer
application" for the Internet.
• There are several important technology challenges and
applications that vary from the sublime to the ridiculous.
• Like most such over hyped concepts, P2P is rather loosely
defined and covers a set of rather disparate ideas.
• Perhaps the only common theme is a client oriented view of the
world and P2P can be thought of as Power to the People.
– Clients do most of the work, communicate with each other and
really are the most important computers;
– Servers may be around and even essential but remain
subservient to the clients.
• We will first discuss the most well-known and popular P2P
system, Napster.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
2
P2P References
• General discussions on P2P technology can be found at two good
web sites;
– http://www.openp2p.com from the O'Reilly group and
– http://www.peer-to-peerwg.org/ as an industry working group
originally initiated by Intel.
• There is a remarkable book Peer-to-Peer: Harnessing the Power
of Disruptive Technologies by Andrew Oram, Nelson Minar, Clay
Shirky, Tim O'Reilly (March 15, 2001, O'Reilly & Associates;
ISBN: 059600110X) ;
– This has several articles describing either key systems or key
services
• Disruptive Technology(DT): One that does something (perhaps a
new capability) far better than previous best practice; it does old
things less well at the start but gradually catches up.
– The Web itself is a DT: hyperlinks empower new ways of doing
things. Browsers are (initially) clumsier user interface than
traditional GUI’s.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
3
Napster I
• Shawn Fanning developed the original Napster application
and service in January 1999 while a freshman at
Northeastern University.
• Napster allowed any client to advertise any MP3 files stored
on its disk and choose to download MP3 files from other
clients connected to the Napster server network.
• It is said that Shawn was taking a computer-programming
course at Northeastern, but had to buy a programming book
to build Napster ……..
• Like most good ideas, Napster was designed to solve a real
need - in this case to allow Shawn, a musician himself, to
share his music with his friends on campus.
– A PSE for getting music
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
4
Napster II
• The system has become staggeringly popular. Quoting a legal
opinion from last summer, approximately 10,000 music files
are shared per second using Napster, and every second more
than 100 users attempt to connect to the system and there
will be 75 million Napster users by the end of 2000.
http://news.cnet.com/News/Pages/Special/Napster/napster_patel.html
– Current legal debate has increased instantaneous use
• Napster has some other typical P2P services; Instant Messenger,
chat rooms, "Buddy lists" and information about "hot music"
(music portal) but the key feature is the ability to share files
between any Internet connected consenting clients.
• This is roughly the Web version of NFS (Network File System)
familiar from traditional computing environments.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
5
Napster III: Key Features
• MP3 files are important as a popular digital encoding for audio
• it is straightforward to "rip" files off an audio CD and look up
key meta-data (Artist, Title etc.) in a CDDB database on the Web
(http://www.gracenote.com/).
– Length of songs uniquely identifies CD’s
• The audio and meta-data can be stored and accessed as a single
unit.
• Although a server is used to establish the initial connection, the
file transfer is done efficiently - directly from client to client; a
P2P service.
• This is an improvement over most NFS systems where use of
distributed files is not easy (except possibly for the originator who
named file) as usually all you have is a cryptic filename.
• The added value of meta-data for files lies at the heart of the
Semantic Web - a vision from the W3C Web Consortium related
to P2P. http://www.w3.org/2001/sw/
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
6
Napster Like Services I
• There some 200 available Napster clones to support this
area
http://www.ultimateresourcesite.com/napster/main.htm
• Currently the most popular is Imesh
[http://www.imesh.com], which has some 2 million users
and can share any type of file.
• Some of the best known file sharing systems are
– MojoNation [http://www.mojonation.net],
– Freenet [http://freenet.sourceforge.net/] ,
– Gnutella [http://gnutella.wego.com/]
• These three are not server based like Napster but rather
support waves of software agents expressing resource
availability and interest propagating among an informal
dynamic networks of peers.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
7
Napster Like Services II
• There are many interesting ideas being explored;
– Breaking shared files into many parts to both increase bandwidth
(parallel I/O) and increase security of content as no one site can
access files without cooperation from its peers. This type of
technology is controversial as it makes censorship very hard.
– MojoNation has a load balancing and scheduling algorithm in the
form of micro payments to reward those who contribute most to
the community of peers.
– Gnutella - which is a family of related products -- is usually
described as a P2P search engine as its interface is nearer that of a
search engine than a Web file system.
• So far we have identified some basic P2P services; file registration,
access and search.
• We can categorize other P2P systems in various ways; we will
choose distributed computing, collaboration, and core
technologies.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
8
P2P for Distributed Computing or Web Computing I
• The distributed computing P2P applications are highlighted by the use
of millions of Internet clients to analyze data looking for extraterrestrial
life (SETI@home http://setiathome.ssl.berkeley.edu/ ) and the
• Newer project examining the folding of proteins ( Folding@home
http://www.stanford.edu/group/pandegroup/Cosm/ ).
• These are building distributed computing solutions for a special class of
applications:
– Those that can be divided into a huge number of essentially independent
computations, and a central server system doles out separate work
chunks to each participating client.
– In the parallel computing community, these problems are called
"pleasingly or embarrassingly parallel".
• This approach is included in the P2P category because the computing is
Peer based even though it does not have the "Peer only communication"
characteristic of all aspects of Gnutella and Napster for information
transfer.
• SETI@home and Folding@home are elegantly implemented as screen
savers that you download.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
9
Parabon
• Pure Java model
• Ensures Security
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
10
Entropia Financial Modeling I
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
11
Entropia Financial Modeling II
• Each basic financial instrument can be calculated independently
• Central Server interprets the total simulation
• Make Money or Learn what causes market swings or ….
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
12
Drug
Structure
Simulations
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
13
United Devices also does Drug
Simulation
• Parameter Study: do billions of simulations – each with different parameters
• Search Engine like interface to simulation
• Works as each calculation fits in a PC – a detailed molecular model would
usually not do this
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
14
Performance of Entropia Network
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
15
P2P for Distributed Computing or Web Computing II
• Other projects of this type include:
• United Devices (http://www.ud.com/home.htm based on
SETI@home),
• AppliedMeta (http://www.appliedmeta.com based on well
known Legion project from the University of Virginia),
• Parabon computation (http://www.parabon.com),
• Condor (from Wisconsin http://www.cs.wisc.edu/condor/)
and
• Entropia (http://www.entropia.com/).
• Other applications for this type of system include financial
modeling, bio-informatics, measurement of web server
performance and the scheduling of different jobs to use idle
time on a network of workstations.
• Ian Foster has given a more detailed review of these activities
at http://www.nature.com/nature/webmatters/grid/grid.html
and related them to computational grids
(http://www.gridforum.org).
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
16
Collaboration with P2P Systems
• Collaborative systems form a rather different type of P2P
network.
• We have a community of clients working together and
sharing different Internet resources.
• Probably the Instant Messenger (IM) or various forms of
chat room are the most used capability in this arena.
– We have a set of clients exchanging messages with each
other.
– Unlike the file-sharing case, one typically needs to multi-cast
the same message to multiple clients at the same time and the
best architecture is still active research.
• Groove Networks ( http://www.groove.net/ ) founded by the
creator of Lotus Notes is the best-known P2P collaboration
project and uses relay servers to implement the P2P multicast. Has underlying XML Object database.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
17
Server
Server
Server
Server
Server
Server
Peer to Peer P2P “Illusion” among collaborating clients
For Napster like Services or Collaboration
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
18
Core P2P technologies or services I
• These include P2P management, messaging, security,
client grouping as well as the file or more generally
object registration, discovery and access capabilities
already discussed for Napster.
• Sun Microsystems has two important technology
projects.
– Jini (http://www.sun.com/jini/ ) has a simple model for
dynamic self defining objects which act like Napster
peers and register with distributed servers allowing other
peers to discover and access them.
– JXTA (from juxtaposition http://www.jxta.org ) is a new
project from Bill Joy aiming at core P2P capabilities
including grouping (of the peers) and security.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
19
Core P2P technologies or services II
• Digital Cash or “Digital Reputation” implemented with Digital
Cash is an important area as even in “free systems” need
mechanism to limit abuse of resources
– “Tragedy of the Commons”
• Digital Cash has many very clever implementations
– Public Key based “coins” issued by digital banks – issues of safety
(no overdrafts or forgery) and anonymity (cash is anonymous,
credit cards are not)
– POW (Proofs of Work) to stop DOS (Denial of Service)
• Network Architecture
– Create the Small World effect where only a few hops needed to get
from any A to any B
– Intelligent (bad worse case) versus random (bad average) strategies
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
20
Core P2P technologies or services III
• I expect research and commercial experience to identify
more base services, as we understand better the common
needs of P2P systems
• Management of resources in such a network must be an
important challenge; it is our Nirvana - the Web
Operating System.
• Maybe society can live in a Gallimaufry of unstructured
knowledge swept back and forth by armies of Gnutella
agents.
– However this will not do for what the Gartner Group
(http://www.oreillynet.com/pub/d/547 ) and O'Reilly
term Enterprise P2P needed by a Fortune 500
organization.
– Here we will need to manage structured information
within a dynamic P2P grouping.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
21
http://www.jxta.org April 25 2001
• Network computer maker Sun Microsystems Inc. on
Wednesday opened a toolbox for programmers to build a
decentralized, Napster - like Web that it hopes will speed up
and deepen the Internet.
• Rather than anarchy, Sun expects pockets of programmers
to build a more secure and reliable network that will include
every device imaginable, from computers to kitchen
appliances.
• ``Our goal at the end of this is to build a completely reliable
system from unreliable parts,'' John Gage, chief researcher
at Palo Alto, California-based Sun, said in an interview.
• Sun's formula for peer-to-peer computing, which links
computers directly to one another without relying on a
server to coordinate communication, rests on a small
program that sets out how machines will talk to each other.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
22
JXTA for the Press II
• ``The idea is that entities on the Net can find each other
and then send information back and forth,'' said Sun's
chief scientist, Bill Joy, who heads Project JXTA,
pronounced ''juxta,'' which released the protocol, or
rules, for peer-to-peer networks.
• Sun bets that it will be able to sell more of its Internetbuilding computers as the Web grows on its software,
similar to the strategy it has taken with its Java
programming language.
• Peer-to-peer computing has achieved a kind of cult
celebrity because of the notoriety of Napster and a
number of companies are looking for commercial
applications for the way that it allows individuals to
work directly with each other, avoiding mediation --and
snoops.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
23
JXTA for the Press III
• Microchip maker Intel Corp. for instance, has used
peer-to-peer technology to link employees' PCs, using
the collective power of the herd to design new chips
faster.
• Song-swapping service Napster holds a central index
which points members to others who have the songs they
want, but members swap directly. Other services avoid a
central index, instead passing lists from member to
member in a type of computer gossip.
• But distributed computing, as peer-to-peer is more
formally known, can also ease network bottlenecks by
making it possible to break down big programs into lots
of little ones which can communicate with each other.
• ``The role of jxta will be as a solvent to dissolve the
applications running on the back end into small pieces to
run more quickly and more spread out,'' Gage said.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
24
JXTA for the Press IV
• Sun said that manufacturers could start building jxta
into devices from mobile handsets to refrigerators to get
them on the web in a year or so.
• The tiny layer of jxta layer released on Wednesday
includes protocols for computers to talk to each other, a
way to organize groups within the broader environment,
and ways to follow transactions and keep them secure.
• ``The monitoring and security will ultimately determine
the success of the thing,'' Joy told a Webcast.
• Sun's archrival Microsoft Corp. also is focusing its
energy on making transactions secure with its own plans
to build the infrastructure of an always-on, alwaysconnected Web, although as usual Sun dismissed
Microsoft's efforts.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
25
JXTA Philosophy
•
•
•
•
Keep it familiar: where practical, use stuff that's standard
and has worked before
Leverage experts: engage a variety of experts early and
often
Encourage open development: the design, the specs, the
code, and whatever the result is.
Vision for the software
– promote communication (not isolation) among applications
– develop administrative commands for peers, peer groups,
and groups of peers in the spirit of UNIX pipes and shells
– keep the core small and elegant: make an architectural
distinction between core mechanisms and optional policies
– support multiple platforms and languages, micro devices to
servers
– address security from the beginning.
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
26
Key JXTA Concepts
•
•
•
•
•
Peers, Messages, Groups, Pipes
JXTA Service Advertisements – XML metadata
JXTA Shell
PDA (Java KVM) Implementation
Li Gong: JXTA: A Network Programming
Environment
– IEEE Internet Computing May June 2001
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
27
JXTA P2P Architecture
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
28
JXTA Projects in Categories
• Applications
– Configurator: A GUI configuration tool for the JXTA platform
– Instantp2p:JXTA Demonstration GUI
– Shell: JXTA Command Line Shell for interactive access to the JXTAP2P
platform
• Core JXTA
– Platform: JXTA P2P platform infrastructure building blocks and protocols.
• This project defines the JXTA core P2P building blocks: peers, peer
groups, pipes, codats and peer group policies. Includes specifications
and reference implementations for core JXTA protocols (peer and peer
group discovery, peer group membership, peer group pipes and codat
sharing).
– Security: JXTA P2P Security Project
• JXTA Services
– Cms: JXTA Content Management System
– Monitoring: Monitoring and Metering
– Rvmanager: RendezVous Manager
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
29
JXTA Protocols -- Discovery
• Project JXTA has defined six protocols so far.The
developer community
• may define more over time.
• Peer Discovery Protocol — enables a peer to find
advertisements on other peers, and can be used to find any
of the peer, peer group, or advertisements.
– This protocol is the default discovery protocol for all peer
groups, including the World Peer Group.
– It is conceivable that someone may want to develop a
premium discovery mechanism that may or may not choose
to leverage this default protocol, but the inclusion of this
default protocol means that all JXTA peers can understand
each other at the very basic level.
– Peer discovery can be done with or without specifying a
name for either the peer to be located or the group to which
peers belong.When no name is specified, all advertisements
are returned. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
11/7/2015
30
JXTA Protocols – Resolver, Information, Membership
• Peer Resolver Protocol — enables a peer to send and
receive generic queries to search for peers, peer groups,
pipes, and other information.
– Typically, this protocol is implemented only by those peers
that have access to data repositories and offer advanced
search capabilities.
• Peer Information Protocol — allows a peer to learn about
the capabilities and status of other peers.
– For example, a ping message can be sent to see if a peer is
alive.
– A query can also be sent regarding a peer’s properties
where each property has a name and a value string.
• Peer Membership Protocol — allows a peer to obtain group
membership requirements, to apply for membership and
receive a membership credential along with a full group
advertisement, to update an existing membership or
application credential, and to cancel a membership or an
application credential.
– Authenticators and
security credentials are used to provide 31
11/7/2015
it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01
JXTA Protocols – Binding, Endpoints
• Pipe Binding Protocol — allows a peer to bind a pipe
advertisement to a pipe endpoint, thus indicating where
messages actually go over the pipe.
– In some sense, a pipe can be viewed as an abstract, named
message queue that supports a number of abstract
operations such as create, open, close, delete, send, and
receive.
– Bind occurs during the open operation, whereas unbind
occurs during the close operation.
• Peer Endpoint Protocol — allows a peer to ask a peer
router for available routes for sending a message to a
destination peer.
– For example, when two communicating peers are not
directly connected to each other, such as they are not using
the same network transport protocol or when they are
separated by firewalls or NATs, peer routers respond to
queries with available route information — that is, a list of
11/7/2015
http://aspen.csit.fsu.edu/it2spring01
32
gateways alongit2xmlp2p01
the route.