Transcript Peer to Peer Grids and Collaboration JavaGrande Meeting Palo Alto June 3 2001 Geoffrey Fox Florida State University Department of Computer Science and CSIT (School of Computational.

Peer to Peer Grids
and Collaboration
JavaGrande Meeting
Palo Alto June 3 2001
Geoffrey Fox
Florida State University
Department of Computer Science and
CSIT (School of Computational Science and Information Technology)
400 Dirac Science Library
Tallahassee
Florida 32306-4120
[email protected]
11/6/2015
p2pgridjgjune01
1
Classic Grid Architecture
Database
Database
Composition
Neos
Netsolve
Security
Portal
11/6/2015
Resources
Clients
Middle Tier
Brokers
Service Providers
Portal
p2pgridjgjune01
Users and Devices
2
Peer to Peer Grids

A Computational Grid can be defined as a collection of
computers, on-line instruments, data archives and
networks that are all tied together by a shared set of
services which, when taken together, provide users with
transparent access through interface devices to the
entire set of resources.
– Initial focus to a power users accessing major resources

A P2P Network can be defined as a collection of users,
interface devices, computers, on-line instruments, data
archives and networks that are all tied together by a
shared set of services which, when taken together,
provide everything transparent access to everything
else.
– Initial focus to building communities and access to commodity
resources such as MP3 files
11/6/2015
p2pgridjgjune01
3
Peers
Peer to Peer Network
User
Service
Resource
Routing
User
Service
Resource
Routing
User
Service
Resource
Routing
Peers are Jacks of all Trades linked to “all” peers in community
User
Service
Resource
Routing
11/6/2015
User
Service
Resource
Routing
p2pgridjgjune01
User
Service
Resource
Routing
4
Peer to Peer Grid
User
Service
Resource
Routing
User
Service
Resource
Routing
GMS Routing
User
Service
Resource
Routing
Services
GMS or GES is Grid Message/Event Service
User
Service
Resource
Routing
11/6/2015
User
Service
Resource
Routing
p2pgridjgjune01
Dynamic
Message or
Event
Routing from
Peers or
Servers
User
Service
Resource
Routing
5
Goals and Features of Garnet

Integrate synchronous and asynchronous
collaboration
– Use same (Java) publish/subscribe Message Service to support
both forms of collaboration

Integrate concept of a portal (web interfaces to
applications) with collaboration
– Use same XML object metadata specification GXOS to
manage and share objects

Supports universal access including PDA’s
collaborating with desktops
– PDA Control of device or PDA alternative kiosk access gotten
as a collaborative session between device and PDA
– Object metadata includes rendering information for different
devices
11/6/2015
p2pgridjgjune01
6
Important Capabilities in Initial Garnet

Standard stuff: built in shared display, whiteboard, HearMe
Audio control, quizzes, annotations, chat/IM (Jabber.org)
– Desktop video will be special case of shared display

Record and replay all features of session (SMIL)
– A/V, Presentation, Annotations, Text Chat
Several Specialized Collaborative Shared Export Viewers: JSP,
HTML, Acrobat ..


Initial SVG (Scalable Vector Graphics) Shared Batik Viewer
– 2D Scientific Visualization/Whiteboard
– Macromedia (Flash~SVG) and Adobe (already “all” to SVG)

Initial source of SVG: Convert PowerPoint VML/WMF to SVG
– Gives shared export model for PowerPoint with each client able
to scale independently at high resolution
11/6/2015
p2pgridjgjune01
7
Garnet Architecture Underpinnings





Most software written in Java (EJB); all data
structures defined in XML GXOS
Unified Event Model – assume all systems
communicate with XML based messages (possibly
wrapped); universal rendering – portalML
All entities – client and server devices, users,
programs -- defined in XML – resourceML
All events archived to enable fault tolerance and
replay
GMS -- Garnet(Grid?) Message Service -- extends
JMS to XML not text properties to allow server side
intelligence
– Dynamic Server Clusters
11/6/2015
p2pgridjgjune01
8
Publish/Subscribe Collaboration





Integrate Asynchronous and Synchronous Collaboration
The web is full of objects – web pages sitting on web servers –
and these support asynchronous collaboration
– You post a web page and I later look at it in my own time
Replacing web document by a “CGI script” or servlet (web
interface to program, database etc.) gives general multi-tier
object sharing
This is Publish/Subscribe mechanism
– If add some mechanism (automatic email or word of mouth) to
tell viewing client when new information is posted
– We build on JMS (Java Message Service) as Industry standard for
publish/subscribe systems
Synchronous Collaboration provides “real-time” notification
and automatic update of changed objects
– Is JMS fast enough to do real-time? need “HP”JMS
11/6/2015
p2pgridjgjune01
9
JMS (Java Message Service) Structure in Garnet
Basic primitive is a topic/property labeled queue = JXTA Pipe
JMS Global (distributed)
Event Receptor (Queue)
Subscribe
Subscribe
Publish
HHMS
Convert Events
to JMS
JavaScript
11/6/2015
Java C++
…..
HHMS (Hand Held Message
Service) Optimized for
Performance.
p2pgridjgjune01
10
Performance of Commercial JMS I
Latency (Non-Persistent)
10000
JMQ
milli-seconds
1000
iBus
SonicMQ
100
FioranoMQ
10
1M
16
K
64
K
25
6K
4K
1K
25
6
64
16
4
0
1
Non-persistent as
We do database
backup outside JMS
Message Size (bytes)
11/6/2015
p2pgridjgjune01
11
Performance of Commercial JMS II
Non-Persistent/Non-Durabl e
Number of Topic : 5
pub/sub per Topic : 10
Messages per Second
Message Size (bytes) : 100
1000
800
iBus
FioranoMQ
SonicMQ
600
400
200
0
Server:
Clients:
11/6/2015
pub
sub
Solaris
1 PC
pub
sub
Windows
pub
1 PC
p2pgridjgjune01
sub
Solaris
2 PCs
Non-durable as
our database
copes with late
clients
12
Proposed GMS Model for Messages

All message publication labels and subscription profiles are
defined in XML
Subscribes to all events to get
Database persistence
Subscriber
Profile Objects
Specify Query to
Event Label
Message Queue
Labeled by (XML)
Topic Object
Subscribers
Publishers
11/6/2015
p2pgridjgjune01
13
Peer to Peer P2P Networks



(Synchronous) Collaboration is critical service in P2P
networks
Publish/Subscribe is mechanism we use to establish
who gets what information
Gnutella and JXTA are different implementations
(from JMS) of P2P information propagation
– GMS can be built on top of JXTA or JMS architecture

JXTA like MyXoS identifies the implicit distributed
operating system
–
–
–
–
Both have message queues as primitives
Both have Shell
Both use XML based messages
JXTA Advertisements are similar to GXOS metaobjects
11/6/2015
p2pgridjgjune01
14
Single Server P2P Illusion
Data
base
JMS/GMS
Server
11/6/2015
p2pgridjgjune01
15
Multiple Server P2P Illusion
Data
base
JMS
Server
JMS
Server
Generate “Automatically”
JMS
Server
11/6/2015
p2pgridjgjune01
16
P2P Grid Event Delivery Service





Dynamic Collection of some billion computers each
of which can either generate, route or consume events
Publisher labels events by an (XML) object which is
at simplest a URI but in general a collection of tagvalues
Subscribers issue some sort of XML Query e.g.
deliver all
p2pgrid://garnet/Education/Graduate/ComputerScience/
Syracuse/Spring2001/CPS616/Lecture3/*
Need Secure, High Performance, Efficient (don’t
propagate events further than they need), Fault
Tolerant delivery service
Shrideep Pallickara PhD June 1 2001
11/6/2015
p2pgridjgjune01
17
Some Results – 22 Servers
Servers are
logically but
not necessarily
physically distinct
from clients
11/6/2015
p2pgridjgjune01
18
Match Rates of 10%
11/6/2015
& Server Hop to client = 1
p2pgridjgjune01
19
Match Rates of 50% & Server Hop to client = 4
11/6/2015
p2pgridjgjune01
20
Integration of Hand Held Clients
Shared Export
PDF/HTML/
SVG Viewer
Part of
Jabber IM
JMS/GMS
Server
Adaptor
XML Processor
GMS Processor
11/6/2015
p2pgridjgjune01
Shared Display
Processor
21
Integration of Hand Held Clients

Client Device (machine) Profile stored in GXOS
specifies O/S, default Screen Size modified by user
(person) preferences
– Dynamically updated with connection bandwidth, user updates


Application Profile stored in GXOS and modified by
event stream specifies data delivered by GMS
Adaptor (Personal Server) looks like a conventional
client to GMS and adapts data to specified client/user
specifications
– If PDA “small”, then SVG viewer on “adaptor” and ship
framebuffer to PDA
– “Resize” on PDA handled by adaptor, scrolling by PDA etc.
– Adaptor can process complex XML queries
11/6/2015
p2pgridjgjune01
22
PowerPoint
Shared
Display
PC to PDA
11/6/2015
p2pgridjgjune01
23
Instant Messenger
Sharing PC to PDA
11/6/2015
p2pgridjgjune01
24
Batik Viewer on PC
PowerPoint can be converted to SVG
via Illustrator or Web export
11/6/2015
SVG Sharing PC to PDA
p2pgridjgjune01
25
MetaObject Event-based Computing Paradigm




All entities are MetaObjects defined in universal GXOS
XML Syntax – Meta means GXOS doesn’t really want to
manage Object, just information required to find, render
and share it
– Rendering includes Palm devices as well as PC’s
– Entities are people, computers, data sources, information
from e-mail, scientific visualization to digitized Bible
All actions including object changes are events – all events
are GXOS objects
– e-mail, paging etc. are all GXOS events uniformly
routed/archived etc.
All action is instantiated or recorded in messages which are
events themselves
There is an MetaObject Shell MyXoS with basic Services
(copy, create, collaborate etc.)
11/6/2015
p2pgridjgjune01
26
Structured and Unstructured Data





Two important hierarchies – both labeled by URI
– Categories for Structured metadata – GXOS
resources are labeled with these (GNDI)
– Locations for Structured and unstructured data
protocol://root/x/y/ … CORBA, Java SOAP .. link
stored in metadata
Xlink/Xpath can identify an object component (value of
an element) by hierarchical external location (URL) or
hierarchical internal name (URI) )
Unstructured data discovered by “Web Search”
Structured data by Category look-up or by “Web
Search”
GXOS like P2P Networks (e.g. Napster) labels objects
with metadata (why it is useful) not just cryptic name
as in filesystems
11/6/2015
p2pgridjgjune01
27
Overall Structure of GXOS for a MegaMeeting
Event
Archive
Capabilities
Global Root
Users
Devices
Documents
MegaMeeting
Meeting
Admin
Multimedia
Have a hierarchy of MegaMeetings
(any collection of meetings)
Course, Degree .. Are MegaMeetings
Meeting
Meeting
gxos://Education/University/FSU/CS/PhD/Course/Lecture
11/6/2015
p2pgridjgjune01
28
Semantic Web: URI and Topics etc.





The URI jxtapipe://fred/jim/agatha is
equivalent to
<label objecttype=“topic”
queuetype=“jxtapipe” field1=“fred”
field2=“jim” field3=“agatha” />
The <label /> syntax is more general
URI labeling allows consistency with W3C
Semantic Web and indeed current practice
Everything is then a “resource” in RDF
resource is a labeled object
11/6/2015
p2pgridjgjune01
29
RDF Examples in GXOS I

These are sample MyXoS Shell Statements

Specify value for property in GXOS tree
<rdf:description
about="gndi://gxosroot/resourcename">
<gxos:property rdf:parseType="literal">
somevalue</gxos:property> </rdf:description>

Specify profile by linking between GXOS tree elements
<rdf:description about="gndi://gxosroot/sessionname">
<gxos:userprofile rdf:resource="gndi://uri_of_user"
gxos:customize="sessionspecificstuff" />
</rdf:description>
11/6/2015
p2pgridjgjune01
30
RDF Examples in GXOS II


Specify MyXoS copy command for meta-objects
<rdf:description
rdf:about="gndi://gxosroot/system/bin/cp"
system:source="gxosobject1"
system:destination="gxosobject2" gxos:execute="true“
/>
Specify alternative locations to find all FSU users
<rdf:description
aboutEachPrefix="gndi://gxosroot/users/fsu">
<gxos:metaobjectlocation> <rdf:alt>
<rdf:li resource="http://main_fsuweblocation" />
<rdf:li resource="http://backup_fsuweblocation" />
</rdf:Alt> </ gxos:metaobjectlocation>
</rdf:description>
11/6/2015
p2pgridjgjune01
31
Interface of XML and Java I

How will we teach computing?
– K-4: Internet Access
– Middle School: (Simple) XML Schema interfaced to some
scripting language
– High School: Java as the programming model with Java
classes (for external data) generated

Probably don’t want to specify objects twice
– Start in Java; generate Schema


Or Start with Schema and generate Java
Seems a natural API of computer code to XML
– DOM or SAX XML Parser or
– JDBC like Interface of Java to Database (~XML)
11/6/2015
p2pgridjgjune01
32
Interface of XML and Java II


Suppose we have a quadrillion (1015)
XML objects as say produced by a
physics accelerator per year
(Enterprise GXOS)
Need to combine:
– Search Interface to select nodes of XML
Tree
 Specify URI
 JDBC or Google like Interfaces
– Castor like Interface to map XML into
Java but need to control depth of
conversion from XML into
Database
XML
Choose
And Convert
Software
11/6/2015
p2pgridjgjune01
33
Controlled Conversion XML to Java

Control expansion by either
– recursively descending tree
– Access other nodes not in current sub-tree
– Access “real object” in a possibly different object
model
Node
URI
Another
Node
Child1
Child2
…
ChildN
URI
11/6/2015
p2pgridjgjune01
Another
Object Model
CORBA Java
SOAP …..
34
Current GXOS API Architecture

Initially implement “Personal” GXOS – Information
Repository small enough that we can afford to read all
possibly relevant information into memory and refine
this
– E.g. Support course data for individual faculty

File.xml  XML Object  Java Object and vice versa
– Use Castor to automate XML Schema to Java Object

Primitives Supported Initially
– Get a “leaf Object”
– Get a Collection (Internal Node) – “handle” and self.xml (the
GXOS properties associated with this node)
– List Contents of a collection (recursively)
– Get Contents of a collection (recursively)
11/6/2015
p2pgridjgjune01
35