Transcript What we do in Grids e-Science CyberInfrastructure and Peer-to-Peer Networks Los Alamos September 23 2003 Geoffrey Fox Community Grids Lab Indiana University [email protected].

What we do in
Grids e-Science
CyberInfrastructure
and Peer-to-Peer
Networks
Los Alamos
September 23 2003
Geoffrey Fox
Community Grids Lab
Indiana University
[email protected]
What do we do?
• Portals: co-chair of Grid Computing Environment
research group at GGF
• Metadata: co-chair at GGF of Semantic Grid RG
– Apply to Earth Science using GML (Geography mark up)
• Collaboration: built a Web Service based collaboration
environment sharing applications and audio/video
conferencing to desktops and PDA’s
• Web Service model for all applications
• Messaging: Open “Grid Messaging System”
NaradaBrokering linking P2P and (Cellular) Grid
• Autonomic services using managed messages
• Applications
– CrisisGrid – Indiana and openGIS Consortium
– SERVOGrid for Earthquake Science
– Biocomplexity Grid-based Computational Environment
Collage of Portals
Earthquakes – NASA
Fusion – DoE
Computing Info – DoD
Publications -- CGL
NaradaBrokering
Audio/Video
Conferencing Client
Computer
Modem
Minicomputer
Server
Peers
NaradaBrokering Broker
Network
Firewall
Workstation
Laptop computer
Peers
PDA
Audio/Video
Conferencing Client
“GridMPI” v. NaradaBrokering


In parallel computing, MPI and PVM provided “all the features
one needed’ for inter-node messaging
NB aims to play same role for the Grid but the requirements and
constraints are very different
• NB is not MPI ported to a Grid/Globus environment

Typically MPI aiming at microsecond latency but for Grid, time
scales are different
• 100 millisecond quite normal network latency
• 30 millisecond typical packet time sensitivity (this is one audio or video
frame) but even here can buffer 10-100 frames on client (conferencing to
streaming)
• 1 millisecond is time for a Java server to “think”


Jitter in latency (transit time through broker) due to routing,
processing (in NB) or packet loss recovery is important property
Grids need and can use software supported message functions and
trade-offs between hardware and software routing different from
parallel computing
NaradaBrokering

Based on a network of cooperating broker nodes
• Cluster based architecture allows system to scale in size


Originally designed to provide uniform software
multicast to support real-time collaboration linked to
publish-subscribe for asynchronous systems.
Now has several core functions
• Reliable order-preserving “Optimized” Message transport
(based on performance measurement) in heterogeneous
multi-link fashion with TCP, UDP, SSL, HTTP, and will add
GridFTP
• General publish-subscribe including JMS & JXTA and
support for RTP-based audio/video conferencing
• Distributed XML event selection using XPATH metaphor
• QoS, Security profiles for sent and received messages
• Interface with reliable storage for persistent events
Laudable Features of NaradaBrokering







Is open source http://www.naradabrokering.org
Will have client “plug-in” as well as standalone brokers
Will have a discovery service to find nearest brokers
Does tunnel through most firewalls without requiring
ports to be opened
Supports JXTA, JMS (Java Message Service) and more
powerful native mode
Transit time < 1 millisecond per broker
Will have setup and broker network administration
module
NaradaBrokering Naturally Supports



Filtering of events to support different client
requirements (e.g,. PDA versus desktop, slow lines,
different A/V codecs)
Virtualization of addressing, routing, interfaces
Federation and Mediation of multiple instances of Grid
services as illustrated by
• Composition of Gridlets into full Grids (Gridlets are single
computers in P2P case)
• JXTA with peer-group forming a Gridlet



Monitoring of messages for Service management and
general autonomic functions
Fault tolerant data transport
Virtual Private Grid with fine-grain Security model
Grid Messaging Substrate
SOAP+HTTP
GridFTP
RTP ….
Standard client-server
style communication.
Substrate mediated
communication removes
transport protocol
dependence.
Consumer
Consumer
Service
SOAP+HTTP
GridFTP
RTP ….
Service
Messaging Substrate
Any Protocol satisfying QoS
Protocols have become overloaded e.g. MUST use UDP for A/V latency
requirements but MUSTn’t use UDP as firewall will not support ………
Heterogeneous Routing in NB
Satellite
UDP
A
Firewall
HTTP
Software Multicast
NB Brokers

Client Filtering
Mediation in Cellular
Grid using NB as
interface agent
• Build Virtual Private
Grid
Gridlets
Grid formed from
Multiple cells
Fast
Link
B1
Hand-Held
Protocol
Dial-up
Filter
B2
B3
Architecture of Message Layer


Need to optimize not only routing of particular messages but
classic publish/subscribe problem of integrating different
requests with related topics (subscribe to sports/basketball/lakers
and sports)
Related to Akamai, AOL … caching and Server optimization
problem
Hypercube of
NB Brokers (logical
not physical)
N≈100 for
Distance
Education
Scale to
billions of
grid clients?
1-> N Grid Clients
Autonomic Services







In a Web (Grid) Service architecture, the state of any service is defined
by its initial condition and all the messages (including ordering) that it
receives
• This how shared event model of collaboration works
This is a “Finite State Change” model analogous to saving file and
“undo” command in many editors
NB plus a robust store can “guarantee” to save all these messages for
(all) services
This allows one to build both "autonomic data transport" and
"autonomic services" since these services can sustain packet losses in
transport and can also sustain failures of apps/brokers
• archived messages (previous invocations, published events etc) can
be retransmitted to reconstruct state at the service or to correct a
transport error.
Further anomalies in message traffic (such as a publisher or subscriber
are silent) can be detected by NB and signal problems
We are building examples of both scenarios using GridFTP as our data
transport example
We will build a sample autonomic visualization service with detection
of failed servers and brokers
Collaborative SVG Web Service

SVG is W3C 2D Vector Graphics standard and is interesting for
visualization and as a simple PowerPoint like application
• Further SVG is built on W3C DOM and one can generalize results
to all W3C DOM-based applications (“all” in future?)

Apache Batik SVG is Java and open source and so it is practical
to modify it to explore
• Real Applications as a Web Service
• Collaboration as a Web Service
• MVC model and web services with implications for portlets

We use NaradaBrokering and XGSP to control collaboration;
support PDA Cell-phone and desktop clients; are restructuring
Batik as MVC Web Service
• Good progress in all areas see
• http://www.svgarena.org for SVG Games
• http://grids.ucs.indiana.edu/ptliupages/projects/carousel/ for PDA
Web Service Model for Application Development
Data
Resource Facing Ports
Application as a Web service
W3C DOM Semantic Events
Model
Narada
Brokering
User Facing
Ports
Events as
Messages
Rendering as
Messages
Control
Natural in
MVC Model
W3C DOM Raw (UI) Events
W3C DOM User Interface
View
Interrupts in traditional monolithic applications become
“real messages” not directly method calls
Natural for collaboration and universal access
Collaborative
SVG As
Aclients
Web Service
Control flow for collaborative
SVG
From
Collaboration
As a WS
Application as a Web service
Application as a Web service
Events
Rendering
From Master
NaradaBrokering
W3C DOM Events
User Interface
Participating Client
From
Collaboration
As a WS
Application as a Web service
Application as a Web service
Events
Rendering
To Collaborative Clients
W3C DOM Events
User Interface
Master Client
Collaborative SVG Chess
Game in Batik Browser
Players
Observers
XGSP Web Service MCU Architecture
Use Multiple Media servers to scale to many codecs and many
versions of audio/video mixing
Session Server
XGSP-based Control
NaradaBrokering
All Messaging
NB Scales as
distributed
Admire
Web
Services
SIP
H323
Media Servers
Filters
High Performance (RTP)
and XML/SOAP and ..
Access Grid
Gateways convert to uniform XGSP Messaging
NaradaBrokering
Native XGSP
Polycom, Access Grid
and RealVideo views of
multiple streams using
CGL A/V Web Service
integrating SIP and H323
Integration of PDA, Cell phone and Desktop Grid Access
NaradaBrokering Communication




Applications interface to NaradaBrokering through
UserChannels which NB constructs as a set of links between NB
Brokers acting as “waystations” which may need to be
dynamically instantiated
UserChannels have publish/subscribe semantics with XML topics
Links implement a single conventional “data” protocol.
• Interface to add new transport protocols within the
Framework
• Administrative channel negotiates the best available
communication protocol for each link
Different links can have different underlying transport
implementations
• Implementations in the current release include support for
TCP,UDP, Multicast, SSL, RTP and HTTP.
• Supports communication through proxies and firewalls such as
iPlanet, Netscape, Apache, Microsoft ISA and Checkpoint.
Transit Delay (Milliseconds)
Mean transit delay for message samples in
NaradaBrokering: Different communication hops
9
8
7
6
5
4
3
2
1
0
hop-2
hop-3
hop-5
hop-7
100
1000
Message Payload Size (Bytes)
Pentium-3, 1GHz,
256 MB RAM
100 Mbps LAN
JRE 1.3 Linux
Standard Deviation for message samples in NaradaBrokering
Different communication hops - Internal Machines
0.8
hop-2
hop-3
hop-5
hop-7
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1000
1500
2000
2500
3000
3500
Message Payload Size
(Bytes)
4000
4500
5000
Average delays per packet for 50 video-clients
NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms
60
NaradaBrokering-RTP
JMF-RTP
Delay (Milliseconds)
50
40
30
20
10
0
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
Average jitter (std. dev) for 50 video clients.
NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms
8
NaradaBrokering-RTP
JMF-RTP
Jitter (Milliseconds)
7
6
5
4
3
2
1
0
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
4.5
NaradaBrokering-RTP
900 audio clients
JMF-RTP
4
3.5
3
2.5
2
1.5
1
0.5
0
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
Transit delay for message samples in Narada, JXTA and Narada-JXTA
Topology - III (8 routers) Internal Machines
300
250
R
R
R
R
R
R
R
200
R
(a)
R
N
R
150
R
R
R
R
R
R
R
R
R
R
N
NaradaBrokering broker
R
JXTA Rendezvous
(b)
100
R
N
N
50
NaradaBr
Pure JXTA
Narada-JXTA
N
(a)
NR
N
R
N
R
N
JXTA Peer
N
NaradaBrokering client
(c)
R
0
500
N
N
R
1000 1500 2000
2500 3000 3500 4000 4500 5000 5500
R
R
Message Payload Size
N
(Bytes)
(b)
N
N
N
Collaboration and Web Services

Collaboration has
a) Mechanism to set up members (people, devices) of a
“collaborative sessions”
b) Shared generic tools such as text chat, white boards, audiovideo conferencing
c) Shared applications such as Web Pages, PowerPoint,
Visualization, maps, (medical) instruments ….

b) and c) are “just shared objects” where objects
could be Web Services but rarely are at moment
•

We can port objects to Web Services and build a general
approach for making Web services collaborative
a) is a “Service” which is set up in many different
ways (H323 SIP JXTA are standards supported by
multiple implementations) – we should make it a WS
Shared Event Collaboration

All collaboration is about sharing events defining state changes
• Audio/Video conferencing shares events specifying in compressed form
audio or video
• Shared display shares events corresponding to change in pixels of a frame
buffer
• Instant Messengers share updates to text message streams
• Microsoft events for shared PowerPoint (file replicated between clients) as
in Access Grid


Finite State Change NOT Finite State Machine architecture
Using Web services allows one to expose updates of all kinds as
messages
• “Event service” for collaboration is similar to Grid notification service
and we effectively define SDE’s (service data elements) in OGSI

Group (Session) communication service is needed for the
delivery of the update events
• Using Event Messaging middleware makes messaging universal
Global-MMCS 2.0 (1) XGSP MCU



We are building an open source protocol independent Web
Service “MCU” which will scale to an arbitrary number of users
and provide integrated thousands of simultaneous users
collaboration services.
We will deploy it globally and hope to test with later this year.
The function of A/V media server will be distributed using
NaradaBrokering architecture.
• Media Servers mix and convert A/V streams

Open XGSP MCU based on the following open source projects
•
•
•
•
openh323 is basis of H323 Gateway
NIST SIP stack is basis of SIP Gateway
NaradaBrokering is open source messaging from Indiana
Java Media Framework basis of Media Servers
Shared Output Port Collaboration
Collaboration as a WS
Set up Session with XGSP
Web Service Message
Interceptor
F
I
WSDL
R
O
Master
U
Application or
Content source
Web Service
Text Chat
Whiteboard
Multiple
masters
O
F
I
Event
(Message)
Service
WS
Viewer
WS
Display
WS
Viewer
WS
Display
Other
Participants
WS
Viewer
WS
Display
Shared Input Port (Replicated WS) Collaboration
Collaboration as a WS
Set up Session with XGSP
R
U
Web
F
Servic
I
I
e
O
O
F
WS
Viewer
WS
Display
Master
U
Web
F
Servic
I
I
e
O
O
F
Event
(Message)
Service
R
R
U
Web
F
Servic
I
I
e
O
O
F
WS
Viewer
WS
Display
Other
Participants
WS
Viewer
WS
Display
XGSP Conference Control
Framework Components

User session management
• User session management supports user sign-in, user
create/terminate/join/leave/invite-into XGSP sessions.

Application Session Management
• XGSP application session management provides the services
to A/V and data application endpoints and communities,
controlling multipoint A/V RTP and data channels.

Floor Control
• Floor control manages the access to shared collaboration
resources.
vic and RealVideo views of multiple streams
Polycom view of multiple video streams
Performance Test : GlobalMMCS1.0

We conducted extensive performance tests on audio
and video servers.

Video:
• The test shows that our video server is capable of supporting 300
clients if there is only one video sender.
• Video Server Machine : 1.2GHz Intel Pentium III dual CPU,
1GB MEM, RedHat Linux 7.3
Audio:
• Our tests show that audio server can support 5 concurrent
sessions (250 participants in total) without any packet droppings.

• Audio Server Machine: 2.5GHz Pentium 4 CPU, 512MB
Windows XP machine

Scale with logarithmic Broker network
memory,
Comparison between the performance of
NaradaBrokering and JMF
Average delays/packet for 12 (of the 400 total) video-clients.
NaradaBrokering Avg=80.76 ms, JMF Avg=229.23 ms
450
NaradaBrokering-RTP
JMF-RTP
400
350
300
250
200
150
100
50
0
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
Comparison between the performance of
NaradaBrokering and JMF
Average jitter/packet for 12 (of the 400 total) video clients.
NaradaBrokering Avg=13.38 ms, JMF Avg=15.55 ms
25
NaradaBrokering-RTP
JMF-RTP
20
15
10
5
0
0
200 400 600 800 1000 1200 1400 1600 1800 2000
Packet Number
Global-MMCS 2.0 (2) Portlets


Collaboration clients will be built into portlets by
creating Java Applet or ActiveX controls for the nonHTML clients and adding them into HTML pages.
A collaboration portlet opens local services for XGSP
application session management and floor control.
• Node Manager portlet invoke the service to control local portlets


Apache Jetspeed seems good open source technology
supporting this model
Portlets such as Access Grid portlet (really a VIC portlet)
can be reused by Grid Portal Developers
Unicast AG Portlet
Multicast Multistream AG Portlet


Java applet supports
multicast AG with
multiple streams
In Jetspeed, easiest to
have fixed size but this
doesn’t fit well natural
range of 1-20 separate
streams
Workflow GCEs and Problem Solving
Environments (PSEs)
• There is some confusion between fields of workflow
(Grid Computing Environments GCE) and PSEs
• To extent PSEs “just” allow manipulation of “nuggets”,
they are indistinguishable from a domain specific GCE
• They are distinct if they support intra nugget operations
such as
– Integration of mesh and simulation
– Closely coupled code linkage
– Generation of code from high level interface like Mathematica
• Even in latter case, a new generation of PSEs should be
built with Grid architecture – e.g. message based – and
using Grid services like metadata and notification
Web Services as a Portlet
• Each Web Service naturally has a
user interface specified as “just
another port”
– Customizable for universal access
• This gives each Web Service a
Portlet view specified (in XML as
always) by WSRP (Web services
for Remote Portals)
• So component model for resources
“automatically” gives a component
model for user interfaces
– When you build your
application, you define portlet
at same time
Application as a WS
General Application Ports
Interface with other Web
Services
WSDL
W
Application or
Content source
Web Service
P
S
R
User Face of
Web Service
WSRP Ports define
WS as a Portlet
Web Services have other
ports (Grid Service) to be
OGSI compliant
Online Knowledge Center built from Portlets
A set of UI
Components
• Web Services provide a component model
for the middleware (see large “common
component architecture” effort in Dept. of
Energy)
• Should match each WSDL component with
a corresponding user interface component
• Thus one “must use” a component model
for the portal with again an XML
specification (portalML) of portal
HTML
Jetspeed
Architecture
Turbine Servlet
JSP template
ECS Root to HTML
Screen Manager
PSML
ECS
PortletController
PortletController
ECS
ECS
ECS
PortletControl
ECS
Portlets
Data
Portlet
XML
RSS, OCS, or other
Local or remote
ECS
Portlet
ECS
Portlet
ECS
Portlet
ECS
Portlet
HTML
JSP or VM
WebPage
Portlets
Local files
Local templates
Remote HTML
User implemented
using Portal API
Portlets and Portal Stacks
Aggregation Portals
(Jetspeed)
User facing Web
Service Ports
Application Grid Web
Services
Core Grid Services
Message Security, Information Services
• User interfaces to Portal
services (Code
Submission, Job
Monitoring, File
Management for Host X)
are all managed as
portlets.
• Users, administrators can
customize their portal
interfaces to just precisely
the services they want.
IU and OGCE Portal Architecture
Largely taken
from other projects
Clients
Portlet Class:
WebForm
Aggregation and Rendering
Clients (Pure HTML, Java Applet ..)
Emphasis
Portlet Class:
IFramePortlet
Portlet Class:
JspPortlet
Portlet Class:
VelocityPortlet
Jetspeed
Internal
Services
Portal
Portlets
(Jetspeed)
Gateway
(IU)
Remote
or Proxy
Portlets
Web/Grid
service
Computing
Web/Grid
service
Data Stores
Web/Grid
service
Instruments
GridPort
Texas
(Java)
COG Kit
Local
Portlets
Libraries
Hierarchical
arrangement
Services
Resources
Jetspeed Computing Portal: Choose Portlets
4 available portlets
linking to Web Services
I choose two
Choose Portlet Layout
Choose 1-column Layout
Original 2-column Layout
File management
Tabs indicate available
portlet interfaces.
Lists user files on
selected host, noahsark.
File operations include
Upload, download,
Copy, rename, crossload
Sample page with
several portlets:
proxy credential manager,
submission, monitoring
Administer Grid Portal
Provide information
about application
and
host parameters
Select application
to edit