Transcript Grid Message Infrastructure Edinburgh e-Science Centre December 11 2002 PTLIU Laboratory for Community Grids Geoffrey Fox, Shrideep Pallickara Computer Science, Informatics, Physics Indiana University, Bloomington.

Grid Message Infrastructure
Edinburgh e-Science Centre December 11 2002
PTLIU Laboratory for Community Grids
Geoffrey Fox, Shrideep Pallickara
Computer Science, Informatics, Physics
Indiana University, Bloomington IN 47404
http://grids.ucs.indiana.edu/ptliupages
[email protected]
11/6/2015
uri="http://www.naradabrokering.org"
email="[email protected]"
1
Some Basic Observations/Goals



Grids manage and share asynchronous resources in a
rather centralized fashion
Peer-to-peer networks are “just like” Grids with
different implementations of services like registration
and look-up
Web Services interact with messages and Everything
will be a Web Service
• Microsoft Office will be a WS?

Computers are fast and getting faster. One can afford
many strategies that used to be unrealistic
• Software/Application-level message processing/routing
sensitive to performance information
• Uniform messaging framework for Middle-tier, “High
performance layer”, Multimedia (audio-video) etc.
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
2



Deduction?
Grids or P2P Networks consist of a sea of message-based Services
Services inject and extract messages whose transport and
manipulation is support by a logically distinct “MessageGrid” of
brokers/routers supplying “Grid Message Layer”
The message brokers support adaptive routing, filtering,
workflow
• They have publish/subscribe queued connection semantics
• They separate logical and actual transport
• They form a federated XML database and support
asynchronous collaboration
• They process real-time messages in about a millisecond and
support synchronous collaboration

NaradaBrokering is a prototype of this idea
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
3
Peers
Database
Database
Service Facing
Web Service Interfaces
Event/
Message
Brokers
Event/
Message
Brokers
Event/
Message
Brokers
Peer to Peer Grid
Peers
User Facing
Peer to Peer Grid
of
Resources supported by
Web Service Interfaces
internal “MessageGrid” of message or event brokers
Events are “just” Time-stamped messages
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
4
Shared Input Port (Replicated WS) Collaboration
Collaboration as a WS
Set up Session with XGSP
R
U
Web
F
Servic
I
I
e
O
O
F
WS
Viewer
WS
Display
Master
U
Web
F
Servic
I
I
e
O
O
F
Event
(Message)
Service
R
R
U
Web
F
Servic
I
I
e
O
O
F
11/6/2015
WS
Viewer
WS
Display
Other
Participants
WS
Viewer
uri="http://www.naradabrokering.org" email="[email protected]"
WS
Display
5
NaradaBrokering

Based on a network of cooperating broker nodes
• Cluster based architecture allows system to scale to arbitrary
size


Originally designed to provide uniform software
multicast to support real-time collaboration linked to
publish-subscribe for asynchronous systems.
Now has four major core functions
• Message transport (based on performance measurement) in
heterogeneous multi-link fashion
• General publish-subscribe including JMS & JXTA and
support for RTP-based audio/video conferencing
• Filtering for heterogeneous clients
• Federation of multiple instances of Grid services
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
6
Role of Event/Message Brokers



We will use events and messages interchangeably
• An event is a time stamped message
Our systems are built from clients, servers and “event brokers”
• These are logical functions – a given computer can have one
or more of these functions
• In P2P networks, computers typically multifunction; in Grids
one tends to have separate function computers
• Event Brokers “just” provide message/event services; servers
provide traditional distributed object services as Web services
There are functionalities that only depend on event itself and
perhaps the data format; they do not depend on details of
application and can be shared among several applications
• NaradaBrokering is designed to provide these functionalities
• MPI provided such functionalities for all parallel computing
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
7
Note on Optimization

Note in parallel computing, couldn’t do much dynamic
optimization as aiming (in MPI) at microsecond latency
• Natural to use hardware routing

In Grid, time scales are different
• 100 millisecond quite normal network latency
• 30 millisecond typical packet time sensitivity (this is one audio
or video frame) but even here can buffer 10-100 frames on
client (conferencing to streaming)
• 1 millisecond is time for a Java server to “think”


Jitter in latency (transit time) due to routing, processing
(in NB) or packet loss recovery is important property
Grid needs and can tolerate significant dynamic
optimization
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
8
Architecture of Message Layer


Need to optimize not only routing of particular messages but
classic publish/subscribe problem of integrating different
requests with related topics (subscribe to sports/basketball/lakers
and sports)
Related to Akamai, AOL … caching and Server optimization
problem
Hypercube of
NB Brokers (logical
not physical)
N≈100 for
Distance
Education
Scale to
billions of
grid nodes?
1-> N Grid Nodes
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
9
Performance in Message-based Architecture I
A
Satellite
UDP
Firewall
HTTP
Fast
Link
Hand-Held
Protocol B2
Software Multicast



B1
Dial-up
Filter
Useful analogies with transportation grids
B3
and parallel computing
In traveling from cities A to B (say 3 separate passengers), one
chooses between and changes transport mechanism at
waystations to optimize cost, time, comfort, scenic beauty …
Waystations are now NB brokers where one chooses transport
protocol
• Able to choose between car, type of car, plane, train etc
• Able to dynamically create waystations to cope with problems
and acts as hubs for multicast messages
• Knows about traffic jams and can assign the “HOV lane”
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
10
Performance in Message-based Architecture II


Application level QoS – can optimize among managed
streams (audio versus video) using performance
subsystem
• This is just a variant of the NP complete load
balancing problem of parallel computing where all
reasonable heuristics worked
• Load-balance in Space-time (strings) not just Space
(particles)
“Performance” needs to measured carefully as
includes QoS
• I delayed shared application update to ensure audio
quality and filtered image to lower resolution
• So “application” has changed to satisfy performance
constraints
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
11
NaradaBrokering Communication




Applications interface to NaradaBrokering through UserChannels
which NB constructs as a set of links between NB Broker waystations
which may need to be dynamically instantiated
UserChannels have publish/subscribe semantics with XML topics
Links implement a single conventional “data” protocol.
• Interface to add new transport protocols within the Framework
• Administrative channel negotiates the best available communication
protocol for each link
Different links can have different underlying transport implementations
• Implementations in the current release include support for
TCP,UDP, Multicast, SSL and RTP.
HTTP, HTTPS support will be available in Feb 2003 release.
• Supports communication through proxies such as iPlanet, Netscape
and Apache.
• Supports communication through firewalls such as Microsoft ISA,
Checkpoint.
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
12
Transit delay for message samples in NaradaBrokering
Different communication hops - Internal Machines
5.5
hop-2
hop-3
hop-5
hop-7
5
4.5
4
3.5
3
Sender/receiver/broker - (Pentium-3, 1
GHz, 256 MB RAM). 100 Mbps LAN.
JDK-1.3, Red Hat Linux 7.3
2.5
2
1.5
1
0.5
50
11/6/2015
100
150
200
250
300
350
Message Payload Size
(Bytes)
uri="http://www.naradabrokering.org" email="[email protected]"
400
450
500
13
80
JMF-RTP
NaradaBrokering-RTP
H.263 video file
70
Average bit-rate of 600Kbps.
Frame-rate of 30 frames/sec
60
50
40
Jitter J = J + (|D(i-1, i)| - J)/16
30
20
10
0
0
200
400
600
800
1000
1200
1400
1600
1800
Packet Number
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
14
JMS Performance: Transit Delay and Std
Deviation (NaradaBrokering & SonicMQ)
Transit Delays for Message Samples in Narada and SonicMQ
Mean
Transit Delay
(MilliSeconds)
30
25
20
15
10
5
0
0
50
100
150
Transit Delay
200
250
Publish Rate 300
350
(Messages/sec)
400
Narada
SonicMQ
550
500
450
400
350
300
250
Payload Size
200
150
(Bytes)
100
Standard Deviation in the Message Samples - Narada and SonicMQ
Standard
Deviation
(MilliSeconds)
14
12
10
8
6
4
2
0
0
50
100
Standard Deviation
150
200
250
Publish Rate 300
350
(Messages/sec)
400
550
500
450
400
350
300
250
Payload Size
200
150
(Bytes)
100
• NaradaBrokering is Java Message Service (JMS) compliant
• Applications ported include
•Anabas – JMS based distance education system.
•OKC – Online Knowledge Center project at IU
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
Narada
SonicMQ
15
Futures










Redo performance with insight of this workshop – we
desperately need a distributed testbed to support scaling tests
Test Scaling with construction of dynamic NB broker network
(what is best topology? Hypercube like?)
Understand clients per broker for various applications
including large multimedia sessions
Complete transport/performance infrastructure
Develop nifty “load balancing heuristics”
Implement Web Service message-based Security
Understand better how to implement distributed XML
database
Develop NB to support federation of Grid Services
Generalize Filtering mechanism
Download from http://www.naradabrokering.org
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
16
NaradaBrokering as an XML database

NB is inevitably a distributed XML database
supporting real-time update and access (JMS uses SQL
like syntax)
• Performance data
• Event Logging
• Published “properties” of publish/subscribe messages
 Publish as XML topic; subscribe using XQuery?

We use NB as a simple XML database for News groups
and other “XML nuggets” see
http://www.xmlnuggets.org
• XML Instance==Message; what is difference between a
message and a database record except performance but
Moore’s law is rewriting rules here

Subscribe a real database (Oracle) to all topics for
reliable storage
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
17
Federation of Services




If you have a service – Notification (as with Grid or JXTA
advertisements), Search, Scheduling, Audio-Video conferencing
….
With a standard which client and server components
Then can build a “server” that services all clients
Alternatively can hierarchically consider collection of existing
Server-client domains
•
•
•
•

IBM or Globus OGSA islands
Sun Grid Engine Schedulers
Polycom/Access Grid A/V sessions
Enterprise Search Engines
NB
Hub
Federation links islands together
• JXTA Search has XML specified federation approach – will try to include
and generalize as a NaradaBrokering federation framework
• DoD High Level Architecture HLA does this for simulation
11/6/2015
uri="http://www.naradabrokering.org" email="[email protected]"
18