Status of Grid Technology/Middleware

(e-Science, Cyberinfrastructure)

May 29 2003

Geoffrey Fox, Indiana University

Note the terms Grid, e-Science Technology/Middleware, and Cyberinfrastructure are NOT distinguished

What is a Grid I?

• Collaborative Environment (Ch2.2,18) • Combining powerful resources, federated computing and a security structure (Ch38.2) • Coordinated resource sharing and problem solving in dynamic multi institutional virtual organizations (Ch6) • Data Grids as Managed Distributed Systems for Global Virtual Organizations (Ch39) • Distributed Computing or distributed systems (Ch2.2,10) • Enabling Scalable Virtual Organizations (Ch6) • Enabling use of enterprise-wide systems, and someday nationwide systems, that consist of workstations, vector supercomputers, and parallel supercomputers connected by local and wide area networks. Users will be presented the illusion of a single, very powerful computer, rather than a collection of disparate machines. The system will schedule application components on processors, manage data transfer, and provide communication and synchronization in such a manner as to dramatically improve application performance. Further, boundaries between computers will be invisible, as will the location of data and the failure of processors. (Ch10)

What is a Grid II?

• Supporting e-Science representing increasing global collaborations of people and of shared resources that will be needed to solve the new problems of Science and Engineering (Ch36) • As infrastructure that will provide us with the ability to dynamically link together resources as an ensemble to support the execution of large-scale, resource-intensive, and distributed applications. (Ch1) • Makes high-performance computers superfluous (Ch6) • Metasystems or metacomputing systems (Ch10,37) • Middleware as the services needed to support a common set of applications in a distributed network environment (Ch6) • Next Generation Internet (Ch6) • Peer-to-peer Network (Ch10, 18) • Realizing thirty year dream of science fiction writers that have spun yarns featuring worldwide networks of interconnected computers that behave as a single entity. (Ch10)

What is Grid Technology?

• Grids support distributed collaboratories or virtual organizations integrating concepts from • The Web • Distributed Objects (CORBA Java/Jini COM) • Globus Legion Condor NetSolve Ninf and other High Performance Computing activities • Peer-to-peer Networks • With perhaps the Web being the most important for “Information Grids” and Globus for “Compute Grids” • Use Information Grids and not usual Data Grids as “distributed file systems” (holding lots of data!) are handled in Compute Grids

Taxonomy of Grid Functionalities

Name of Grid Type

Compute/File Grid

Description of Grid Functionality

Desktop Grid Information Grid Complexity or Hybrid Grid Campus Grid Run multiple jobs with distributed compute and data resources (Global “UNIX Shell”) “Internet Computing” and “Cycle Scavenging” with secure sandbox on large numbers of untrusted computers Grid service access to distributed information, data and knowledge repositories Hybrid combination of Information and Compute/File Grid emphasizing integration of experimental data, filters and simulations Grid supporting University community computing Enterprise Grid Grid supporting a company’s enterprise infrastructure Note: Term Data Grid not used consistently in community so avoided

OGSA-DAI Grid Services Grid HPC Simulation Analysis Control Visualize This Type of Grid integrates with Parallel computing as on TeraGrid Distributed Filters

massage data For simulation

Complexity Grid Computing Model

Taxonomy of Grid Operational Style

Name of Grid Style

Semantic Grid Peer-to-peer Grid

Description of Grid Operational or Architectural Style

Integration of Grid and Semantic Web meta data and ontology technologies Grid built with peer-to-peer mechanisms Lightweight Grid Collaboration Grid R3 or Autonomic Grid Grid designed for rapid deployment and minimum life-cycle support costs Grid supporting collaborative tools like the Access Grid, whiteboard and shared applications.

Fault tolerant and self-healing Grid Robust Reliable Resilient R3

“Central” Architecture/Functionality/Style Gaps

• Substantial comments on “hosting environments” OGSI and “permeating principles” – Agreement on Web service model 6: Domain-Specific (Application) Grid Services 5: OGSA-compliant System Grid Services

“Modular” Services natural for distributed teams Specific Gaps

4: Key OGSA Services 3: Permeating Principles and Policies WS WS WS WS

“Central Services And Architecture” Central Gaps

1: Hosting Environment 2: OGSI Web service Enhancements

An OGSA Grid Architecture in detail (from GGF GPA)

Permeating Principles and Policies

• Meta-data rich Message-linked Web Services as the permeating paradigm • “User” Component Model such as “Enterprise JavaBean (EJB)” or .NET. • Service Management framework including a possible Factory mechanism • High level Invocation Framework describing how you interact with system components.

– This could for example be used to allow the system to built from either W3C or GGF style (OGSI) Web Services and to protect the user from changes in their specifications.

• Security is a service but the need for fine grain selective authorization encourages • Policy context that sets the rules for each particular Grid. – Currently OGSA supports policies for routing, security and resource use.

• The Grid Fabric software.

or set of resources needs mechanisms to manage them. This includes automatic recording of meta-data and configuration of • Quality of service (QoS) for the Network and this implies performance monitoring and bandwidth reservation services. – Challenging as end-to-end and not just backbone QoS is needed.

• Messaging systems like MQSeries from IBM provide robustness from asynchronous delivery and can abstract destination and allow customization of content such as converting between different interface specifications.

• Messaging is built on transport mechanisms which can be used to support mechanisms to implement QoS and to virtualize ports

World Wide Grid Service Activities I

• Commercial activities especially those of IBM, Avaki, Platform, Sun, Entropia and United Devices • The GT2 and GT3 Globus Toolkits Grids. classified as Services • Trillium ( GriPhyn , iVDGL and . Here we effectively covering not just the Globus team but the major projects such the NASA Information Power Grid that have blazed the trail of “productizing” – Note that we can “already” see GT3 (Grid Service) like functionality from GT2 wrapped with the various (Java, Perl, Python, CORBA) CoG kits. So GT2 capabilities can be PPDG ) and NeesGrid ; the major NSF (DoE for PPDG) projects in the USA. – Condor from the University of Wisconsin which is being integrated into Grid services through the Trillium and NMI activities.

• The NSF Middleware Initiative (NMI) packaging a suite of Globus, Condor and Internet2 software. – This has overlaps with the VDT ( Virtual Data Toolkit from GriPhyn)

World Wide Grid Service Activities II

• Unicore (GRIP) , GridLab , the European Data Grid (EDG) and LCG (LHC Computing Grid) – Many other (20) EU Projects but these have most of technology development • Storage Resource Broker SRB-MCAT from SDSC • The DoE Science Grid and related activities such as the Common Component Architecture in portals.

(CCA) project • Examination of services from a collection of US from Argonne, Indiana, Michigan, NCSA and Texas. – This includes best practice discussion from Global Grid Forum • Review of contributions to the recent book portal projects in the

Grid Computing: Making the Global Infrastructure a Reality

Geoffrey Fox and Tony Hey, John Wiley & Sons, Chichester, England, ISBN 0-470-85319-0, March 2003 – This includes other major projects like edited by Fran Berman, Cactus, NetSolve, Ninf • Some 6 Core and other application specific UK e-Science Projects

Categorization of Technical Gaps and Grid Services Section Numbers in Report available Mid June

Architecture and Style 8.1

Portals PSE’s 8.10

Basic Technology Runtime and Hosting Environment 8.2

Grid Services: Application Specific Resource Specific Generic Compute Resources Network 8.11

Information 8.7

Compute/File Information Security 8.3

Workflow 8.4

Notification 8.5

Meta-data 8.6

Other 8.9

8.8

• • • • • •

Categories of Worldwide Grid Services

1) Types of Grid • – – – R3 Lightweight P2P 7) Information Grid Services – OGSA-DAI/DAIT – Integration with compute resources – P2P and database models – Federation and Interoperability – – – – – – 2) Core Infrastructure and Hosting Environment Service Management Component Model Service wrapper/Invocation Messaging 3) Security Services Certificate Authority Authentication • 8) Compute/File Grid Services – Job Submission – Job Planning Scheduling Management – Access to Remote Files, Storage and Computers – Replica (cache) Management – Virtual Data – Parallel Computing – Authorization – – – – – Policy 4) Workflow Services and Programming Model Enactment Engines (Runtime) Languages and Programming Compiler Composition/Development 5) Notification Services • 9) Other services including – Grid Shell – Accounting – Fabric Management – Visualization Data-mining and Computational Steering – Collaboration 6) Metadata and Information Services • 10) Portals and Problem Solving Environments – Basic including Registry – – – Semantically rich Services and meta-data Information Aggregation (events) Provenance • 11) Network Services – Performance – Reservation – Operations

Features of Worldwide Grid Services

• UK activities have a strong web service and Information Grid emphasis – Important compute/file activities as well (White Rose, RealityGrid, UK part of EDG etc.) • Non UK activities are dominantly focused on compute/file Grids – Submit jobs in distributed UNIX shell (Gridshell) fashion – Gather data from instruments (accelerator, satellite, medical device); process in batch mode mapping between filesets • Little emphasis on lightweight or R3 Grids but NSF in USA and EDG have aimed at better support and software quality – EDG has useful “tension” between technology and application focus working groups – NMI and even GT3 have changed packaging and added service view – have not changed “underlying” architecture for robustness • Coordinated set of Portal activities in USA • Little work on integrating parallel computing and Grid although TeraGrid in USA could change this

Central Gaps: Gaps in Grid Styles and Execution Environment

• Need for both robust (fault tolerant) and lightweight (suitable for small groups) Grid styles identified – Peer-to-peer style supports smaller decentralized virtual organizations • Note opportunities for modern middleware used – lightweight, message-based ideas to be • Note that Enterprise JavaBeans not optimized for Science which has high volume dataflow • Federated Grid Architecture natural for integration of heterogeneous functionality, style and security • Bioinformatics and other fields require integration of Information and Compute/File Grids

Dynamic light-weight Peer-to-peer Collaboration Training Grid

Enterprise Grid Students Information Grid R1 R2 Compute Grid Teacher Campus Grid

Overlapping Heterogeneous Dynamic Grid Islands

Core Service Application Service (a) Layered OGSA Grid Application Service Application Service Core Service Core Service Core Service OGSA Interface (b) Federated OGSA Grid Appl.

Service Appl.

Service Appl.

Service Appl.

Service Core Service Core Service Core Service Core Service Core Service Core Service Grid-1 OGSA Mediation OGSA or non OGSA Interface-1 Grid-2 OGSA or non OGSA Interface-2

Many Gaps in Generic Services

• Some gaps like Workflow and Notification are to make production versions of current projects – Just in UK workflow from DAME, DiscoveryNet, EDG, Geodise, ICENI, myGrid, Unicore plus Cardiff, NEReSC ….

• RGMA and (Globus) • Security Semantic Grid offer improved meta-data and Information services compared to UDDI and MDS – Need comprehensive federated Information service requires architecture supporting dynamic fine grain authorization • UK e-Science has pioneered Information Grids but gap is continuation of • Functionality of Campus Grids OGSA-DAI, integration with other services and P2P decentralized models Compute/File Grids quite advanced but services probably not robust enough for LCG or

Gaps in Other Grid services

• Portals and User Interfaces – Noted gap that many not using Grid Computing Environment “best practice” with component based user-interfaces matching component-based middleware • Programming Models (using workflow runtime) • Fabric Management central service management and Information system), (should be integrated with Computational Steering , Visualization , Datamining , Accounting , Gridmake , Debugging , Semantic Grid tools (consistent with Information system), this type Collaboration , provenance • Application-specific services • Note new production central Infrastructure can support both research and production services of

PPPH: Paradigms Protocols Platforms and Hosting I

• We will start from the Web view and assert that basic paradigm is • Meta-data rich Web Services communicating via messages • These have some basic support from some runtime such as .NET, Jini (pure Java), Apache Tomcat+Axis (Web Service toolkit), Enterprise JavaBeans, WebSphere (IBM) or GT3 (Globus Toolkit 3)

– These are the distributed equivalent of operating system functions as in UNIX Shell

• Called Hosting Environment or platform

OGSA OGSI & Hosting Environments

• Start with Web Services in a hosting environment • Add OGSI to get a Grid service and a component model • Add OGSA to get Interoperable Grid “correcting” differences in base platform and adding key functionalities

Functional Level above OGSA

• • • • • • • • •

Systems Management and Automation Workload / Performance Management Security Availability / Service Management Logical Resource Management Clustering Services Connectivity Management Physical Resource Management Perhaps Data Access belongs here

OGSI Open Grid Service Interface

• http://www.gridforum.org/ogsi-wg • It is a “ component model – Defines an – Service collections ” for web services.

• It defines a set of behavior patterns that each OGSI service must exhibit.

• Every “Grid Service” portType extends a common base type.

introspection model • A set of standard portTypes for for the service – You can query it (in a standard way) to discover • What methods/messages a port understands • What other port types does the service provide?

• If the service is “stateful” what is the current state?

– Message subscription and notification • Each service is identified by a URI called the “ Grid Service Handle ” • GSHs are bound dynamically to Grid Services References docs) – A GSR may be transient. GSHs are fixed.

– Handle map services translate GSHs into GSRs.

(typically wsdl

OGSI and Stateful Services

• Sometimes you can send a message to a service, get a result and that’s the end – This is a statefree service • However most non-trivial services need state to allow persistent asynchronous interactions • OGSI is designed to support Stateful services through two mechanisms – Information Port: where you can query for SDE (Service Definition Elements) – “Factories” that allow one to view a Service as a “class” (in an object-oriented language sense) and create separate instances for each Service invocation • There are several interesting issues here – Difference between Stateful interactions and Stateful services – System or Service managed instances

Factories and OGSI

• Stateful interactions are typified by amazon.com where messages carry correlation information allowing multiple messages to be linked together – Amazon preserves state in this fashion which is in fact preserved in its database permanently • Stateful services have state that can be queried outside a particular interaction • Also note difference between implicit and explicit factories – Some claim that implicit factories scale as each service manages its own instances and so do not need to worry about registering instances and lifetime management • See WS-Addressing from largely IBM and Microsoft http://msdn.microsoft.com/webservices/default.aspx?pull=/library/en-us/dnglobspec/html/ws-addressing.asp

Implicit Factory

F A C T O R Y

1 2 3 4

Explicit Factory

1 2 3

F A C T O R Y

4

Two-level Programming I

• The paradigm implicitly assumes a two-level Programming Model • We make a Service (same as a “distributed object” or “computer program” running on a remote computer) using conventional technologies – C++ Java or Fortran Monte Carlo module – Data streaming from a sensor or Satellite – Specialized (JDBC) database access • Such nuggets accept and produce data from users files and databases

Nugget

Data

• The Grid is built by coordinating such nuggets assuming we have solved problem of programming the nugget

Two-level Programming II

• The Grid is discussing the linkage and distribution of the nuggets with the only addition runtime interfaces

Nugget1 Nugget2

to Grid as opposed to UNIX data streams

Nugget3 Nugget4

• Familiar from use of UNIX Shell, PERL or Python scripts to produce real applications from core programs • Such interpretative environments are the single processor analog of Grid Programming or Workflow • Some projects like GrADS from Rice University are looking at integration between nugget levels but dominant effort looks at each level separately

User Services System Services System Services Portal Services Application Service Middleware System Services Actual Application Grid Computing Environments System Services System Services “Core” Grid Raw (HPC) Resources Database

PPPH: Paradigms Protocols Platforms and Hosting II

• Self-describing programs/interfaces are key to scaling – Minimize amount of work system has to do – Hide as much as possible in services and applications • Protocols describe (in “principle” at least) those rules that system obeys and uses to deliver information between services (processes) • Interfaces • HTTP tell the service what to do to interpret the results of communication is the dominant transport protocol of the Web • HTML is the “interface” telling browser how to render • But you can extend interface PowerPoint using “helper applications” which are (with more or less convenience) “automatically” downloaded if not already available to allow PDF, multimedia, – “Mime types” essentially self-describe” each interface

Protocol/Interface Analogy with Web II

• HTTP and HTML are the analogies on the client side • A “Web Service” generalizes a parameters of program to run • Roughly • Web uses WSDL other protocols CGI Script on server side – CGI is essentially a Distributed Object technology allowing server to access an arbitrary program labeled by a URL plus an ugly syntax to specify name and (Web Service Description Language) is a better way to specify program name and its parameters – HTTPS for secure links and RTP etc. for multimedia (UDP) streams – These again are required to integrate system – codecs like MPEG are interfaces interpreted by client – There are further protocols like H323 and SIP which will be replaced (IMHO) by HTTP plus RTP etc. We should minimize number of protocols to get maintainable systems

PPPH: Paradigms Protocols Platforms and Hosting III

• There are set of system capabilities which cannot be captured as standalone services and permeate Grid • Meta-data rich Message-linked Web Services is permeating paradigm • Component Model such as “Enterprise JavaBean (EJB)” or OGSI describes the formal structure of services – EJB if used lives inside OGSI in our Grids • Invocation Framework • Security in fine grain fashion to provide selective authorization (Globus and EDG WP6) • Policy context describes how you interact with system describes rules for this particular Grid • Transport mechanisms Service abstract concepts like ports and Quality of • Messaging abstracts destination and customization of content • Network (monitoring, performance) EDG WP7 • Fabric (resources) EDG WP4

A C T U A L A B S T R A C T

Architecture in Pictures I

Services Abstract Model OGSI Messaging Services Hosting Environment determines physical model Invocation Framework Messaging Network Resources

Architecture in Pictures II OGSA Interoperable Grid

OGSA Interfaces Exposed by every OGSI Grid Services Messaging Network Monitoring and Scheduling Network Resources

Architecture in Pictures III

Mediation Service

OGSA Federated Grid

Mediation Service converting between OGSA and “native” services Native Services not necessarily OGSI Messaging Network Monitoring and Scheduling Network Resources

  

Federation/Interoperability Problem?

Have a collection of Web Services running in Grids defined by different suppliers?

Interoperability – “particular application Web Service of supplier X” can utilize “core service of supplier Y” Federation – “core service of supplier X” can be integrated with “core service of supplier Y” to provide a integration/amalgam that is also a realization of core service. Need mediation to link different Grid Islands AWS1 JXTA Avaki Grid Service2 IBM Grid Resource2 AWS2 GT3 Grid AWS3 Platform Service 1 Jini Grid Resource1

Standards Compliant InterGrids Federation Environment

Resource

Federation Architecture

Routing Node Mediation Node

Resource Resource Resource Resource

R

Grid Instance Service Service

M

Grid Instance Service Service Resource Service Service Resource

M R M

Resource Resource

R M R M M R M R

Resource Resource Grid Instance Service Service Service

M R

Grid Instance Service Service Service Resource Resource Resource Resource Resource

Virtualization

• The Grid could and sometimes does virtualize various concepts • Location: URI (Universal Resource Identifier) virtualizes URL • Replica management (caching) virtualizes file location generalized by GriPhyn virtual data concept • Protocol: message transport and WSDL bindings virtualize transport protocol as a QoS request • P2P or Publish-subscribe messaging virtualizes matching of source and destination services • Semantic Grid virtualizes Knowledge as a meta-data query • Brokering virtualizes resource allocation • Virtualization implies references can be indirect

IFS: Interfaces and Functionality and Semantics I

• The Grid platform tries to minimize detail in protocols and maximize detail in interfaces to enhance scaling • However rich meta-data and semantics are critical for correct and interesting operation – Put as much semantic interpretation as you can into specific services – Lack of Semantic interoperation today’s Grids and Web services is in fact main weakness of • Everything becomes a service (See example of education) whether system or application level • There are some very important “Global Services” – Discovery (look up) and Registration of service metadata – Workflow – MetaSchedulers

IFS: Interfaces and Functionality and Semantics II

• There are many other generally important services • OGSA-DAI The Database Service • Portal Service linked to by WSRP (Web services for Remote Portals) • Notification of events • Job submission • Provenance – interpret meta-data about history of data • File Interfaces • Sensor service – satellites … • Visualization • Basic brokering/scheduling

OGSA/OGSI Top Level View

Chapters 7 to 9 of Book http://www.gridforum.org/Meetings/ggf7/docs/default.htm

http://www.globusworld.org/globusworld_web/jw2_program_tut.htm

• OGSA is the set of “core” Grid services

Domain-specific services – Stuff you can’t live without – If you built a Grid you would need to invent these things More specialized services: data replication, workflow, etc., etc. Broadly applicable services: registry, authorization, monitoring, data access, etc., etc.

OGSI

Host. Env. & Protocol Bindings

Open Grid Service Architecture

• OGSA-WG chaired by

– Ian Foster, ANL and Univ. of Chicago – Jeff Nick, IBM – Dennis Gannon, IU

• Active Members from

– IBM, Fujitsu, NEC, SUN, Hitachi, Avaki – Univ. of Mich, Chicago, Indiana (not much academic involvement)

OGSA Core Services I

• Registries, and namespace bindings – Registry is a collection of services indexed by service metadata.

• “find me a service with property X.” – Directory is a map from a namespace to GSHs.

– A namespace is a human understandable version of a Grid Handle • Queues – For building schedulers and resource brokers – Jobs and other requests are in queues – This is high-level messaging

Security

• Base this on Web Services Security • Authentication – 2-way. Who are you and who am I?

• Authorization – What am I authorized to use/see/modify • Accounting/Billing – (not really security – see monitoring) • Privacy • Group Access – Easily create a group to share access to a virtual Grid.

• Very complex issues related to services and message delivery.

Common Resource Model

• Every resource on the grid that is manageable is represented by a service instance

– CRM is the Schema hierarchy that defines each resource (with its meta-data) – Service for a resource presents its management interface to authorized parties.

Policy Management

• Policy management services – Mechanism to publish policy and the services it applies to. – Policy life-cycle mgmt.

• Policy languages exist for routing, security, resource use Producer of Policies * Canonical Policies 1 * Consumer of Policies

Policy Enforcement Point

Canonical Policies * 1 *

Policy Service Manager

1..n

* *

Policy Service Agent

* 1

XML Repository

1 Non-Canonical Policy Component Requirements:  A management control point for policy lifecycle (PSM)  A canonical way to express policies (AC 4-tuple)    A distribution point for policy dissemination (PSA) A way to express that a service is “policy aware” (PEP) A way to effect change on a resource (CRM)

Grid Service Orchestration

• Creating new services by composing other services • Two types of Orchestration

– Composition in space • One services is directly invoking another – Composition in time • Managing the workflow – First do this.

– Then do this and that – When that is done do this » If something goes wrong do this – And so on…

Data Services

• Distributed Data Access • Data Caching • Data Replication Services • Metadata Catalog Services • Storage Services

Metering Resource Consumption

• At what granularity do services report resource consumption? • How do they report it?

• How are services metered?

Transactions

• Two threads/workflows must synchronize and agree they have done so before moving on.

– Usually involves modification to two or more persistent states – WS-transactions has been “proposed”.

Messaging, Events, Logging

• Messaging – Delivery Model – Queuing and Pub/Sub message delivery (not clear to me why these are different as publish/subscribe implemented as topic labeled queues) • Events – Time stamped messages – Standard XML schemas • Standard Logging • MQSeries (IBM), JMS (Java Message Service) and NaradaBrokering (Indiana) provide this but most naturally at level of “platform/hosting environment”

Where should Messaging be?

• One can define messaging at the OGSA level “above the hosting environment” but that makes it difficult to virtualize messaging and support network performance – Publish-subscribe or better queued messaging naturally supports optimized routing based on network performance • One can naturally support collaborative Web services in same fashion in a way that it MUCH easier that GrooveNetworks and other collaborative environments (WebeX, Placeware(Microsoft)) do as long as every application is a Web service • OGSA location of messages is fine for low volume logging or notification events – Not good for events on “video” application where each frame is an update event

The Overall Architecture

• The Grid is defined by a collection of distributed Services – For many users the primary interaction with the Grid will be through a portal Portal Server MyProxy Server Event and logging Services Metadata Directory Service(s) Application Factory Services Messaging and group collaboration Directory & index Services

Web Services as a Portlet

• Each Web Service user interface another port” naturally has a specified as “just

Application as a WS

General Application Ports Interface with other Web Services – Customizable for universal access WSDL • This gives each Web Service a W S Portlet view specified (in XML as Application or Content source always) by WSRP (Web services P R Web Service for Remote Portlets) • So component model for resources “automatically” gives a component model for user interfaces – When you build your application, you define portlet at same time User Face of Web Service WSRP Ports define

WS as a Portlet

Web Services have other ports ( Grid Service ) to be OGSI compliant

Online Knowledge Center built from Portlets

A set of UI Components

• Web Services

provide a

component model

for the middleware (see large “

common component architecture

” effort in Dept. of Energy) • Should match each WSDL component with a corresponding user interface component • Thus one “must use” a

component model for the portal

with again an XML specification (

portalML

) of portal component

InterGrids



Develop Grid software including both the packaging of Grid Islands and federation between Grids



Key underlying principles:

•

Support small Grids ( university departments, Private Grids) with easy deployment (e.g. JXTA Jini …)

•

Support composability (P2P) and federation with itself and other Grids

•

Use existing ( Avaki, Globus JXTA ICENI ..) bits and pieces if possible – encourage such projects to produce modules useable in other Grid systems like InterGrids

•

Good test/benchmark/tutorial material and good support

    

Features of InterGrids I

At outer levels of Grid, just need interoperable services but these services would need mediation as they access resources in a different grid

•

Interoperable Services have interfaces allowing them to access appropriate other services in their own grid and through mediation the analogous services in other grids At inner levels of Grid, need to federate services

•

Federated services produce results which need to be merged with other services of the same type when Grids are joined together In case of Jini based Grid, we keep Jini services as Jini and don’t wrap them Rather the OGSA wrapping of Jini occurs in the mediator Issues if multiple Grids overlap

•

Can multiple brokers manage same resources (schedulers on computers)

•

What happens if one shares resources between virtual organizations (Grids)

  

Features of InterGrids II

Mediation service integrates

• • • •

VPN with adaptable port Firewalls and security negotiation between domains Mediation Difference between PURL/URI and URL Mediation Service is distributed i.e. is not a single proxy server Each logical message (this could be several physical messages) in system is examined – destination URI is “just” mapped to URL if internal

•

If external routed to “best mediator” in same way NaradaBrokering optimally routes messages

•

Logical messages are self contained i.e. have enough information to fully specify any transformations needed in mediator i.e. they are “all” the Schema instance



Features of InterGrids III

When message arrives at mediator service, its source and destination Grids are examined

• Note Grids must register themselves in mediation service and in that registration (or a later update) give specification for service and any needed transformations) • This registration involves introduction of concept of Grid Gridtype with specification attached to a particular Gridtype • If they have same specification for this service, then message is routed on unchanged • If specification different, the mediator looks to see if any special translations available for given source or destination • If no special actions, the source mapper is used to convert message to FGSA (Federated Grid Service Architecture -- hopefully same as OGSA) and the destination inverse mapper is used to map FGSA to destination format  This can generate an error returned to calling service and logged for both Grids

  

Features of InterGrids IV

This mediation is sufficient for an “interoperable” service Some services such as “search” or “look-up” are registered as federated

•

i.e. requests to them are multi-cast to multiple grids and results merged

•

Rules must be defined for federated services defining nature of result



Is more than one answer allowed



If >1 answer possible, then rules for merging results of services in each Grid must be specified Another complication is if a single service in one Grid corresponds to composition of multiple services in another Grid

•

One sees this for look-up which could involve different levels of meta-data in different Grids

•

This seems complicated but relatively clear what one has to do in composing dynamically services

Features of InterGrids V

  

There are two problems mentioned earlier Shared Resource Reference Services – this must be a standard issue in federation. This occurs when federation of say look up services leads to duplicate results. The look-up may not be quite the same and one would want to remove or combine duplicate responses Shared Resource Access Services – these are services in different Grids that access and affect the same backend resource. This issue comes up even inside a single Grid

•

It is possible that mediator could be used to resolve this problem but an heuristic needs to be developed for this