Grid of Grids Information Management Kick off Meeting

Anabas June 30 2006

ACKNOWLEDGMENTS We are grateful to: • Bill McQuay, • Raymund Garcia •The Air Force Research Laboratory - AFRL IFSD • Indiana University CGL (Subcontractor Partner) • Ball Aerospace (Subcontractor Partner)

General Goal

• Build Net Centric Core Enterprise Services in fashion compatible with GGF/OGF and Industry • Add key additional services including those for sensors and GIS • Support Systems of Systems by federating Grids of Grids supporting a heterogeneous software production model allowing DoD greater sustainability and choice of vendors • Build tool to allow easy construction of Grids of Grids

• • • •

Gerstner and Commercialization of GoG Architecture

Reference [1]: Who Says Elephants Can't Dance, Louis Gerstner, Jr., 2002 is practically a primer on IBM's vision and strategy to re-gain its influence to setting IT industry agenda. "The industry's disaggregation into thousands of niche players would make IT integration services a huge growth segment of the industry." [1] –

Grid of Grids builds the information management infrastructure for efficient integration of components

.

"Over the next decade, customers would increasingly value companies that could provide solutions - solutions that integrated technology from various suppliers and more importantly, integrated technology into the processes of an enterprise." and "The two companies that have enjoyed the highest market valuation in the IT industry in over much of the last decade have been component manufacturers -- Intel and Microsoft. There is no doubt that a strategy built around providing fundamental building blocks of the computing infrastructure has proven to be extremely successful in this industry." [1] –

Grid of Grids has huge commercial potential by offering both the integration infrastructure for component manufacturers and providing Network-Centric Enterprise Services as essential building blocks .

"The dazzlingly wired-world would also change the course of computing in profound ways. For one thing, it was virtually certain that world would be built on open industry standards. There would be no other way to fulfill the promise of ubiquitous connections among all the businesses users, devices, and systems that would participate in a truly networked world. If that standards-based world came to pass, it would represent a major shift in the prevailing competitive landscape." [1] –

Grid of Grids information management with its foundation on GGF/OGF open standards is a key to integration.

Technical Advisory Board

• Build a group of some 6-10 knowledgeable people to guide project • Mainly from Industry including Boeing, Lockheed Martin, Ball Aerospace, Aerospace Corporation, Mitre, DISA, and the core technology companies (IBM, Microsoft) • Use to guide both technology and commercialization strategy • Meet F2F, email and telecons.

• Brings significant external expertise to Grid of Grids team, and visibility of Grid of Grids to other world-renowned institutions.

Raw Data Another Grid Another Service Another Grid SS SS SS SS SS SS SS



S S FS FS FS FS FS O S Data S S



FS S S Information



FS Another Grid S S Knowledge MD MD O S O S MD FS MD O S O S FS FS FS FS MD O S FS FS O S FS MD FS Filter Service FS F S MD O S FS O S FS F S FS MD O S O S



Wisdom Decisions O S M O ther S ervice MD eta D ata S S S S S S S S S S S S S S S S S S Sensor Service Database Another Service

The Grid and Web Service Institutional Hierarchy

4: Application or Community of Interest (CoI) Specific Services

such as “Map Services”, “Run BLAST” or “Simulate a Missile”

XBML XTCE VOTABLE CML CellML 3: Generally Useful Services and Features (OGSA and other GGF, W3C)

Such as “Collaborate”, “Access a Database” or “Submit a Job”

OGSA GS-* and some WS-* GGF/W3C/….

XGSP (Collab) 2: System Services and Features (WS-* from OASIS/W3C/Industry)

Handlers like WS-RM, Security, UDDI Registry

1: Container and Run Time (Hosting) Environment (Apache Axis, .NET etc.) WS-* from OASIS/W3C/ Industry Apache Axis .NET etc.

Must set standards to get interoperability

The Ten areas covered by the 60 core WS-* Specifications WS-* Specification Area 1: Core Service Model 2: Service Internet 3: Notification 4: Workflow and Transactions 5: Security 6: Service Discovery 7: System Metadata and State 8: Management 9: Policy and Agreements 10: Portals and User Interfaces Examples

XML, WSDL, SOAP WS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM WS-Notification, WS-Eventing (Publish-Subscribe) BPEL, WS-Choreography, WS-Coordination WS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation UDDI, WS-Discovery WSRF, WS-MetadataExchange, WS-Context WSDM, WS-Management, WS-Transfer WS-Policy, WS-Agreement WSRP (Remote Portlets)

Activities in Global Grid Forum Working Groups

GGF Area 1: Architecture 2: Applications 3: Compute GS-* and OGSA Standards Activities

High Level Resource/Service Naming (level 2 of slide 6), Integrated Grid Architecture Software Interfaces to Grid, Grid Remote Procedure Call, Checkpointing and Recovery, Interoperability to Job Submittal services, Information Retrieval, Job Submission, Basic Execution Services, Service Level Agreements for Resource use and reservation, Distributed Scheduling

4: Data 5: Infrastructure 6: Management 7: Security

Database and File Grid access, Grid FTP, Storage Management, Data replication, Binary data specification and interface, High-level publish/subscribe, Transaction management Network measurements, networking, Data transport Role of IPv6 and high performance Resource/Service configuration, deployment and lifetime, Usage records and access, Grid economy model Authorization, P2P and Firewall Issues, Trusted Computing

Net-Centric Core Enterprise Services

Core Enterprise Services NCES1: Enterprise Services Management (ESM) NCES2: Information Assurance (IA)/Security NCES3: Messaging NCES4: Discovery NCES5: Mediation NCES6: Collaboration NCES7: User Assistance NCES8: Storage NCES9: Application Service Functionality

including life-cycle management Supports confidentiality, integrity and Implies reliability and autonomic features Synchronous or asynchronous cases Searching data and services availability.

Includes translation, aggregation, integration, correlation, fusion, brokering publication, and other transformations for services and data. Possibly agents Provision and control of sharing with emphasis on synchronous real-time services Includes automated and manual methods of optimizing the user GiG experience (user agent) Retention, organization and disposition of all forms of data Provisioning, applications.

operations and maintenance of

Produce the Needed Core Services

• We can classify services in many ways and following 2 charts are one way; slightly changed from proposal as NCOW and our work changed a little.

• Green is “in hand”; our team leaders • Orange is “in hand” with outside but available solutions • Red has problems – Security does not have industry consensus while current Scheduling work does not address DoD real-time service and network requirements

The Core

F

eatures/

S

ervice Areas I

Service or Feature WS-* GS-* NCES (DoD) Comments A: Broad Principles FS1: Use SOA: Service Oriented Arch.

FS2: Grid of Grids

WS1 Core Service Architecture , Build Grids on Web Services. Industry best practice Distinctive Strategy for legacy subsystems and modular architecture

B: Core Services FS3: Service Internet, Messaging FS4: Notification FS5 Workflow FS6 : Security FS7: Discovery

WS2 WS3 WS4 WS5 WS6 GS7 NCES3 Streams/Sensors. Team NCES3 NCES5 NCES2 NCES4 JMS, MQSeries. Grid Programming Grid-Shib, Permis Liberty Alliance ...

UDDI

FS8: System Metadata & State FS9: Management FS10: Policy

WS7 WS8 WS9 GS6 NCES1 ECS Globus MDS Semantic Grid, WS-Context CIM

The Core

F

eature/

S

ervice Areas II

NCES Service or Feature FS12: Computing WS-* B: Core Services (Continued) FS11: Portals and User assistance

WS10

GS-*

GS3 NCES7

Comments

Portlets JSR168, NCES Capability Interfaces

FS13: Data and Storage

GS4 NCES8

FS14: Information FS15: Applications and User Services FS16: Resources and Infrastructure FS17: Collaboration and Virtual Organizations FS18: Scheduling and matching of Services and Resources

GS4 GS2 GS5 GS7 GS3 NCES9 NCES6 NCOW Data Strategy Federation at data/information layer major research area; CGL leading role JBI for DoD, WFS for OGC Standalone Services Proxies for jobs Ad-hoc networks XGSP, Shared Web Service ports Current work only addresses scheduling “batch jobs”. Need networks and services

Additional Services

• Sensors have low level support listed as FS3; higher level integration using SensorML and Filters well understood. Some work in phase I • GIS Grid services shown in phase I pioneered by team and already • Mediation (Interoperability) Services needed to link Grids (defined as a collection of ≥ 1 Services) – Need to generalize existing solutions for Sensor Grids and for MQSeries-SOAP Mediation – View NaradaBrokering as a SOAP Intermediary

Grid of Grids Building Tool

• This will provide a graphical interface to build grids from existing libraries of Services and Grids • Meta-data (provenance) needs to be specified • Grids (services) need to be linked • This will be built by extending an existing workflow engine which is aimed at a more tightly couple version of the builder problem – We will consider HPSearch (CGL), Taverna (Open source from UK OMII), BPEL with user interface (OMII or IU LEAD project) – Semantic Grid provenance “add-ons” – We have experience with all these approaches

Applications

• Indiana University will apply to Earthquake Science and Emergency Response • Ball Aerospace will apply to Matrix • Anabas will work with Technical Advisory Board to identify commercialization plan and possible prototype applications in first 12 months • Anabas will implement appropriately in last half of project

Out of Scope for Phase II

• Many areas are still evolving significantly – Mediation/Interoperation – Security – Scheduling of non-compute Resources – Data/Information Federation – Semantic Grid and management • We will not test scalability on large number of services, sensors and component Grids • Integrating legacy systems not addressed • Grid of Grids building tool is “new idea” – can expect will benefit from further work

Research Tasks

The R&D Effort is divided into five major tasks: • Task 1: Implementation of Collaboration Grid Middleware • Task 2: Enhanced NCOW Core Enterprise Services (NCES) with Enterprise Control Services and Metadata Services • Task 3: Design and implementation of Grid of Grids mediation algorithms and NCOW services • Task 4: Design and Implementation of Net-Centric Collaboration Grid Builder Tool • Task 5: Technology Demonstration

Task Organization

1. Requirements 2. Analysis 3. Design 4. Implement 5. Test & Measure 6. Repeat

Task Assignments

Task Requirements Collaboration Grid Middleware (NCGGCM) NCOW, GiG CES+ (11 service layers) GWSA Grid of Grids (GoG) Mediation Algorithms K 2 service NCOW services (e.g., JBI publish, etc.) Net-Centric Collaboration Grid Builder Tool CES+ Wizards for each service Collaboration and Messaging GIS, Sensor and additional services GIS and sensor K 2 service NCOW services (e.g., JBI publish, etc.) C2IEDM service Demonstration/Commercial Products Crisis Management for Earthquake Customer provided suggestions for DoD product Anabas (P) (P) (P) (P) (P) (P) (S) (S) (P) (P) (P) (P) (P) (P) (S) (S) (S) (C) (C) (C) Ball (S) (S) (S) (S) (S) (S) (P) (P) (S) (S) (S) (S) (S) (S) (P) (P) (P) (C) (S) (C) (S) (C) (S) (S) (S) (S) (S) (S) CGL (S) (S) (S) (S) (S)

Task Assignments

Task Research and Concept Analysis Collaboration Grid Middleware (NCGGCM) Grid of Grids (GoG) Net-Centric Collaboration Grid Builder Tool GIS, Sensor and additional services Demonstration/Commercial Products Anabas (P) (P) (P) (P) (P) Ball (S) (S) (S) (S) (S) CGL (S) (S) (S) (S) (S)

Task Assignments

Task Design Collaboration Grid Middleware (NCGGCM) Grid of Grids (GoG) Mediation Algorithms K 2 service NCOW services (e.g., JBI publish, etc.) Net-Centric Collaboration Grid Builder Tool GIS, Sensor and additional services GIS and sensor K 2 service NCOW services (e.g., JBI publish, etc.) C2IEDM Services Demonstration/Commercial Products Anabas (P) (P) (P) (S) (S) (P) (P) (P) (S) (S) (S) (P) Ball (S) (S) (S) (P) (P) (S) (S) (S) (P) (P) (P) (S) CGL (S) (S) (S) (S) (S) (S)

Task Assignments

Task Implement Collaboration Grid Middleware (NCGGCM) Grid of Grids (GoG) Mediation Algorithms K 2 service NCOW services (e.g., JBI publish, etc.) Net-Centric Collaboration Grid Builder Tool GIS, Sensor and additional services GIS and sensor K 2 service NCOW services (e.g., JBI publish, etc.) C2IEDM service Demonstration/Commercial Products Anabas (P) (P) (P) (S) (S) (P) (P) (P) (S) (S) (S) (P) Ball (S) (S) (S) (P) (P) (S) (S) (S) (P) (P) (P) (S) CGL (S) (S) (S) (S) (S) (S)

Task Assignments

Task Testing & Evaluation Collaboration Grid Middleware (NCGGCM) Grid of Grids (GoG) Mediation Algorithms K 2 service NCOW services (e.g., JBI publish, etc.) Net-Centric Collaboration Grid Builder Tool GIS, Sensor and additional services GIS and sensor K 2 service NCOW services (e.g., JBI publish, etc.) C2IEDM service Demonstration/Commercial Products Anabas (P) (P) (P) (S) (S) (P) (P) (P) (S) (S) (S) (P) Ball (S) (S) (S) (P) (P) (S) (S) (S) (P) (P) (P) (S) CGL (S) (S) (S) (S) (S) (S)

Task Assignments

Task Refine and Refactor Collaboration Grid Middleware Grid of Grids (GoG) Net-Centric Collaboration Grid Builder Tool GIS, Sensor and additional services Demonstration/Commercial Products Anabas (P) (P) (P) (P) (P) Ball (S) (S) (S) (S) (S) CGL (S) (S) (S) (S)

Milestones & Timelines

Task and Event

Requirements Research & Analysis 1 Design Implement Testing & Evaluation Refine & Refactor Documentation Monthly Report Final Report Reviews & Demonstrations 2 3 4 5 6 7 8 9 10 11 Month from Contract Start 12 13 14 15 16 17 18 19 20 21 22 23 24