Building Grid Portals with OGCE: Big Red Portal and GTLAB

Mehmet A. Nacar, Jong Youl Choi, Marlon Pierce, Geoffrey Fox Community Grids Lab Indiana University

Introduction

 This is a two-part talk on OGCE activities  Part 1: using OGCE’s current grid portal release to build a basic gateway for IU’s BigRed  Jong Youl Choi  Zero to Done in 2 months. YMMV  Part 2: simplifying the process for building portals like this using tag libraries.

 Mehmet A. Nacar  Finishing up ~2 years of software development  Big Red supercomputer is part of TeraGrid at Indiana University  2048 cluster nodes, 4 terabyte memory  For more information on Big Red, see the Monday tutorial.

2

Portlets for Big Red

 We have developed several portlets for Big Red based on the OGCE software  Improved batch job submission for LoadLeveler (IU) and PBS (NCSA).

 MEME job submission  Interactive and batch  Includes both GridFTP and GRAM clients  Job tracking.

 MOAB queue monitoring for both entire machine and the specific user.

 Access to existing IU resources like the IU Knowledge Base.

 These portlets also work with other TeraGrid resources.

 We tested across IU and NCSA resources.

Meme Portlet

The MEME portlet uses OGCE portal libraries to upload and download files, submit jobs, and monitor their progress.

5

Dashboard Portlet

The dashboard portlet allows users to track jobs on the selected resource. The user can view either his own set of jobs or get information on all submitted jobs.

6

OGCE IFrame Portlet can be used to integrate external sites.

OGCE GPIR portlet can interoperate with TeraGrid GPIR services.

Manage TeraGrid MyProxy credentials with the OGCE ProxyManager portlets.

OGCE file management client portlets interact with TeraGrid GridFTP servers.

In addition to MEME, general purpose batch and interactive job submission to GRAM, WS-GRAM is supported.

Using TeraGrid Resources

 Although utilizing Big Red, the portlets can be used with any other gateway in TeraGrid  Certificate username and local system user names may be different.

 To avoid confusion, we submit a whoami command to retrieve correct username at first place.

 Aside: Why can’t we have just one user name for all TeraGrid resources?

 GSI-enabled SSH users: see http://communitygrids.blogspot.com/2007/06/teragrid-single-sign on.html

 Dashboard portlet facilitates this schema by using GRAM service.

13

Problems with Portlet Development

 Grid portlets typically wrap each single Grid capability in a separate portlet  Problem is that Grid portlets need to combine these operations  Portlets are entire web applications, so we need a component model for portlets: reusable portlet parts  JSF uses Inversion of Control pattern to implement its Model View-Controller architecture   Remove dependencies on the Servlet API.

Backing beans are just beans, so can be reused more easily outside of web and portlet applications.

  JSF also provides an extensible framework (tag libraries) Apache JSF portlet bridge allows you to convert standalone JSF applications ( development phase ( deployment phase ).

) into portlets

Grid Tag Libraries and Beans (GTLAB)

       GTLAB provides common components for building portlets using tags and reusable parts.

The goal of GTLAB to simplify Grid portlet development  Enable rapid development GTLAB capabilities include Grid operations with XML based tags within

Java Server Faces

(JSF) framework.

Grid tag libraries are built using JSF custom component development techniques Grid tags are interfaces to backing Grid beans  End users pass values to Grid beans by using tag attributes.

We build on

Java CoG 4’s abstraction layer

.

Each backing Grid bean has equal capability with a portlet application in case of Grid portlet approach.

15

GTLAB Example

• • Grid tags are associated with Grid services via Grid beans • Grid Beans wrap the Java COG Kit (version 4) We show an example JSF page section below.

• This allows you to develop new Grid portlets with no additional Java code.

<

o:submit

id=”test” action=”next_page” /> <

o:myproxy

id=”pr” hostname=”gf1.ucs.indiana.edu” port=”7512” lifetime=”2” username=“mnacar” password=”***” /> <

o:jobsubmit

id=”task” hostname=”cobalt.ncsa.teragrid.org” provider=”GT4” executable=”/bin/ls” stdout=”tmp/result stderr=”tmp/error” /> <

/o:submit

> 16

Grid Tags

Associated Grid Beans Features

ComponentBuilderBean

Creating components, job handlers, submitting jobs

MonitorBean

Handling monitoring page actions

MultitaskBean

Constructing simple workflow

MultitaskBean MyproxyBean

Defining dependencies among sub jobs Retrieving myproxy credential

FileOprationBean

Providing Gridftp operations

JobSubmitBean FileTransferBean ResourceBean

Providing GRAM job submissions Providing Gridftp file transfer Describes common properties among all tags and beans. Passing values given by standard visual JSF components.

Multi-staged task Task A File Operation (mkdir) depends

How to integrate GTLAB

Input Task B File Transfer depends Task C Job Submission depends Task D File Transfer Output  Multitask provides a simple Directed Acyclic Graph (DAG)  This example demonstrates a composite Grid job using multi staged

multitask

 GTLAB handles lifecycle of DAG within JSF application 18

How to prepare application pages

 Developers embed Grid tags snippet into JSF page  These components are non-visual and are not displayed in HTML.

 Resource bean provides bridging with form inputs and GTLAB framework.



 Dynamic values to Grid tag attributes are provided by Resource bean.

 Only visual component is

tag that is associated with action method of GTLAB.

19

GTLAB Dashboard Portlet Example

<

o:submit

< id=”track” action=”list_page” />

o:multitask

id=”dashboard” taskname=”track” persistent=”true” > <

o:myproxy

id=”proxy” hostname=”gf1.ucs.indiana.edu” lifetime=”2” username=”#{resource.username}” password=”#{resource.password}” /> <

o:jobsubmit

id=”jobA” hostname=”cobalt.ncsa.teragrid.org” provider=”GT4” executable=”/bin/whoami” stdout=”tmp/result” stderr=”tmp/error” /> <

o:jobsubmit

id=”jobB” hostname=”cobalt.ncsa.teragrid.org” provider=”GT4” executable=”/bin/showq” stdin=”tmp/result” stdout=”tmp/list” stderr=”tmp/error” /> <

o:dependency

id=”depend” task=”jobB” dependsOn=”jobA” /> <

/o:multitask

> <

/o:submit

> 20

Tracking and Managing Jobs

 GTLAB manages lifecycles of jobs and monitor their status.

 Grid operations are usually batch processes  We provide callback mechanism to follow up the jobs  GTLAB creates handlers for jobs and persistently stores them.

 GTLAB handlers manages the job events such as stop, cancel or resuming the running jobs.

 GTLAB provides archive for job metadata and allows managing the archive  Handler tag helps to organize user’s job repository 

21

GTLAB Related Work

 Grid Portlets 1.3 of GridSphere  Now they are trying to decouple with GridSphere. It’s called

Vine (Portlet Vine)

as separate project  Grid Portlets 1.3 provide API and UI tags to build Grid portlets  RSF (Reasonable Server Faces)  Derived from JSF, but it separates HTML pages and backing beans  RSF provides non-visual components unlike JSF  Beans can be contained by Spring like containers.

 Lifecycles of beans managed by Spring  OGCE portlets  Packages Velocity, JSP and JSF portlets  Provides portlet package for several Grid applications such as Globus, Condor, SRB and GPIR

More Information

 GTLAB version 1.0 Beta release available at  http://grids.ucs.indiana.edu/users/manacar/GTLAB website  See link from main OGCE web site  http://www.collab-ogce.org

 Contact OGCE:

[email protected]

 Contact me:

[email protected]

 Contact Mehmet:

[email protected]

23

Conclusion and Future Work

 We have described a science gateway to IU’s Big Red supercomputer.

 Initial effort was to build the portal based on OGCE portlets  Described the process for creating Grid portlets using GTLAB.

 We gained rapid development by using reusable components  We are investigating abstracting our approach to support different workflow engines.

 Investigate integrating AJAX with our session management and monitoring.

 Please come to the Web 2.0 BOF Tuesday evening.

24

Acknowledgments

 This work is supported by the National Science Foundation’s Information Technology Research (NSF grant ITR 0428774, 0427264, 0426867 VLab) and Middleware Initiative (NSF Grant SCI 0330613) programs.