Transcript Recommending a Strategy

Portal Software:
Browser-based Monte Carlo
Zisis Papandreou
University of Regina
GlueX Collaboration Meeting
JLab, December 11-13, 2003
SPARRO Group, University of Regina
1
Outline

Grid2003 Conference


Grid Services
Regina Portal
Application framework, Globus
 Meta queuing


Collaboration tools:

GlueX Forum, Database Administrator
SPARRO Group, University of Regina
2
Grid2003 Conference (Ed, Zisis)



Phoenix, AZ, November 16-17, 2003
Concurrent with SC2003 (8,000 hackers)
Interactive and coached tutorial:

How to Build a Grid Service using the
Globus Toolkit® 3 (Argonne people)


Written in Java
Built (JDK) and run on Ed’s RH9.0 linux laptop
SPARRO Group, University of Regina
3
OGSA
GT3 Architecture





SPARRO Group, University of Regina
Core: Grid Services
Security: SSL, X.509
Base: Job Services,
Index Services, RFT
Data: Replica
Management (data subsets)
Other: foreign grid
services
4
Creating a Grid Service
Web Services Description Language





Write WSDL directly
Generate WSDL from
interface language (Java)
Decorate it
Generate stubs
Implement it
package gt3tutorial.core.first.impl;
public interface Math
{
public int add(int a, int b);
public int subtract(int a, int b);
public int multiply(int a, int b);
public float divide(int a, int b);
}
javac Math.java
java org.apache.axis.wsdl.Java2WSDL
-P MathPortType -S MathService
-l http://localhost/ogsa/services/core/first/MathService
-n http://first.core.gt3tutorial/Math gt3tutorial.core.first.impl.Math
SPARRO Group, University of Regina
5
Exercises accomplished





Grid Services / GT3
FileShare Service
deployment
File sharing with
identifiers
Adding service data
Register with
community index
SPARRO Group, University of Regina





Lifetime management
File searching
Security
Transience: create
and destroy
FileShares
Grid Service
Publishing
6
Demonstration:
FileShare Grid Service
(Ed Brash)
SPARRO Group, University of Regina
7
Regina Portal: Beefed up
https://www.phys.uregina.ca/GlueX/


Glasgow: job submission, review (PHP)
New features:
ColdFusion Application Framework (CFML)
 User registration & authentication
 Integration with GridPort TK, Globus
 Upload XML Geometry and compile
 Experiment or commit executables to db

SPARRO Group, University of Regina
8
Demonstration:
Regina Portal
(Zisis Papandreou)
SPARRO Group, University of Regina
9
Grid Computing




“The Grid”: distributed computing infrastructure (both
software and hardware) that provides access to high-end
computational capabilities
Grid computing enables the shared use of computational
resources such as data archives and simulations.
Resource owners offer resources subject to constraints
Resource consumers also place constraints on properties
SPARRO Group, University of Regina
10
Queuing, Scheduling &
Meta Scheduling
(Aadila Soobrattee)



Regina openSpace cluster (50 CPUs) is governed by
OpenPBS local queuing software
Need capabilities of scheduling workload among a
computer grid where local scheduling software and local
policies may differ: meta-scheduler
Schemes:



Centralized: surface info, locals dispatch jobs
Hierarchical: meta and local share info; no flexible re-routing
Distributed: scheduler/dispatcher model; flexible, scalable
SPARRO Group, University of Regina
11
Distributed Scheduling
Jobs
Job flow
Information flow
Metascheduler
Local
Scheduler/
Dispatcher




Local
Scheduler/
Dispatcher
Local
Scheduler/
Dispatcher
Meta schedulers at every site
Jobs submitted to meta scheduler where job originates
Meta schedulers query each other to collect instantaneous load
info; job is transferred to the site with the lowest load
Dagman, Condor/G, Nimrod/G, GT3/Platform, Silver/Maui
SPARRO Group, University of Regina
12
Silver/Maui
Jobs
Jobs
Meta-Scheduler
Meta-Scheduler
Local Scheduler
Local Scheduler
Jobs
Meta-Scheduler
Silver:
•reduces average job queue
time by enabling access to a
larger set of computer
resources
•intelligently decomposes jobs
into multi-system sub-jobs to
improve job turnaround time
•manages site-dependent job
environment and resource
requirements
•supports jobs which require
co-allocated resources which
expand multiple systems
•supports jobs too large to run
at any single site by spanning
jobs across multiple sites
Local Scheduler

Silver provides grid scheduling services for clusters using
OpenPBS, can operate on all major UNIX operating systems
and is capable of operating with or without Globus
http://www.supercluster.org/
SPARRO Group, University of Regina
13
1. User logs into Globus and gets Globus
credential
2. User submits job from site A via sqsub
client
3. sqsub translates job to global job and
stores job and credential in SILVER's
global queue
4. SILVER determines list of feasible
resources based on resources requested
5. SILVER queries Globus and determines
machines that support the specific
credential
6. SILVER queries each feasible resource
and obtains matrix of resource availability
times
7. SILVER determines resource
associated with earliest job completion
time (site B)
8. SILVER reserves resources on site B
9. SILVER stages input data and
executable to site B using Globus file
transfer (gssftp)
10. SILVER translates global job to RSL
and submits job to Globus with 'run on site
B' request
11. Globus translates RSL job to resource
manager command language and submits
the job to resource manager on site B.
12. MAUI detects job, executes it on
reserved resources
13. job completes
14. MAUI creates accounting record,
removes reservation, and updates SILVER
15. Output data is staged to requested
destination using Globus file transfer
SPARRO Group, University of Regina
14
Collaboration Tools

GlueX Forum (at UConn)

http://zeus.phys.uconn.edu/forums/gluex.org/index.php

Web-based email forum (written in PHP)
Category-based email, threads, moderators


PostgreSQL Database Administration



pgAdminIII (win & unix)
Remote connection to db servers
Create/delete tables, enter data, view data
SPARRO Group, University of Regina
15
What’s next?

Expand portal further:




Install Meta Queuing System at Regina
Web/Grid Services (document uploading, indexing and
searching)
XML Data Transfer (Web Distributed Data Exchange –
WDDX)
Communications:



Teleconferencing: AT&T
Videoconferencing: VRVS or Polycom
Email threaded discussions: GlueX Forum
SPARRO Group, University of Regina
16