Transcript Geodise Middleware

Geodise Middleware
& Optimisation
Graeme Pound, Hakki Eres, Gang Xue
& Matthew Fairman
Summer 2003
© Geodise Project, University of Southampton, 2003.
Computational Toolbox
• Access to Globus compute resources from
Matlab
• Proxy certificate management, job submission
and file transfer
• Complex grid-enabled workflows may be
coordinated from Matlab
• Uses the Java CoG 1.1
© Geodise Project, University of Southampton, 2003.
Computational Toolbox
gd_createproxy.m
Creates a Globus proxy certificate for the user's credentials
gd_destroyproxy.m
Destroys the local copy of the user's Globus proxy certificate
gd_certinfo.m
Returns information about the user's certificate
gd_proxyinfo.m
Returns information about the user's proxy certificate
gd_proxyquery.m
Queries whether a valid proxy certificate exists
gd_jobsubmit.m
Submits a compute job to a Globus GRAM job manager
gd_jobstatus.m
Gets the status of a Globus GRAM job
gd_jobpoll.m
Queries the status of a Globus GRAM job until complete
gd_jobkill.m
Kills a Globus GRAM specified by job handle
gd_putfile.m
Puts a remote file using GridFtp
gd_getfile.m
Retrieves a remote file using GridFtp
gd_rmfile.m
Deletes a remote file using GridFtp
gd_makedir.m
Creates a remote directory using GridFtp
gd_rmdir.m
Deletes a remote directory using GridFtp
© Geodise Project, University of Southampton, 2003.
Jython Computational Toolbox
• Python interface to functionality of Matlab
Computational Toolbox
• Jython is a 100% Java implementation of the
Python scripting language
• Lightweight alternative to the Matlab
Computational Toolbox
© Geodise Project, University of Southampton, 2003.
Job Submission Service Client
• Matlab client to job submission web service
• User detached from compute endpoint – Condor pool
• Machines publish available resources:
– HasProEngineer = TRUE
– ProEngineerVersion = "2001400"
– ProEngineerPath = "C:\Program Files\proe2001\bin
• DIME file transfer over http
• MATLAB Client Functions:
–
–
–
–
–
grid_platform – Describes the platform requirements of the job
grid_submit – Submits the job to the web service, returns a job ID
grid_poll – Polls job ID
grid_status – Queries job status
grid_results – Retrieves the output files
© Geodise Project, University of Southampton, 2003.
OptionsMatlab
• Matlab interface to the Options design
exploration system
• State of the art design search and optimisation
algorithms
• Objective and constraint functions exposed as
Matlab functions
• Grid-enabled job brokers easily incorporated
© Geodise Project, University of Southampton, 2003.
OptionsMatlab Parallel Architecture
Matlab environment
OptionsMatlab.dll
optjob
optfun
optcon
optjob.m
optfun.m
optcon.m
• optjob.m allows
evaluation of multiple
design points
• Concurrent evaluation of
optfun.m and optcon.m
using Objective Function
Service Client
• Single evaluation via
optjob.m from the optfun
and optcon subroutines
© Geodise Project, University of Southampton, 2003.
Objective Function Client
• Grid-enabled objective function evaluation
• Enables multiple concurrent evaluations
• Matlab functions provided:
– gd_objsubmit - Transfers required files, submits job and returns
handle
– gd_objvalue - Retrieves function evaluation, prints standard output of
failed jobs and cleans up files
– gd_objcleanup - Removes remote files of specified jobs
• Multiple Matlab processes, one Matlab licence
• Configuration file defines available objective function
servers
© Geodise Project, University of Southampton, 2003.
Objective Function Client
Matlab
gd_archive.m
Globus server
1
Geodise
file archive
2
gd_objsubmit.m
3
4
Matlab
gd_jobpoll.m
objfun.m
gd_objvalue.m
5
© Geodise Project, University of Southampton, 2003.
A Level 2 Grid Application
•
•
•
•
•
The aim is to find enhanced vibration
isolation performance for satellite
designs
Matlab is the scripting environment
Geodise toolkit is utilized
The GA toolbox for Matlab is used for
optimization
UK e-Science Grid resources used
–
–
–
–
–
–
Cambridge e-Science Centre
CLRC e-Science Centre
London e-Science Centre
Oxford e-Science Centre
Southampton e-Science Centre
Southampton Information Systems
Services
© Geodise Project, University of Southampton, 2003.
Parallel Job Submission (11x11)
© Geodise Project, University of Southampton, 2003.
Results (Sample points = 11x11)
© Geodise Project, University of Southampton, 2003.
Conclusions
• Submitting compute jobs is the easy bit.
• Complexity of grid computing includes:
– applying for authorisation at a numerous locations
– managing the transfer of files
– keeping track of heterogeneous compute resources
• Requirements for:
–
–
–
–
–
Grid enabled data management
Transparent Grid-enabled tools
Consistent computing environments
Reservation and co-allocation of resources
Grid debugging
© Geodise Project, University of Southampton, 2003.