Transcript Grid - Istituto Nazionale di Fisica Nucleare

Globus Toolkit
Massimo Sgaravatto
INFN Padova
Introduction


Grid Services: LHC regional centres
need distributed computing
Analyze toolkits that integrate
distributed computational resources:




Globus
Legion
Globe
….
Massimo Sgaravatto
Globus


Basic research on GRID problems
Globus Toolkit
 “Bag of services” for developing
GRID tools and applications
 Available for most Unix platforms
 Limited functionalities for Win32
Massimo Sgaravatto
Globus Architecture
Applications
Testbed Status
High-level Services and
Tools CC++
MPI
MPI-IO
Nimrod/G
globusrun
GlobusView
DUROC
Core Services
Nexus
Metacomputing
Directory
Service
Gloperf
Condor
MPI
LSF
Easy
NQE
Globus
Security
Interface
Local
Services
Massimo Sgaravatto
GRAM
Heartbeat
Monitor
AIX
GASS
TCP
UDP
Irix
Solaris
Core Services








Security (GSI)
Information Service (MDS)
Resource Management (GRAM)
Remote Data Access (GASS)
Communication (Nexus)
Network Performance Monitor (GLOPERF)
Fault detection (HBM)
Executable Management (GEM) (???)
Massimo Sgaravatto
Globus Security Infrastructure

Main focus: authentication

single authentication



not necessary to manage authentication on
each GRID node
Mapping between GRID user-id and local
user-id
Digital signature mechanisms
Massimo Sgaravatto
Metacomputing Directory Service

Static and dynamic information about GRID
components (host architecture, memory, load,
network bandwidth, latency. ...)


Information updated by:





Possibility to define new attributes
Globus services (i.e. GRAM Reporter)
Users/Applications
Other “providers” (i.e. SNMP)
Tools and API for discovering, publishing, and
accessing information
Standard protocol and interface: LDAP

Single MDS server (located in USA)
Massimo Sgaravatto
Resource Management




Management of computational resources
GRAM (Globus Resource Allocation Manager)
responsible for a set of “local” resources
Single interface to different resource managers
(Condor, LSF, NQE, fork, ...)
Resource Specification Language (RSL) for defining:


resource requirements (machine type, memory, ...)
job definition (executable, parameters, ...)
I.e.: & (ResourceManagementContact =
“cond.pd.infn.it:/C=US/.../CN=cond.pd.infn.itcondor”) (count=100) (executable=myapp)
Massimo Sgaravatto
Globus Resource Manager Architecture
Application
Resource and (High level RSL expression)
Job specification
MDS
Resource
discovery
RSL specialization
Broker
Create jobs
(Ground RSL expressions)
Site1
GRAM
GRAM
GRAM
CONDOR
LSF
fork
Site2
Massimo Sgaravatto
Site3
Remote Data Access



GASS (Global Access to Secondary Storage)
Supported servers: GASS, HTTP, FTP
Uniform naming using URL:


I.e.: x-gass://host1.pd.infn.it:1234/filename
Possibility to access remote data sets (distributed
file system)

Necessary to modify source code:




open()  globus_gass_open()
close()  globus_gass_close()
Caching
Possibility to move files from/to remote data
storage systems
Massimo Sgaravatto
Other core services

Nexus



HBM (Heartbeat Monitor)


Unicast and multicast low level communication library
Supports different programming models (message
passing, RPC, etc…)
Monitors the status of registered processes
GLOPERF (Globus Network Performance
Measurement Tool)

Measures throughput and latency between node pairs
Massimo Sgaravatto
Evaluation of the Globus Toolkit


5 sites Testbed (Bologna, CNAF, LNL, Padova,
Roma1)
Use case: HLT CMS studies


MC Prod.  Complete HLT chain
Services

Resource management



fork()  Interface to different local resource managers (Condor, LSF)
Resources chosen by hand  Broker to implement a Global resource
manager
Data Mover


to stage executable and input files
to retrieve output files
Massimo Sgaravatto
Use Case: CMS HLT studies
Massimo Sgaravatto
Status


Globus installed on 5 Linux PCs in 3 sites
GSI


MDS


Seems working
Lots of problems, maybe temporary?
GRAM, GASS, Gloperf, HBM
 Work in progress
Massimo Sgaravatto
Other problems


Documentation not well organized
Collaboration with HPC Lab, University
of Lecce is very useful !
Massimo Sgaravatto