Grid - Istituto Nazionale di Fisica Nucleare
Download
Report
Transcript Grid - Istituto Nazionale di Fisica Nucleare
Globus Toolkit
Massimo Sgaravatto
INFN Padova
Introduction
Grid Services: LHC regional centres
need distributed computing
Analyze toolkits that integrate
distributed computational resources:
Globus
Legion
Globe
….
Massimo Sgaravatto
Globus
Basic research on GRID problems
Globus Toolkit
“Bag of services” for developing
GRID tools and applications
Available for most Unix platforms
Limited functionalities for Win32
Massimo Sgaravatto
Globus Architecture
Applications
Testbed Status
High-level Services and
Tools CC++
MPI
MPI-IO
Nimrod/G
globusrun
GlobusView
DUROC
Core Services
Nexus
Metacomputing
Directory
Service
Gloperf
Condor
MPI
LSF
Easy
NQE
Globus
Security
Interface
Local
Services
Massimo Sgaravatto
GRAM
Heartbeat
Monitor
AIX
GASS
TCP
UDP
Irix
Solaris
Core Services
Security (GSI)
Information Service (MDS)
Resource Management (GRAM)
Remote Data Access (GASS)
Communication (Nexus)
Network Performance Monitor (GLOPERF)
Fault detection (HBM)
Executable Management (GEM) (???)
Massimo Sgaravatto
Globus Security Infrastructure
Main focus: authentication
single authentication
not necessary to manage authentication on
each GRID node
Mapping between GRID user-id and local
user-id
Digital signature mechanisms
Massimo Sgaravatto
Metacomputing Directory Service
Static and dynamic information about GRID
components (host architecture, memory, load,
network bandwidth, latency. ...)
Information updated by:
Possibility to define new attributes
Globus services (i.e. GRAM Reporter)
Users/Applications
Other “providers” (i.e. SNMP)
Tools and API for discovering, publishing, and
accessing information
Standard protocol and interface: LDAP
Single MDS server (located in USA)
Massimo Sgaravatto
Resource Management
Management of computational resources
GRAM (Globus Resource Allocation Manager)
responsible for a set of “local” resources
Single interface to different resource managers
(Condor, LSF, NQE, fork, ...)
Resource Specification Language (RSL) for defining:
resource requirements (machine type, memory, ...)
job definition (executable, parameters, ...)
I.e.: & (ResourceManagementContact =
“cond.pd.infn.it:/C=US/.../CN=cond.pd.infn.itcondor”) (count=100) (executable=myapp)
Massimo Sgaravatto
Globus Resource Manager Architecture
Application
Resource and (High level RSL expression)
Job specification
MDS
Resource
discovery
RSL specialization
Broker
Create jobs
(Ground RSL expressions)
Site1
GRAM
GRAM
GRAM
CONDOR
LSF
fork
Site2
Massimo Sgaravatto
Site3
Remote Data Access
GASS (Global Access to Secondary Storage)
Supported servers: GASS, HTTP, FTP
Uniform naming using URL:
I.e.: x-gass://host1.pd.infn.it:1234/filename
Possibility to access remote data sets (distributed
file system)
Necessary to modify source code:
open() globus_gass_open()
close() globus_gass_close()
Caching
Possibility to move files from/to remote data
storage systems
Massimo Sgaravatto
Other core services
Nexus
HBM (Heartbeat Monitor)
Unicast and multicast low level communication library
Supports different programming models (message
passing, RPC, etc…)
Monitors the status of registered processes
GLOPERF (Globus Network Performance
Measurement Tool)
Measures throughput and latency between node pairs
Massimo Sgaravatto
Evaluation of the Globus Toolkit
5 sites Testbed (Bologna, CNAF, LNL, Padova,
Roma1)
Use case: HLT CMS studies
MC Prod. Complete HLT chain
Services
Resource management
fork() Interface to different local resource managers (Condor, LSF)
Resources chosen by hand Broker to implement a Global resource
manager
Data Mover
to stage executable and input files
to retrieve output files
Massimo Sgaravatto
Use Case: CMS HLT studies
Massimo Sgaravatto
Status
Globus installed on 5 Linux PCs in 3 sites
GSI
MDS
Seems working
Lots of problems, maybe temporary?
GRAM, GASS, Gloperf, HBM
Work in progress
Massimo Sgaravatto
Other problems
Documentation not well organized
Collaboration with HPC Lab, University
of Lecce is very useful !
Massimo Sgaravatto