Transcript Comparison of LCG

Enabling Grids for E-sciencE
Comparison of LCG-2
and gLite
Author E.Slabospitskaya
Location IHEP
www.eu-egee.org
INFSO-RI-508833
Outline
Enabling Grids for E-sciencE
• What are LCG-2 and gLite? Roles.
• gLite Architecture review
– What is gLite? Provenance of gLite from Alien, Grid3 et al.
– Schema of gLite Services. The main subsystems of gLite.
Comparison to LCG-2.
o
o
o
o
o
o
Workload management.
Data management
Information System
Security
Accounting
Alien
• Current status of gLite
• Mapping to physical machines
INFSO-RI-508833
Future EGEE middleware
Enabling Grids for E-sciencE
http://glite.web.cern.ch/glite
 Architecture:
https://edms.cern.ch/document/476451
 Design:
https://edms.cern.ch/document/487871
/
LCG-1
LCG-2
Globus 2 based
INFSO-RI-508833
gLite-1
gLite-2
Web services based
VDT
EDG
...
AliE
n
LCG
...
Provenance of gLite
Enabling Grids for E-sciencE
The gLite middleware is a Service Oriented Grid middleware providing services for managing
distributed computing and storage resources and the required security, auditing and information services.
Target server platform is Red Hat Linux 3.0 or any binary compatible distribution, such as Scientific
Linux, and Windows.
AliEn is a lightweight Grid framework which is built around Open Source components using the Web
Services model. It has been initially developed by the ALICE collaboration as a production
environment for the simulation, reconstruction, and analysis of Physics data in a distributed way.
The gLite Middleware is presently using a number of services taken from the AliEn system,
namely the File Catalogue and the Task Queue (part of the AliEn Core Services), the Package
Manager (part of the AliEn Site Services) and the GAS or Grid Access Service.
INFSO-RI-508833
gLite and LCG-2
LCG
Enabling Grids for E-sciencE
LCG-2
focus on production, large-scale
data handling
•
•
•
•
infrastructure
Operation, VO
management
gLite
focus on analysis
prototyping •
The service for the 2004 data
challenges
Provides experience on operating
and managing a global grid service
Development programme driven by
data challenge experience
 Data handling
 Strengthening the

•
LCG-2 (=EGEE-0)
2004
•
prototyping
•
product
2005
Evolves to LCG-3 as components
progressively replaced with new
middleware
product
•
•

•
-- target is to minimise the
discontinuities of migration
to the new generation
Aim for migration plan by end of
year

LCG-3
Developed by EGEE project in
collaboration with VDT (US)
LHC applications and users closely
involved in prototyping &
development (ARDA project)
Short development cycles
Co-existence with LCG-2
Profit as far as possible from LCG2 infrastructure, experience
Ease deployment – avoid separate
hardware
As far as possible - completed
components integrated in LCG-2
improved testing, easier
displacement of LCG-2
EGEE-1
les robertson - cern-it-5
INFSO-RI-508833
gLite Services
Enabling Grids for E-sciencE
INFSO-RI-508833
GAS
Enabling Grids for E-sciencE
The Grid Access Service provides access to different Grid Services
as an adapter exposes a flat interface to the collection of componen
File Catalog
Clien
t
GAS
Metadata
catalogue
WMS
API*for
appl
INFSO-RI-508833
Job Management
Enabling Grids for E-sciencE
•A CE interfaces the local resource management system (e.g. LSF, PBS) to the Grid middleware. The
currently
released CE will be enhanced with a CE Monitor that pulls jobs from the WMS.
.
•The Worker Nodes behind the local resource management system host all the necessary clients to
interact with the Grid middleware from within a job.
•The Package Manager allows the dynamic installation of application software.
INFSO-RI-508833
Enabling Grids for E-sciencE
INFSO-RI-508833
WMS
AliEn Task Queue and EDG/LCG RB
Enabling Grids for E-sciencE
WMS
GA
S
User
INFSO-RI-508833
L&
B
PUS
H
SUBMIT
CE
TASK
QUEUE
PULL
REQUEST
MONITOR
Submit
jobs
Enabling Grids for E-sciencE
CE
• Works in push and pull mode
• Site policy enforcement
• Exploit new globus GK and
CondorC (close interaction with
globus and condor team)
INFSO-RI-508833
CEA … Computing Element Acceptance
JC … Job Controller
MON … Monitoring
LRMS … Local Resource Management
System
Data Management
Enabling Grids for E-sciencE
3 main service groups that relate to data and file access are:
.
Storage Element,
Catalogue Services
Data Transfer Scheduling .
Closely related to the data services are the security-related services and the Package Manager.
•Data is stored in an SRM based storage system.
•The gLite-I/O server allows posix-like access to files stored in the SRM
•The local catalogue (LC) keeps track of the LFN:GUID:SURL mapping of local files
The file transfer/file placement service is used for moving files.
INFSO-RI-508833
Catalogs
Enabling Grids for E-sciencE
•
File Catalog
•
– Filesystem-like view on logical file names
– Keeps track of sites where data is stored
– Conflict resolution
Metadata
Catalog
Metad
ata
GUID
– Keeps information at a site
•
(Meta Data Catalog)
– Attributes of files on the logical
level
– Boundary between generic
middleware and application layer
GUID
LFN
LFN
Replica Catalog
SURL
Site ID
Site ID
LFN
GUID
SURL
Replica Catalog Site A
INFSO-RI-508833
File Catalog
SURL
SURL
Replica Catalog Site B
Storage Element Interfaces
Enabling Grids for E-sciencE
• SRM interface
– Management and control
– SRM (with possible evolution)
Control
• Posix-like File I/O
– File Access
– Open, read, write
– Not real posix (like rfio)
User
INFSO-RI-508833
POSIX
API
File
I/O
SRM interface
rfio
dcap
chirp
aio
Castor
dCache
NeST
Disk
Information System
Enabling Grids for E-sciencE
.
The R-GMA (Relational Grid Monitoring Architecture) servlet accepts connection
from clients (producers), i.e. the services publishing and user jobs publishing information, and
forwards the information to the appropriate consumers.
R-GMA is composed of the following services:
a.
R-GMA server
b. R-GMA client is a set of client API in C,C++,Java and Python for the access the information
and monitoring functionality of the R-GMA system
c. R-GMA site-publisher (each site) is responsible for publishing site information to the R-GMA
server.
d. R-GMA service tool -regularly scans config files and updates a current Service Status RGMA Table. CLI exists for modify these files and query the service table.
INFSO-RI-508833
R-GMA
Enabling Grids for E-sciencE
R-GMA is based on the Grid Monitoring Architecture (GMA) from the Grid Global Forum
(GGF), which is a simple Consumer-Producer model that models the information infrastructure
of a Grid as a set of consumers (that request information), producers (that provide information)
and a central registry which mediates the communication between producers and consumers.
R-GMA offers a global view of the information as if each Virtual Organization had one large
relational database.
Producers contact the registry to announce their intention to publish data, and consumers
contact the registry to identify producers, which can provide the data they require. The data
itself passes directly from the producer to the consumer: it does not pass through the registry.
R-GMA adds a standard query language (a subset of SQL) to the GMA model,
INFSO-RI-508833
Mapping gLite Services onto physical machines
Enabling Grids for E-sciencE
INFSO-RI-508833
gLite Services and components
Enabling Grids for E-sciencE
The following high level services are part of gLite middleware
--Accounting
Service (DGAS)
•R-GMA Servers, Client and Service Tools
- AliEn File Catalogue (global catalogue – GC)
•VOMS and VOMS administration tools
- File & Replica Catalogue (local catalogue – LC)
•Authorization, Authentication and Delegation Services
- Metadata Catalogue
•Standard Worker node (WN)
- Computing Element (CE)
•User Interface
- File Transfer and Placement Service
•Workload Manager System (WMS)
- gLite-I/O Server and Client
•Grid Access Service (GAS)
- Logging and Bookkeeping Server (LB)
•Job Provenance
- Package Manager
INFSO-RI-508833