EDG Tutorial - Architecture Overview
Download
Report
Transcript EDG Tutorial - Architecture Overview
The EU DataGrid Architecture
The European DataGrid Project Team
http://www.eu-datagrid.org
[email protected]
Contents
Middleware architecture overview
EDG structure
Job scheduling
Fabric management
Data Management
Monitoring
Storage
Networking
Summary
The EDG Architecture Tutorial - n° 2
EDG middleware architecture
Globus hourglass
Current EDG architectural functional blocks:
Basic Services ( authentication, authorization, Replica Catalog,
secure file transfer,Info Providers) rely on Globus 2.0
(GSI, GRIS/GIIS,GRAM, MDS)
Specific
application layer
ALICE
ATLAS
CMS
LHCb
Other apps
VO common
application
layer
GRID
middleware
GLOBUS
2.0
LHC
Other apps
High level GRID
middleware
Basic Services
OS & Net services
The EDG Architecture Tutorial - n° 3
DataGrid Architecture
Local Computing
Grid
Local Application
Local Database
Grid Application Layer
Data
Management
Job
Management
Metadata
Management
Object to
File Mapping
Collective Services
Information
& Monitoring
Replica
Manager
Grid
Scheduler
Underlying Grid Services
Database
Services
Computing
Element
Services
Storage
Element
Services
Replica
Catalog
Authorization
Authentication
& Accounting
Logging &
Bookkeeping
Grid
Fabric
Fabric services
Resource
Management
Configuration
Management
Monitoring
and Fault
Tolerance
Node
Installation &
Management
Fabric Storage
Management
The EDG Architecture Tutorial - n° 4
EDG middleware architecture: EDG
interfaces
Application
Developers
System
Managers
Local
Application
Grid Application Layer
Data
Job
Managem.
Managem.
Local Database
Scientists
Metadata
Managem.
Object to
File Map
Certificate
Authorities
Collective Services
Grid
Scheduler
File
Systems
Replica
Manager
Underlying Grid Services
SQL
Computing
Database
Element
Services
Services
Storage
Element
Services
Info &
Monitor
Replica
Catalog
Authorization
Authentication
Accounting
Logging &
Bookkeeping
Operating System
Fabric services
User Accounts
Resource
Managem.
Config
Managem.
Monitoring
Fault
Tolerance
Node
Installation
Managem.
Fabric
Storage
Managem.
Batch Systems
Mass Storage
Systems
HPSS, Castor
Storage
Elements
Computing Elements
The EDG Architecture Tutorial - n° 5
EDG middleware architecture:
The Workload Management System (WP1)
WP1 is responsible for the Workload Management System (WMS).
The WMS is currently composed by the following parts:
User Interface (UI) : access point for the user to the GRID ( using JDL)
Resource Broker (RB) : the broker of GRID resources, matchmaking
Job Submission System (JSS) : Condor-G; interfacing batch systems
Information Index (II) : an LDAP server used as a filter to select
resources
Logging and Bookkeeping services (LB) : MySQL databases to store Job
Info
The EDG Architecture Tutorial - n° 6
WP1: Work Load
Management
Local Application
Grid Application Layer
Job
Managem.
Components
Job Description Language
Grid
Scheduler
SQL
Database
Services
User Interface
UI : python (LB client : C++)
RB : C++
JSS : C++, python
II : LDAP server
LB: MySQL, C++
Input/Output Sandboxes: GridFTP
Object to
File
Mapping
Replica
Manager
Info &
Monitor
Computing
Element
Services
Storage
Element
Services
Replica
Catalog
Authorization
Authentication
Accounting
Logging &
Bookkeeping
Fabric services
Resource
Managem.
Implementation:
Metadata
Managem.
Underlying Grid Services
Information Index
Logging & Bookkeeping Service
Data
Managem.
Collective Services
Resource Broker
Job Submission Service
Local Database
Config
Management
Monitoring
Fault
Tolerance
Node
Installation
Management
Fabric Storage
Management
WMS main interfaces:
Globus Gatekeeper
WP2 Replica Catalog APIs
WP3 Information Systems
WP7 network monitoring info providers
End User (using JDL files, on the UI)
The EDG Architecture Tutorial - n° 7
EDG middleware architecture: WP1 (WMS)
The EDG Architecture Tutorial - n° 8
File Management
Site A
Site B
Storage Element A
Storage Element B
File Transfer
File A File X
File B File Y
File A File C
File B File D
The EDG Architecture Tutorial - n° 10
File Management
Replica Catalog:
Map Logical to Site files
Site A
Site B
Storage Element A
Storage Element B
File Transfer
File A File X
File B File Y
File A File C
File B File D
The EDG Architecture Tutorial - n° 11
File Management
Replica Catalog:
Replica Selection:
Map Logical to Site files
Get ‘best’ file
Site A
Site B
Storage Element A
Storage Element B
File Transfer
File A File X
File B File Y
File A File C
File B File D
The EDG Architecture Tutorial - n° 12
File Management
Replica Catalog:
Replica Selection:
Map Logical to Site files
Get ‘best’ file
Pre- Post-processing:
Prepare
Site
A
files for transfer
Validate files after transfer
Site B
Storage Element A
Storage Element B
File Transfer
File A File X
File B File Y
File A File C
File B File D
The EDG Architecture Tutorial - n° 13
File Management
Replica Catalog:
Replica Selection:
Map Logical to Site files
Get ‘best’ file
Pre- Post-processing:
Replication Automation:
Prepare
Site
A
files for transfer
Validate files after transfer
Data Source
subscription
Site B
Storage Element A
Storage Element B
File Transfer
File A File X
File B File Y
File A File C
File B File D
The EDG Architecture Tutorial - n° 14
File Management
Replica Catalog:
Replica Selection:
Map Logical to Site files
Get ‘best’ file
Pre- Post-processing:
Replication Automation:
Prepare
Site
A
files for transfer
Validate files after transfer
Data Source
subscription
Site B
Load balancing:
Replicate based on usage
Storage Element A
Storage Element B
File Transfer
File A File X
File B File Y
File A File C
File B File D
The EDG Architecture Tutorial - n° 15
Replica Manager:
‘atomic’ File
replication
operation
Management
single client interface
orchestrator
Replica Catalog:
Replica Selection:
Map Logical to Site files
Get ‘best’ file
Pre- Post-processing:
Replication Automation:
Prepare
Site
A
files for transfer
Validate files after transfer
Data Source
subscription
Site B
Load balancing:
Replicate based on usage
Storage Element A
Storage Element B
File Transfer
File A File X
File B File Y
File A File C
File B File D
The EDG Architecture Tutorial - n° 16
Replica Manager:
‘atomic’ File
replication
operation
Management
single client interface
orchestrator
Replica Catalog:
Replica Selection:
Map Logical to Site files
Get ‘best’ file
Pre- Post-processing:
Replication Automation:
Prepare
Site
A
files for transfer
Validate files after transfer
Data Source
subscription
Site B
Load balancing:
Metadata:
LFN
metadata
Storage
Element A
Transaction information
Access patterns
File A File X
File B File Y
Replicate based on usage
Storage Element B
File Transfer
File A File C
File B File D
The EDG Architecture Tutorial - n° 17
Replica Manager:
‘atomic’ File
replication
operation
Management
single client interface
orchestrator
Replica Catalog:
Replica Selection:
Map Logical to Site files
Get ‘best’ file
Pre- Post-processing:
Replication Automation:
Prepare
Site
A
files for transfer
Validate files after transfer
Data Source
subscription
Site B
Load balancing:
Metadata:
LFN
metadata
Storage
Element A
Transaction information
Access patterns
File A File X
File B File Y
Replicate based on usage
Storage Element B
File Transfer
File A File C
File B File D
The EDG Architecture Tutorial - n° 18
Current State
File Transfer: Use GridFTP – deployed
Close collaboration with Globus
NetLogger (Brian Tierney and John Bresnahan)
Replication: GDMP – deployed
Wrapper around Globus ReplicaCatalog
All functionality in one integrated package
Using Globus 2
Uses GridFTP for transferring file
Replication: edg-replica-manager – deployed
Replication: Replica Location Service Giggle – in testing
Distributed Replica Catalog
Replication: Replica Manager Reptor – in testing
Optimization: Replica Selection OptorSim – in simulation
Metadata Storage: SQL Database Service Spitfire – deployed
Servlets on HTTP(S) with XML (XSQL)
GSI enabled access + extensions
GSI interface to CASTOR – delivered
The EDG Architecture Tutorial - n° 19
WP2: Data
Management
Local Application
Grid Application Layer
Job
Managem.
Deployed Components
GridFTP
Grid
Scheduler
SQL
Database
Services
Replica
Manager
Info &
Monitor
Computing
Element
Services
Storage
Element
Services
Replica
Catalog
Authorization
Authentication
Accounting
Logging &
Bookkeeping
Fabric services
Resource
Managem.
Object to
File
Mapping
Underlying Grid Services
GDMP
Spitfire
Metadata
Managem.
Data
Managem.
Collective Services
Replica Manager - edg-replicamanager
Replica Catalog - globus-replicacatalog
Local Database
Config
Management
Monitoring
Fault
Tolerance
Node
Installation
Management
Fabric Storage
Management
Implementation:
RM: C++ classes (under
development)
RC : Globus Replica Catalog wrapper
GDMP : C++
Spitfire : Java, Web Services
WP2 main interfaces:
The GRID Storage Element
WP1 Resource Broker APIs
WP3 GRID Info services
WP7 network monitoring info providers
End User (using GDMP)
The EDG Architecture Tutorial - n° 20
Example of Data Management by LHCb
Copy data file to storage element:
globus-url-copy file:///${chemin}/L69999
gsiftp://lxshare0219.cern.ch/flatfiles/SE1/lhcb/L69999
Register stored data in the catalog:
/opt/globus/bin/globus-job-run lxshare0219.cern.ch /bin/bash -c "export
GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf;/opt/edg/bin/gdmp_register_local_file -d
/flatfiles/SE1/lhcb"
Publish catalog:
/opt/globus/bin/globus-job-run lxshare0219.cern.ch /bin/bash -c "export
GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf; /opt/edg/bin/gdmp_publish_catalogue n"
Copy output to MSS:
rfcp L1600061 /castor/cern.ch/lhcb/mc/L1600061
The EDG Architecture Tutorial - n° 21
The Replica Manager APIs
Client
Replica Catalogue
Replica Manager
Replica
Optimiser
CE
physical file transfer
SE
Replica Manager
communication
Replica
Optimiser
CE
SE
The EDG Architecture Tutorial - n° 22
The Replica Manager APIs
RM.copy(PhysicalFileName source,
PhysicalFileName destination,
String protocol):Status
allows for third-party transfer
transfer between:
two StorageElements or
ComputingElement and Storage Element
Space management policies under development
The EDG Architecture Tutorial - n° 23
The Replica Manager APIs
RM.add/deletePhysicalFileName(LogicalFileName lfn,
PhysicalFileName pfn)
Replica Catalogue operations only - no file transfer
RM.copyAndAddPhysicalFile(PhysicalFileName source,
PhysicalFileName destination,
LogicalFileName lfn,
String protocol):Status
third-party transfer but :
files can only be registered in Replica Catalogue if destination PFN
contains a valid SE (i.e. needs to be registered in the RC)!
RM.deletePhysicalFile(LogicalFileName lfn,
PhysicalFileName pfn)
The EDG Architecture Tutorial - n° 24
WP2 next generation Replication
Services
Reptor
Replica Manager
Optimization
Optor
Client
Transaction
Consistency
Postprocessing
File Transfer
Preprocessing
Replica Location
Subscription
GDMP
Giggle
Replica Metadata RepMeC
The EDG Architecture Tutorial - n° 25
Replication Services Architecture
User Interface
Replica
Replica
Replica
Location
Location
Location
Index
Index
Index
Replica
Metadata Catalog
Resource Broker
Site
Core API
Site
Replica Manager
Optimisation API
Optimiser
Processing API
Pre-/Postprocessing
Computing
Element
Local
Replica
Catalog
Replica Manager
Optimiser
Local
Replica
Catalog
Pre-/Postprocessing
Storage
Element
Computing
Element
Storage
Element
The EDG Architecture Tutorial - n° 26
Metadata Management and Security
Project Spitfire
'Simple' Grid Persistency
Grid Metadata
Application Metadata
Unified Grid enabled front end to relational databases.
Metadata Replication and Consistency
Publish information on the metadata service
Secure Grid Services
Grid authentication, authorization and access control mechanisms
enabled in Spitfire
Modular design, reusable by other Grid Services
The EDG Architecture Tutorial - n° 27
Spitfire Architecture
XSQL Servlet as one access mode
Atomic RDBMS is always consistent
for ‘simple’ web access
No local replication of data
Web/Grid Services Paradigm
Role-based authorization
SOAP interfaces
JDBC interface to RDBMS
Plugability and extensibility
SOAP
Connecting Layer
Global Spitfire
Layer
OracleLayer
DB2Layer
PGLayer
MyLayer
Oracle
DB2
PostGres
MySQL
Local Spitfire
Layer
The EDG Architecture Tutorial - n° 28
WP3 : GRID monitoring and Info
Providers
WP3’s task is to provide information about
The Grid itself
This includes information about resources (ComputingElements,
StorageElements and the Network), for which the Globus MDS is
a common solution; and job status information
(as implemented by WP1's Logging and Bookkeeping).
Grid applications
This is information published by user jobs. This is used for
performance monitoring.
The EDG Architecture Tutorial - n° 29
WP3 : GRID monitoring and Info
Providers
Main WP3 components:
MDS v 2.1: the Globus Monitoring and Discovery Services based on Soft
State Registration protocols and LDAP aggregate directory services
Ftree : EDG developed directory service based on OpenLDAP plus caching
to address shortcoming in MDS v1, optimizing data access performances
R-GMA: Relational GMA (Grid Monitoring Architecture [Consumers,
Producers and Directory Services, GGF] ) implementation which makes
information from producers available to consumers as relations (tables) .
It also uses relations to handle the registration of producers. R-GMA is
consistent with GMA principles.
GRM / PROVE: Application monitoring and visualization tools of the PGRADE graphical parallel programming environment, properly modified for
application monitoring in the DataGrid. The instrumentation library of GRM
is generalized for a flexible trace event specification. The components of
GRM will be connected to the R-GMA using its Producer and Consumer
APIs.
The EDG Architecture Tutorial - n° 30
R-GMA
Producer
subscribe
Registry
lookup
Consumer
Use the GMA from GGF
A relational
implementation
Applied to both
information and
monitoring
Creates impression that
you have one RDBMS per
VO
The EDG Architecture Tutorial - n° 31
Relational Approach
Producers announce:
publish:
SQL “CREATE TABLE”
SQL “INSERT”
Consumers
SQL “SELECT”
collect:
The EDG Architecture Tutorial - n° 32
R-GMA
Application Code
Consumer Servlet
Consumer
API
Registry
API
Registry Servlet
API – Servlet communication
http(s) in
Schema
API
XML back
Producer
API
Sensor Code
Registry
API
Schema Servlet
ProducerServlet
The EDG Architecture Tutorial - n° 33
Schema & Contributions
CPULoad (Global Schema)
Country
Site
Facility
Load
Timestamp
UK
RAL
CDF
0.3
19055711022002
UK
RAL
ATLAS
1.6
19055611022002
UK
GLA
CDF
0.4
19055811022002
UK
GLA
ALICE
0.5
19055611022002
CH
CERN
ALICE
0.9
19055611022002
CH
CERN
CDF
0.6
19055511022002
CPULoad (Producer 2)
CPULoad (Producer 1)
UK
RAL
CDF
0.3
19055711022002
UK
RAL
ATLAS
1.6
19055611022002
UK
GLA
CDF
0.4
19055811022002
UK
GLA
ALICE
0.5
19055611022002
CPULoad (Producer3)
CH
CERN
ATLAS
1.6
19055611022002
CH
CERN
CDF
0.6
19055511022002
The EDG Architecture Tutorial - n° 34
Contributions are Views
CPULoad (Producer 1)
UK
RAL
CDF
0.3
19055711022002
UK
RAL
ATLAS
1.6
19055611022002
SELECT * FROM
cpuLoad
WHERE country = ’UK’ AND site = ’RAL’
CPULoad (Producer 2)
UK
GLA
CDF
0.4
19055811022002
UK
GLA
ALICE
0.5
19055611022002
SELECT * FROM
cpuLoad
WHERE country = ’UK’ AND site = ’GLA’
The EDG Architecture Tutorial - n° 35
WP3: GRID
Monitoring
Local Application
Grid Application Layer
Job
Managem.
Components
SQL
Database
Services
Info &
Monitor
Computing
Element
Services
Storage
Element
Services
Replica
Catalog
Authorization
Authentication
Accounting
Logging &
Bookkeeping
Fabric services
Implementation:
Resource
Managem.
MDS : LDAP, Globus GRIS, GIIS
FTree : OpenLDAP, caching
Object to
File
Mapping
Underlying Grid Services
GRM/Prove
Metadata
Managem.
Replica
Manager
Grid
Scheduler
R-GMA
Data
Managem.
Collective Services
MDS / FTree
Local Database
Config
Management
Monitoring
Fault
Tolerance
Node
Installation
Management
Fabric Storage
Management
WP3 main interfaces:
RGMA : Java , C++, MySQL, TomCat
GRM / PROVE : P-GRADE
WP1 Resource Broker ( InfoIndex)
WP2 RM optimizer
all GRID services producing info
(SE,CE..)
WP7 network monitoring
The EDG Architecture Tutorial - n° 36
EDG middleware architecture:
WP4 : Fabric Management Components
WP4 is responsible to deliver a computing fabric comprised of all
the necessary tools to manage a center providing grid services on
clusters of thousands of nodes. The computing fabric is called the
Computing Element in EDG.
User Job Control and Management (Grid and local jobs) on
fabric batch and/or interactive CPU services
Gridification – Grid interface to fabric resources
Resource Management – manage underlying batch services
Automated System Administration for Computing Fabric
Elements. These subsystems are reserved for system
administrators and operators for performing system maintenance
Configuration Management
Installation Management
Fabric Monitoring
The EDG Architecture Tutorial - n° 37
WP4 Architecture logical overview
Resource
Broker
(WP1)
Grid User
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Data Mgmt
(WP2)
Resource
Management
Monitoring &
Fault Tolerance
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
Configuration
Management
(Mass storage,
Disk pools)
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 38
WP4 Architecture logical overview
- Interface between
Grid-wide services and
local fabric;
- Provides local Grid User
authentication,
authorization and
mapping of grid
credentials.
Resource
Broker
(WP1)
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Data Mgmt
(WP2)
Resource
Management
Monitoring &
Fault Tolerance
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
Configuration
Management
(Mass storage,
Disk pools)
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 39
WP4 Architecture logical overview
Resource
Broker
(WP1)
Grid User
- provides transparent
Data Mgmt
(WP2)
access (both job
and
admin) to different
cluster batch systems;
- enhanced capabilities
(extended schedulingLocal User
policies, advanced
reservation, local
accounting).
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Farm B (PBS)
Grid Data
Storage
(WP5)
Configuration
Management
(Mass storage,
Disk pools)
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 40
WP4 Architecture logical overview
Resource
Broker
(WP1)
Grid User
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Data Mgmt
(WP2)
Resource
Management
Monitoring &
Fault Tolerance
Local User
- provides the tools to
install
and
manage
all
Grid Data
software running
on the
Storage
fabric nodes;
Farm A (LSF)
Farm B (PBS)
Configuration
Management
(WP5)
-Agent to install, upgrade,
(Mass storage,
remove and configure
Disk pools)
software packages
on the
nodes.
-bootstrap services and
software repositories.
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 41
WP4 Architecture logical overview
Resource
Broker
(WP1)
Grid User
Data Mgmt
(WP2)
WP4 subsystems
Grid Info
Services
(WP3)
-provides a central storage
Fabric
and
management of all
Gridification
fabric configuration
information;
-Compile HLD templates to
LLD node profiles
Resource
- central DB and set of
Management
Local User
Other Wps
Monitoring &
Fault Tolerance
protocols and APIs to
store and retrieve
information.
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
Configuration
Management
(Mass storage,
Disk pools)
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 42
- provides the tools for
WP4 Architecture logical overview
gathering monitoring
information on fabric
nodes;
Resource
Broker
(WP1)
Grid User
Fabric
Gridification
Grid Info
Services
(WP3)
WP4 subsystems
-central measurement
repository stores all
monitoring Other
information;
Wps
- fault tolerance
correlation engines
detect failures and
trigger recovery actions.
Data Mgmt
(WP2)
Resource
Management
Monitoring &
Fault Tolerance
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
Configuration
Management
(Mass storage,
Disk pools)
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 43
User job management (Grid and local)
Resource
Broker
(WP1)
Grid User
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Data Mgmt
(WP2)
Resource
Management
Monitoring
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
(Mass storage,
Disk pools)
The EDG Architecture Tutorial - n° 44
User job management (Grid and local)
Resource
Broker
(WP1)
- Submit job
Grid User
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Data Mgmt
(WP2)
Resource
Management
Monitoring
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
(Mass storage,
Disk pools)
The EDG Architecture Tutorial - n° 45
User job management (Grid and local)
Resource
Broker
(WP1)
Grid User
Fabric
Gridification
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
- publish resource and
accounting information
Data Mgmt
(WP2)
Resource
Management
Monitoring
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
(Mass storage,
Disk pools)
The EDG Architecture Tutorial - n° 46
User job management (Grid and local)
Resource
Broker
(WP1)
Grid User
- Optimized selection of
site
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Data Mgmt
(WP2)
Resource
Management
Monitoring
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
(Mass storage,
Disk pools)
The EDG Architecture Tutorial - n° 47
User job management (Grid and local)
Resource
Broker
(WP1)
Grid User
- Authorize
- Map grid local
credentials Data Mgmt
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
(WP2)
Resource
Management
Monitoring
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
(Mass storage,
Disk pools)
The EDG Architecture Tutorial - n° 48
User job management (Grid and local)
Resource
Broker
(WP1)
Grid User
WP4 subsystems
Grid Info
Services
(WP3)
Other Wps
Fabric
Gridification
Data Mgmt
- Select an optimal
(WP2)batch
queue and submit
- Return job status and
output
Resource
Management
Monitoring
Local User
Farm A (LSF)
Farm B (PBS)
Grid Data
Storage
(WP5)
(Mass storage,
Disk pools)
The EDG Architecture Tutorial - n° 49
Automated management of large clusters
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 50
Automated management of large clusters
- Node malfunction detected
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 51
Automated management of large clusters
-Remove node from queue
WP4 subsystems
-Wait for running jobs(?)
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 52
Automated management of large clusters
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
- Update configuration
templates
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 53
Automated management of large clusters
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
- Trigger repair
The EDG Architecture Tutorial - n° 54
Automated management of large clusters
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
- Repair (e.g. restart,
reboot, reconfigure, …)
Configuration
Management
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 55
Automated management of large clusters
- Node OK detected
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 56
Automated management of large clusters
- Put back node in queue
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
The EDG Architecture Tutorial - n° 57
Automated management of large clusters
WP4 subsystems
Other Wps
Resource
Management
Farm A (LSF)
Monitoring &
Fault Tolerance
Information
Invocation
Farm B (PBS)
Configuration
Management
Installation &
Node Mgmt
Automation
The EDG Architecture Tutorial - n° 58
LCFG (Local ConFiGuration system)
Widely used fabric tool, whose purpose is to handle automated
installation and configuration in a very diverse and evolving
environment
Mechanism:
Abstract configuration parameters are stored in a central
repository located in the LCFG server.
Scripts on the host machine (LCFG client) read these configuration
parameters and either generate traditional configuration files, or
directly manipulate various services.
The EDG Architecture Tutorial - n° 59
WP4: Fabric
Management
Local Application
Local Database
Grid Application Layer
Job
Managem.
Data
Managem.
Metadata
Managem.
Components
Object to
File
Mapping
Collective Services
LCFG
Replica
Manager
Grid
Scheduler
Fabric Monitoring
PBS & LSF info providers
Info &
Monitor
Underlying Grid Services
SQL
Database
Services
Image installation
Computing
Element
Services
Storage
Element
Services
Replica
Catalog
Authorization
Authentication
Accounting
Logging &
Bookkeeping
Config. Cache Mgr
Fabric services
Resource
Managem.
Implementation:
LCFG : C++, XML, HTTP
Config
Management
Monitoring
Fault
Tolerance
Node
Installation
Management
Fabric Storage
Management
WP4 main interfaces:
WP1 Resource Broker ( InfoIndex)
WP2 Data management
WP5 Storage Element
WP3 GRID Info Services
The EDG Architecture Tutorial - n° 60
WP5 : Mass Storage Management
WP5 delivers the Grid interface to Storage.
Its service, the Storage Element (SE) is interfacing to
underlying Mass Storage Systems or simple storage services.
The EDG Architecture Tutorial - n° 61
The SE architecture
Clients
CLIs)
( RB,JSS, RM, GDMP, InfoServices(WP3),User Applic running on CEs,
Top
layer
Interface
1
Interface
2
Session Manager
Core
MetaData
Message Queue
System Log
Bottom
layer
Interface
3
House Keeping
MSS
Interface
MSS
Interface
MSS1
MSS2
Storage
Element
The EDG Architecture Tutorial - n° 62
SE Interactions
Replica
Manager/Catalog
1
2
6
Client
6
1.
2.
3.
4.
5.
6.
SE
3
4
5
Storage
The Client asks a catalog to provide the location of a file
The catalog responds with the name of an SE
The client asks the SE for the file
The SE asks the storage system to provide the file
The storage system sends the file to the client through the SE or
directly
The EDG Architecture Tutorial - n° 63
WP5:
Mass Storage
Management
Achievements
Local Application
Local Database
Grid Application Layer
Job
Managem.
Definition of Architecture and Design
for DataGrid storage Element
Data
Managem.
Metadata
Managem.
Object to
File
Mapping
Collective Services
Replica
Manager
Grid
Scheduler
Collaboration with Globus on
GridFTP/RFIO
Info &
Monitor
Underlying Grid Services
Collaboration with PPDG on control API
SQL
Database
Services
Staging from/to CASTOR at CERN
succesfully implemented and tested
Computing
Element
Services
Storage
Element
Services
Replica
Catalog
Authorization
Authentication
Accounting
Logging &
Bookkeeping
Fabric services
Succesfully Interfaced to GDMP
Resource
Managem.
Config
Management
Monitoring
Fault
Tolerance
Node
Installation
Management
Fabric Storage
Management
Supported Storage Systems:
UNIX disk systems
HPSS (High Performance Storage
System)
CASTOR (through RFIO)
GridFTP servers
DMF
Enstore
WP5 (SE) main interfaces:
WP1 Resource Broker & JSS
WP2 RM, RC
WP7 for GRIDftp monitoring
WP3 GRID Info Services
The EDG Architecture Tutorial - n° 64
WP6: TestBed Integration and
demonstrators
Local Application
WP6
goals: the EDG testbed
Integration of EDG sw releases
(currently 1.2) and deployment all
over the EDG testbed : the
integration team
Grid Application Layer
Job
Managem.
Data
Managem.
Metadata
Managem.
Object to
File
Mapping
Collective Services
Replica
Manager
Grid
Scheduler
Working implementation of multiple
VOs & basic security infrastructure
Underlying Grid Services
Definition of acceptable usage
contracts and creation of
Certification Authorities group
Fabric services
Set up of the Authorization
Working Group to manage
authorization policies on the
testbed
Local Database
SQL
Database
Services
Resource
Managem.
Computing
Element
Services
Storage
Element
Services
Config
Management
Info &
Monitor
Replica
Catalog
Monitoring
Fault
Tolerance
Authorization
Authenticatio
n Accounting
Node
Installation
Management
Logging &
Bookkeeping
Fabric Storage
Management
Components
Support for test-VO, mkgridmap tools
Globus packaging & EDG config
Build tools, CVS central s/w repository
End-user documents
The EDG Architecture Tutorial - n° 65
Further Information
DataGrid Dx.2 Deliverables: x=1..5
DataGrid D12.4 Deliverable
The EDG Architecture Tutorial - n° 66