Transcript Lezione 001 6 Aprile 2009 - INFN

Università degli Studi di Bari – Corso di Laurea Specialistica in Informatica
“Tecnologia dei Servizi “Grid e cloud computing”
A.A. 2009/2010
Giorgio Pietro Maggi [email protected], http://www.ba.infn.it/~maggi
Lezione 9b - 16 Dicembre 2009
Il materiale didattico usato in questo corso è stato mutuato da quello
utilizzato da Paolo Veronesi per il corso di Griglie Computazionali
per la Laurea Specialistica in Informatica tenuto nell’anno
accademico 2008/09 presso l’Università degli Studi di Ferrara.
Paolo Veronesi
[email protected], [email protected]
http://www.cnaf.infn.it/~pveronesi/unife/
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
0
Today’s focus: Information Services
Execution Management
• Job description & submission
• Scheduling
• Resource provisioning
Data Services
• Common access facilities
• Efficient & reliable transport
• Replication services
Resource Management
Self-Management
• Discovery
• Monitoring
• Control
• Self-configuration
• Self-optimization
• Self-healing
OGSA
Information Services
Security
• Registry
• Notification
• Logging/auditing
• Cross-organizational users
• Trust nobody
• Authorized access only
DONE
OGSA “profiles”
Web services foundation
DONE
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
1
Outline

What is the Information System

Data Model: the GLUE Schema
 Overview
 Core
entities

OpenLDAP server introduction LCG

Information Service Architecture

Top BDII and Site BDII

Information upgrade process
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
2
GLUE Schema Structure
Site
Collection of
resources owned by a
sinle organisation.
Contains info on the
location, the
administrator, web
page and so on
1
1
1
*
Service
Description of
deployed service
Cluster
* Set of heterogeneous
resources. Contains
info on shared
directory
1
Host
Contains details of
hardware (features
and performance) and
software
1
VOview
Sub-Cluster
Set of homogeneous
resources. Contains
the size of the set
*
Job
Info
*
State
Policy
*
*
StorageElement
ComputingElement
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
3
Information & Monitoring Services
Berkeley Database
Information Index
BDII
top-level
Queries
WMS
2 minutes
BDII
site-level
Site
WN
UI
FTS
BDII
resource
BDII
Resource
provider
provider
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
- Based on ldap
- Standardized information provider (GIP)
- GLUE-1.3 schema
- Top level Used with 230+ sites
- Roughly 60 instances in EGEE
4
BDII overview

Every node (except UI and WNs) has a bdii service
in order to publish its informations

A node in every site collects all site BDIIs and
publishes them using a site BDII;

The top BDII collects all site BDIIs

User can run a set of commands to query the top
BDII.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
5
Generic Information Provider

The Generic Information Provider, GIP, is a highly configurable information
provider that makes a separation between static and dynamic information. It can
be used to produce any kind of information for use with LDAP based grid
information systems.

An information provider in its simplest form is a script that prints an LDIF file to
standard out. The LDIF file should conform to the schema used in the GIP.

The problem is with dynamic information. Usually there are only a few attributes
that have to be found dynamically. The idea with the GIP is to use dynamic
plug-ins to obtain these values and use a common framework for everything
else.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
6
GIP configuration file

The GIP configuration file contains all the configuration parameters for the GIP.


The directory to use as the GIPs cache.


response
How long the GIP will wait for dynamic plugins before continuing.


cache_ttl
How long the cache is valid.


freshness
How long to use the cache before running the dynamic plugins again.


provider_dir
The directory where to put links to the information providers.


static_dir
The directory where to put links to the static ldif files.


plugin_dir
The directory where to put links to the dynamic plugins.


temp_dir
timeout
Timeout for the dynamic plugins in seconds.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
7
GIP
1.
The GIP script reads in all the ldif files from the static_dir ;
2.
run all the providers in the plugin_dir.
3.
It will then run any dynamic plug-ins found in the plugin_dir to obtain
the dynamic values.
4.
It will print the LDIF to standard out but use the any dynamic values
found from the plugins.
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
8
R-GMA: New System ?
Why a new protocol?
Disadvantages of the old system:
¤ LDAP does not allow to query information from different entries
¤ MDS/BDII is not flexible enough to allow for dynamic publication of data from
user applications
Advantages of the new system:
¤ R-GMA is quite flexible and allows cross queries between different entries
¤ Anyone can introduce new information in the system in a very easy way
¤ It is quite dynamic with new Producers of information being notified by existing
Consumers
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
9
R-GMA: Characteristics
GMA (Grid Monitoring Architecture)
• From GGF (Global Grid Forum)
• Very simple; it does not define:
→ Data model
→ Data transfer mechanism
→ Registry implementation
R-GMA (Relational GMA): Relational implementation
• Powerful data model and query language
• All data modeled as tables
• SQL as query language. It can express most queries in one
expression
• You have a Relational DB for each VO
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
10
R-GMA: Design
REGISTRY
Info
???
???
Info
Info
Collector of Information
Info
Collector of Information
M Box
Info
Collector of Information
M Box
M Box
SITE
Client
Data
Base
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
SITE
SITE
Data
Base
Data
Base
11
R-GMA Architecture
Store Location
Producer
Registry
Transfer
Data
Consumer
Producers: Register themselves with the Registry and
describe the type and structure of the information they
want to make available to Grid
Consumers: Query the Registry to find out the
information available and locate Producers which
provide such information. They can connect directly the
Producers
Registry: General collector, its arrow line represents
the main flow of data
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
12
R-GMA tools: Browser
The user can retrieve the
R-GMA information via
the browser servlet
http://lcgic01.grid
pp.rl.ac.uk:8080/RGMA/index.html
It shows the schema, the
registered producers and
allows simple queries
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
13
References

gLite doc http://glite.web.cern.ch/glite/documentation/default.asp

gLite userGuide https://edms.cern.ch/file/722398//gLite-3-UserGuide.pdf,
Section 5.

EGEE: The Information System
https://twiki.cern.ch/twiki/bin/view/EGEE/InformationSystemOverview


Generic Information Provider https://twiki.cern.ch/twiki//bin/view/EGEE/GIP
Usage of Glue Schema v1.3 for WLCG Installed Capacity information:
https://twiki.cern.ch/twiki/pub/LCG/WLCGCommonComputingReadinessChalleng
es/WLCG_GlueSchemaUsage-1.8.pdf
Tecnologia dei Servizi “Grid e cloud computing” - Lezione 9b
14