Transcript Объединенный институт ядерных исследов

Распределенная грид-инфраструктура
для обработки и анализа данных с
Большого адронного коллайдера
Кореньков В.В. (ЛИТ ОИЯИ)
Дубна, 07.10.2011
1
T.Strizh (LIT, JINR)
Some history






2001-2003 - EU DataGrid project
• middleware & testbed for an operational grid
2002-2005 – LHC Computing Grid – LCG
• deploying the results of DataGrid to provide a
production facility for LHC experiments
2004-2006 – EU EGEE project phase 1
• starts from the LCG grid
• shared production infrastructure
• expanding to other communities and sciences
2006-2008 – EU EGEE-II
• Building on phase 1
• Expanding applications and communities …
2008-2010 – EU EGEE-III
2010-2012 - EGI-InSPIRE
T.Strizh (LIT, JINR)
CERN
350 sites
55 countries
150,000 CPUs
150 PetaBytes
>15,000 users
>300 VOs
>1 mln jobs/day
Archeology
Astronomy
Astrophysics
Civil Protection
Comp. Chemistry
Earth Sciences
Finance
Fusion
Geophysics
High Energy Physics
Life Sciences
Multimedia
Material Sciences
…
T.Strizh (LIT, JINR)
Tier 0 at CERN: Acquisition,
First pass reconstruction,
Storage & Distribution
1.25 GB/sec
(ions)
T.Strizh
(LIT, JINR)
[email protected]
4
Tier 0 – Tier 1 – Tier 2
Tier-0 (CERN):
•Data recording
•Initial data
reconstruction
•Data distribution
Tier-1 (11 centres):
•Permanent storage
•Re-processing
•Analysis
Tier-2 (>200 centres):
• Simulation
• End-user analysis
T.Strizh
(LIT, JINR)
[email protected]
5
T.Strizh (LIT, JINR)
6
Russian Data Intensive Grid
infrastructure (RDIG)
The Russian consortium RDIG (Russian Data Intensive Grid), was set
up in September 2003 as a national federation in the EGEE project.
Now the RDIG infrastructure comprises 17 Resource Centers with
> 10000 kSI2K CPU and > 3500 TB of disc storage.
RDIG Resource Centres:
– ITEP
T.Strizh (LIT, JINR)
– JINR-LCG2 (Dubna)
– RRC-KI
– RU-Moscow-KIAM
– RU-Phys-SPbSU
– RU-Protvino-IHEP
– RU-SPbSU
– Ru-Troitsk-INR
– ru-IMPB-LCG2
– ru-Moscow-FIAN
– ru-Moscow-MEPHI
– ru-PNPI-LCG2 (Gatchina)
– ru-Moscow-SINP
- Kharkov-KIPT (UA)
- BY-NCPHEP (Minsk)
- UA-KNU
The main directions in development and
maintenance of RDIG e-infrastructure are as
the following:









- support of basic grid-services;
- Support of Regional Operations Center (ROC);
- Support of Resource Centers (RC) in Russia;
- RDIG Certification Authority;
- RDIG Monitoring and Accounting;
- participation in integration, testing, certification of gridsoftware;
- support of Users, Virtual Organization (VO) and application;
- User & Administrator training and education;
- Dissemination, outreach and Communication grid activities.
T.Strizh (LIT, JINR)
Worldwide LHC Computing Grid
Project (WLCG)
The protocol between CERN, Russia and JINR on
participation in LCG Project was approved in 2003.
MoU on Worldwide LHC Computing Grid (WLCG) signed
by Russia and JINR in October, 2007
The tasks of the Russia & JINR in the WLCG (2011 years):
•
•
•
•
•
Task 1. MW (gLite) testing (supervisor O. Keeble)
Task 2. LCG vs Experiments (supervisor I. Bird)
Task 3. LCG monitoring (supervisor J. Andreeva)
Task 4. Tier3 monitoring (supervisor J. Andreeva, A.Klementov)
Task 5/6. Genser/ MCDB ( supervisor W. Pokorski)
T.Strizh (LIT, JINR)
JINR Central Information and Computing Complex
(CICC)
Availability and Reliability = 99%
CICC comprises 1584 Core
Total performance ~4100 kSI2K
~ 300 CPU and ~320 TB disk
storage will be added during
the next few month
Disk storage capacity 1068 TB
Performance
Growth of the
JINR CICC
resources in
2003 - 2010
Performance/Capacity (kSI2K/TB)
3000
2500
Estimates of the JINR CICC resources
increase in the future
2000
1500
500
0
2003
2011
2012-2013
CPU (kSI2k)
4500
7000
Disk systems (TB)
1500
2500
Disk storage capacity
1000
2004
2005
2006
2007
2008
2009
2010
Integration with Google Earth
USER- INTERFACE AND VISUALIZATION
SERVICE DEVELOPMENT FOR VIRTUAL
ORGANIZATION SUPPORT IN HIGH ENERGY
PHYSICS
T.Strizh (LIT, JINR)
Tier 3 sites monitoring project






Tier-3 sites consist of resources mostly dedicated for the data analysis by the
geographically close or local scientific groups. Set of Tier 3 sites can be joined to
federation.
Many Institutes and National Communities built (or have plans to build) Tier-3
facilities. Tier-3 sites comprise a range of architectures and many do not possess
Grid middleware, which would render application of Grid monitoring systems
useless.
Joined effort of ATLAS, JINR and CERN IT (ES group)
Objectives for Tier3 monitoring
• Monitoring of Tier 3 site.
• Monitoring of Tier 3 sites federation.
Monitoring of Tier 3 site
• Detailed monitoring of the local fabric (overall cluster or clusters monitoring,
monitoring each individual node in the cluster, network utilization)
• Monitoring of the batch system.
• Monitoring of the mass storage system (total and available space, number of
connections, I/O performance)
• Monitoring of VO computing activities at a site
Monitoring of Tier 3 sites federation
• Monitoring of the VO usage of the Tier3 resources in terms of data transfer and
job processing and the quality of the provided service based on the job
processing and data transfer monitoring metrics.
T.Strizh (LIT, JINR)
Country Normalized CPU time per Country LHC VO
(July 2009 - October 2011)
Всего - 3,520,919,756 часов
Россия- 88,014,873 (2.5%)
13
T.Strizh (LIT, JINR)
Russia Normalized CPU time per SITE LHC VO
(September 2009 - September 2011)
Всего по РДИГ - 84,057,552 часов
ОИЯИ 34,244,089 часов (40.7%)
КИ 18,073,012 часов (21.5%)
ИФВЭ 10,610,480 часов (12.6%)
ПИЯФ 5,509,617 часов (6.6%)
ИТЭФ 5,150,221 часов (6.1%)
НИИЯФ МГУ - 4,522,040 часов (5.4%)
ИЯИ 2,093,048 часов 14(2.5)
T.Strizh (LIT, JINR)
Russia Normalised CPU time per LHC VO
(September 2009 – September 2011)
15
T.Strizh (LIT, JINR)
Russia Normalised CPU time by SITE and VO
(September 2009 – September 2011)
T.Strizh (LIT, JINR)
Member States of JINR Normalized CPU time per
countries (September 2009 - September 2011)
JINR member states
Normalised CPU time
Armenia
21
Belarus
60,225
Bulgaria
1,623,837
Czech Republic
21,948,110
Poland
43,682,563
Romania
13,391,632
Russia
JINR
84,057,552
34,244,089
Slovakia
3,428,272
Ukraine (Kharkov-KIPT-LCG2)
T.Strizh (LIT, JINR)
700+ 1,312,299
JINR Grid-infrastructure for training and
education – first step towards construction of
the JINR Member States grid-infrastructure
Consists of three grid sites at JINR
and one at each of the
following sites:

Institute of High-Energy
Physics - IHEP (Protvino),

Institute of Mathematics and
Information Technologies AS of
Republic of Uzbekistan - IMIT
(Tashkhent, Uzbekistan),

Sofia University "St. Kliment
Ohridski" - SU (Sofia, Bulgaria),

Bogolyubov Institute for
Theoretical Physics - BITP
(Kiev, Ukraine),

National Technical University of
Ukraine "Kyiv Polytechnic
Institute" - KPI (Kiev, Ukraine).
https://gridedu.jinr.ru
T.Strizh (LIT, JINR)
Letters of Intent with Moldova
“MD-GRID”, Mongolia “MongolGrid”, Kazakhstan
Project with Cairo University
WEB-PORTAL “GRID AT JINR” – “ГРИД В ОИЯИ”: http://grid.jinr.ru
A new informational resource
has been created at JINR:
web-portal “GRID AT JINR”.
The content includes the
detailed information on the JINR
grid-site and JINR’s participation
in grid projects:
•Grid Conception
•Grid-technologies
•Grid-projects
•RDIG Consortium
•JINR Grid-site
•Infrastructure and services
•Scheme
•Statistics
•VO and experiments support
•ATLAS
•CMS
•CBM and PANDA
•HONE
•How to start
•JINR in Grid-Projects
•WLCG
•GridNNN
•EGEE
•RFBR Projects
•INTAS Projects
•SKIF-Grid
•Grid Middleware testing
•Cooperation with JINR Member-States
•Monitoring&Accounting
•RDIG-monitoring
•dCache-monitoring
•Dashboard
•FTS-monitoring
•Н1 МС-monitoring
•Grid- Conferences of JINR
•GRID
•NEC
•Education
•Grid-infrastructure for education
•Courses & Lectures
•Text books
•Documentations
•Articles
•Materials for education
•News
T.Strizh (LIT, JINR)

















Frames for Grid cooperation of JINR
Worldwide LHC Computing Grid (WLCG);
Enabling Grids for E-sciencE (EGEE) - Now is EGI-InSPIRE
RDIG Development
Now is E-ARENA
CERN-RFBR project “Grid Monitoring from VO perspective”
BMBF grant “Development of the Grid-infrastructure and tools to provide joint investigations
performed with participation of JINR and German research centers”
“Development of Grid segment for the LHC experiments” was supported in frames of JINRSouth Africa cooperation agreement;
Development of Grid segment at Cairo University and its integration to the JINR GridEdu
infrastructure
NATO project "DREAMS-ASIA“ (Development of gRid EnAbling technology in
Medicine&Science for Central ASIA);
JINR - FZU AS Czech Republic Project “The GRID for the physics experiments”
NASU-RFBR project “Development and support of LIT JINR and NSC KIPT gridinfrastructures for distributed CMS data processing of the LHC operation”
JINR-Romania cooperation Hulubei-Meshcheryakov programme
JINR-Moldova cooperation (MD-GRID, RENAM)
JINR-Mongolia cooperation (Mongol-Grid)
JINR-Slovakia cooperation
JINR- Kazakhstan cooperation (ENU Gumelev)
Project "SKIF-GRID" (Program of Belarussian-Russian Union State).
21
Project GridNNN (National Nanotechnological Net)
T.Strizh (LIT, JINR)
Perspectives of JINR GRID Activities
-
Tier 3 sites monitoring project
- Common JINR-CERN Project “Global data transfer monitoring system
for WLCG infrastructure”
- development of unified Grid-environment of the JINR Member States
- Participation in project “WLCG Tier1 center in Russia”
Proposal to create the LCG Tier1 center in Russia (official letter by Minister of
Science and Education of Russia A. Fursenko has been sent
to CERN DG R. Heuer in March 2011).
The corresponding point to include in the agenda of next 5x5 meeting RussiaCERN.
- for all four experiments ALICE, ATLAS, CMS and LHCb
- ~10% of the summary Tier1 (without CERN) resources
- increase by 30% each year
- draft planning (proposal under discussion) to have prototype in the end
of 2011 - beginning 2012, and full resources in 2013 to meet the start of
next working LHC session.
Discussion about distributed Tier1 in Russia for LHC and FAIR
22
T.Strizh (LIT, JINR)
The Fourth International Conference "Distributed Computing and
Grid-technologies in Science and Education“ – GRID’2010





252 participants from 21 countries: Armenia,
Belarus, Bulgaria, Hungary, Germany, Greece,
Georgia, Iceland, Kazakhstan, Moldova,
Myanmar, Poland, Russia, Romania, USA,
Uzbekistan, Ukraine, France, Czechia,
Switzerland, Sweden as well as from CERN and
JINR.
56 universities and research centers of Russia.
8 sections: WLCG - worldwide Grid for
processing data from LHC at CERN, Gridapplications, Grid in business, distributed
computing and Grid-technologies in education,
GridННС – Grid of the national nanotechnology
network, methods and algorithms for
distributed computing, Grid-infrastructure and
"cloud" computing.
Round tables on using grid-technologies in
business and on training in grid-technologies
and their application in education.
36 plenary talks, 78 sectional talks.
T.Strizh (LIT, JINR)