Transcript The rise of cloud computing

High Energy Physics – A big data use case
Bob Jones
Head of openlab
IT dept
CERN
Franco-British Workshop on Big Data in Science
London, 6-7 November 2012
This document produced by Members of the Helix Nebula consortium is licensed under a Creative Commons Attribution 3.0 Unported License.
Permissions beyond the scope of this license may be available at http://helix-nebula.eu/. The Helix Nebula project is co-funded by the European Community Seventh Framework
Programme (FP7/2007-2013) under Grant Agreement no 312301
Accelerating Science and Innovation
2
Data flow to permanent storage: 4-6 GB/sec
200-400 MB/sec
1-2 GB/sec
1.25 GB/sec
1-2 GB/sec
3
WLCG – what and why?
•
A distributed computing
infrastructure to provide the
production and analysis
environments for the LHC
experiments
•
Managed and operated by a
worldwide collaboration between
the experiments and the
participating computer centres
•
The resources are distributed – for
funding and sociological reasons
•
Our task was to make use of the
resources available to us – no
matter where they are located
•
Secure access via X509 certificates
issued by network of national
authorities - International Grid
Trust Federation (IGTF)
–
http://www.igtf.net/
Tier-0 (CERN):
• Data recording
• Initial data reconstruction
• Data distribution
Tier-2 (~130 centres):
• Simulation
• End-user analysis
Tier-1 (11 centres):
•Permanent storage
•Re-processing
•Analysis
4
WLCG: Data Taking
•
Castor service at Tier 0 well adapted to
the load:
– Heavy Ions: more than 6 GB/s to tape
(tests show that Castor can easily
support >12 GB/s); Actual limit now
is network from experiment to CC
– Major improvements in tape
efficiencies – tape writing at ~native
drive speeds. Fewer drives needed
– ALICE had x3 compression for raw
data in HI runs
HI: ALICE data into Castor > 4 GB/s (red)
HI: Overall rates to tape > 6 GB/s (r+b)
Accumulated Data 2012
0.1
0.2 0.8
0.0
0.9
1.2
1.3
0.8
ALICE
AMS
ATLAS
CMS
4.6
23 PB data written in 2011
16 PB in 2012 !
5.8
COMPASS
LHCB
NA48
De
c-
11
-1
1
t- 1
1
No
v
ALICE
Oc
ATLAS
p11
CMS
Se
1.5M jobs/day
Ju
l- 1
1
Au
g11
50000000
Ju
n11
Jan-12
Dec-11
Nov-11
Oct-11
Sep-11
Aug-11
Jul-11
Jun-11
May-11
Apr-11
Mar-11
Feb-11
Jan-11
Dec-10
Nov-10
Oct-10
Sep-10
Aug-10
Jul-10
Jun-10
May-10
Apr-10
Mar-10
Feb-10
Jan-10
Dec-09
60000000
Ap
r-1
1
M
ay
-1
1
10
Ja
n11
Fe
b11
M
ar
-1
1
De
c-
-1
0
LHCb
t- 1
0
800000000
No
v
900000000
Oc
p10
500000000
Se
600000000
Ju
n10
Ju
l- 1
0
Au
g10
700000000
Ap
r-1
0
M
ay
-1
0
Oct-09
Nov-09
LHCb
Sep-09
Aug-09
Jul-09
Jun-09
30000000
May-09
Apr-09
Mar-09
40000000
Ja
n10
Fe
b10
M
ar
-1
0
Jan-09
0
Feb-09
Overall use of WLCG
Usage continues to
grow even over end of
year technical stop
- # jobs/day
- CPU usage
20000000
10000000
1E+09
109 HEPSPEC-hours/month
(~150 k CPU continuous use)
CMS
ATLAS
ALICE
400000000
300000000
200000000
100000000
0
CPU – 11.2010-10.2011
TRIUMF
NDGF
CERN KIT PIC CCIN2P3
1%
1%
2%
4%
CNAF
3% 5%
4%
Sara/NIKHEF
4%
ASGC
RAL 1%
4%
Tier-2's
61%
BNL
5%
Significant use of Tier 2s for
analysis
FNAL
5%
Brazil
USA
32%
UK
16%
Ukraine
Greece
Australia
Republic of
Korea
Taipei
France
10%
Turkey
Estonia
Germany
9%
Hungary
Italy
6%
India
Finland
Spain
4%
Norway
Pakistan
Austria
Belgium
Portugal
Slovenia
Russian Federation
4%
Poland
3%
Switzerland
Japan
Canada
Czech
Republic
China
Israel
Romania
Sweden
Broader Impact of the LHC
Computing Grid
• WLCG has been leveraged on both sides of
the Atlantic, to benefit the wider scientific
community
– Europe (EC FP7):
• Enabling Grids for E-sciencE
(EGEE) 2004-2010
• European Grid Infrastructure
(EGI) 2010--
– USA (NSF):
• Open Science Grid (OSG)
2006-2012 (+ extension?)
• Many scientific applications 
Archeology
Astronomy
Astrophysics
Civil Protection
Comp. Chemistry
Earth Sciences
Finance
Fusion
Geophysics
High Energy
Physics
Life Sciences
Multimedia
Material Sciences
8
…
EGEE – What do we deliver?
Enabling Grids for E-sciencE
• Infrastructure operation
– Sites distributed across many countries
 Large quantity of CPUs and storage
 Continuous monitoring of grid services & automated site
configuration/management
 Support multiple Virtual Organisations from diverse
research disciplines
• Middleware
– Production quality middleware distributed under
business friendly open source licence
 Implements a service-oriented architecture that virtualises
resources
 Adheres to recommendations on web service inter-operability
and evolving towards emerging standards
• User Support - Managed process from first contact
through to production usage
–
–
–
–
Training
Expertise in grid-enabling applications
Online helpdesk
Networking events (User Forum, Conferences etc.)
EGEE-III INFSO-RI-222667
May 2008
9
Sample of Business Applications
Enabling Grids for E-sciencE
•
SMEs
– NICE (Italy) & GridWisetech (Poland): develop services on
open source middleware for deployment on customer inhouse IT infrastructure
– OpenPlast project – (France) Develop and deploy Grid
platform for plastics industry
– Imense Ltd (UK) - Ported gLite application and GridPP sites
•
Energy
– TOTAL, UK - Ported application using GILDA testbed
– CGGVeritas (France) – manages in-house IT infrastructures
and sells services to petrochemical industry
•
Automotive
• DataMat (Italy) – Provides grid services to automotive
industry
EGEE-III INFSO-RI-222667
May 2008
10
CERN openlab in a nutshell
• A science – industry partnership to drive R&D
and innovation with over a decade of success
• Evaluate state-of-the-art technologies in a
challenging environment and improve them
• Test in a research environment today what will
be used in many business sectors tomorrow
• Train next generation of engineers/employees
CONTRIBUTOR (2012)
• Disseminate results and outreach to new
audiences
Bob Jones – CERN openlab 2012
12
Virtuous Cycle
CERN
requirements
push the limit
Produce
advanced
products and
services
Apply new
techniques
and
technologies
Test
prototypes in
CERN
environment
Joint
development
in rapid
cycles
A public-private partnership between the research community and industry
Bob Jones – CERN openlab 2012
13
openlab III (2009-2011)
• Fellows: 4
• Summer Students: 6
• Publications: 37
• Presentations: 41
• Reference Activities: over 15
• Product enhancements: on 8 product lines
• Inter-partner collaborations: 2
CERN openlab Board of Sponsors 2012
ICE-DIP
Marie Curie proposal submitted in January 2012 to EC and accepted for funding
(total 1.25M€ from EC):
ICE-DIP, the Intel-CERN European Doctorate Industrial Program, is an EID scheme
hosted by CERN and Intel Labs Europe.
ICE-DIP will engage 5 Early Stage Researchers (ESRs).
Each ESR will be hired by CERN for 3 years and will spend 50% of their time at Intel.
Academic rigour and training quality is ensured by the associate partners, National
University of Ireland Maynooth and Dublin City University, where the ESRs will be
enrolled in doctorate programmes.
Research themes: usage of many-core processors for data acquisition, future
optical interconnect technologies, reconfigurable logic and data acquisition
networks.
Focus is the LHC experiments’ trigger and data acquisition systems
15
How to evolve WLCG?
A distributed computing infrastructure to provide the production and
analysis environments for the LHC experiments
• Collaboration - The resources are distributed and provided “in-kind”
• Service - Managed and operated by a worldwide collaboration
between the experiments and the participating computer centres
• Implementation - Today general grid technology with high-energy
physics specific higher-level services
Evolve the Implementation while preserving the collaboration & service
16
CERN-ATLAS flagship configuration
Monte Carlo jobs (lighter I/O)
10s MB in/out
~6-12 hours/job
Ran ~40,000 CPU days
Difficulties overcome
Different vision of clouds
Different APIs
Networking aspects
Ramón Medrano Llamas,Fernando Barreiro, Dan van der Ster (CERN IT), Rodney Walker (LMU Munich)
Conclusions
• The Physics community took the concept of a grid and turned into a global
production quality service aggregating massive resources to meet the needs of the
LHC collaborations
• The results of this development serve a wide range of research communities;
have helped industry understand how it can use distributed computing;
have launched a number of start-up companies and
provided the IT service industry with new tools to support their customers
• Open source licenses encourage the uptake of the technology by other research
communities and industry while ensuring the research community contribution is
acknowledged
• Providing access to computing infrastructures by industry and research
communities for prototyping purposes reduces the investment and risk for the
adoption of new technologies
October 2012 - The LHC Computing Grid Bob Jones
Conclusions
• Many research communities and business sectors are now facing an unprecedented
data deluge. The Physics community with its LHC programme has unique
experience in handling data at this scale
• The on-going work to evolve the LHC computing infrastructure to make use of cloud
computing technology can serve as an excellent test ground for the adoption of
cloud computing in many research communities, business sectors and government
agencies
• The Helix Nebula initiative is driving the physics community exploration of how
commercial cloud services can serve the research infrastructures of the future and
provide new markets for European industry
October 2012 - The LHC Computing Grid - Bob Jones