Transcript fundamental science&technology

LHC-Computing Grid
LCG
LCG
„Eventually, users will be unaware they are using any computer
but the one on their desk, because it will have the capabilities
to reach out across the (inter-) national network and obtain
whatever computational resources are necessary”
(Larry Smarr and Charles Catlett, 1992)
Hans F Hoffmann, CERN
November 2002
SPC
http://cern.ch/Hans.Hoffmann/DESY-Nov02hfh.ppt
High Energy Physics is leading the way in data
intensive science
4 large detectors for the Large Hadron Collider (LHC)
CMS
Storage –
ATLAS
Raw recording rate 0.1 – 1 GByte/sec
Accumulating data at 10-14 PetaBytes/year
~ 20 million CDs each year
LHCb
10 PetaBytes of disk
Processing –
150,000 of today’s fastest PCs
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
2
Grids
Mobile Access
Supercomputer, PC-Cluster
G
R
I
D
Desktop
M
I
D
D
L
E
W
A
R
E
Data-storage, Sensors, Experiments
Internet, networks
Visualising
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
3
LCG
a GridComputing
virtual LHC
Grid TheBuilding
Centre
Collaborating
Computer
Centres
Alice VO
CMS or Atlas or LHCb VOs
[email protected]
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
5
LCG
Virtual Computing Centre
The User --sees the image of a single cluster
does not need to know - where the data is
- where the processing capacity is
- how things are interconnected
- the details of the different hardware
and is not concerned by the conflicting policies of the
equipment owners and managers
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
6
LCG
What is the LHC Computing Grid Project?
Mission of Phase 1 –
prepare the computing environment for the analysis of LHC data
applications software - environment, tools, frameworks, common
developments
deploy and coordinate a global grid service
encompassing the LHC Regional Centres
coordination of significant efforts in middleware support,
with emphasis on robustness, monitoring, error recovery
(resilience)
strategy and policy for resource allocation
authentication, authorisation, accounting, security
monitoring, modelling & simulation of Grid operations
tools for optimising distributed systems
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
7
LCG
Summary of Computing Capacity Required for
all LHC Experiments in 2008
-------------- CERN -------------Tier 0
Tier 1
Total
Processing (K SI2000)
Disk (PetaBytes)
Magnetic tape (PetaBytes)
12,000
1.1
12.3
8,000
1.0
1.2
20,000
2.1
13.5
Other
Tier 1
Total
Tier 1
CERN as
% of Tier 1
Total
Tier 0 + 1
CERN as
% of total
Tier 0 + 1
49,000
8.7
20.3
57,000
9.7
21.6
14%
10%
6%
69,000
10.8
33.9
29%
20%
40%
CERN will provide the data reconstruction & recording service (Tier 0)
-- but only a small part of the analysis capacity
current planning for capacity at CERN + principal Regional Centres
2002: 650 KSI2000  <1% of capacity required in 2008
2005: 6,600 KSI2000  < 10% of 2008 capacity
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
8
LCG
Centres taking part in the LCG-1
around the world  around the
clock
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
9
Project structure
LCG
POB
SC2
requirements
Architects
Forum
Design decisions,
implementation strategy
for physics applications
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
PEB
Grid Deployment
Board
Coordination, standards,
management policies
for operating the
LCG Grid Service
10
LCG
LCG: SC2 (+ PEB) Roles
SC2 – Software&Computing Committee
PEB – Project Execution Board
SC2 brings together the four experiments and Tier 1 Regional Centers
It identifies common solutions and sets requirements for the project
may use RTAGs – Requirements and Technical Assessment Groups
limited scope, two-month lifetime with intermediate report
one member per experiment + experts
PEB manages the implementation
organizing projects, work packages
coordinating between the Regional Centers
collaborating with Grid projects
organizing grid services
SC2 approves the work plan, monitors progress
[email protected]
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
11
LCG
SC2 Monitors Progress of the Project
Requirements for several key work packages were successfully defined
PEB has turned these into work plans:
Data Persistency, Software Support, Mass Storage
Other work plans are in preparation:
Grid use cases, Mathematical Libraries
Key requirements are finishing in October:
Detector Simulation
‘Blueprint for LHC experiments software architecture’
This will trigger important further activity in application developments
[email protected]
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
12
LCG
Project Execution Board
Two bodies set up to coordinate & take decisions:
Architects Forum
software architect from each experiment, the application area manager
makes common design decisions and agreements between experiments in
the applications area
supported by a weekly applications area meeting open to all participants
Grid Deployment Board
representatives from the experiments and from each country with an
active Regional Centre taking part in the LCG Grid Service
prepares the agreements, takes the decisions, defines the standards and
policies that are needed to set up and manage the LCG Global Grid Service
[email protected]
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
13
250
LCG Recruitment status - snapshot
LCG
LCG Phase 1 Agreed Personnel Profile
Black Line: Total Requested
Blue Line: IT Complement
200
150
IT Computing for LHC experiments
FTE * Weight
EU
USA
CERNMat
Sweden
100
Israel
IT Infrastructure physics
Hungary
Portugal
Switzerland
Spain
50
France
IT Infrastructure non-physics
Germany
Italy
UK
IT Engineering and Controls
0
2001
25-Nov-2002
2002
2003
[email protected]
Years
2004
DESY-Nov-02; H F Hoffmann/CERN
2005
14
LCG Materials at CERN
LCG
60
External Income + MTP
Medium Term Plan
50
Staff for Tier0/1 20 FTE
Short term staff for Phase 2
40
MCHF
Tier 0 +1 investment
30
Outsourced admin/operation
Prototype Tier 0 +1
20
Physics WAN
CC preparation
Production Computing (LHC Experiments)
Infrastructure (LHC experiments)
10
Production Com puting (LEP + Fixed Target)
Infrastructure (non-physics)
Engineering and Control
Systems
0
2002
25-Nov-2002
2003
2004
2005
2006
2007
DESY-Nov-02; H F Hoffmann/CERN
2008
2009
2010
15
LCG Level 1 Milestones
LCG
Hybrid Event Store available for general users
applications
Distributed production using grid services
Distributed end-user interactive analysis
Full Persistency Framework
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
2002
2003
2004
grid
2005
LHC Global Grid TDR
“50% prototype” (LCG-3) available
LCG-1 reliability and performance targets
[email protected]
25-Nov-2002
First Global Grid Service (LCG-1) available
DESY-Nov-02; H F Hoffmann/CERN
16
ATLAS DC1 Phase 1 : July-August 2002
LCG
3200 CPU‘s
110 kSI95
71000 CPU days
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
25-Nov-2002
1,41%
39 Institutes in 18 Countries
10,92%
grid tools used at 11 sites
0,02%
Australia
14,33%
Austria
Canada
CERN
Czech Republic 3,99%
France
1,89%
Germany
4,33%
Israel
Italy
3,15%
Japan
2,22%
Nordic
Russia
Spain
10,72%
Taiwan
UK
4,94%
USA
2,36%
1
2
3
Contribution to the overall
CPU-time (%) per country
4
5
6
7
28,66%
0,01%
9,59%
8
9
10
1,46%
11
DESY-Nov-02; H F Hoffmann/CERN
12
5*10*7 events generated
1*10*7 events simulated
3*10*7 single particles
30 Tbytes
35 000 files
13
14
15
16
17
LCG
ATLAS DC2 : October 2003 - March 2004
Use Geant4
Perform large scale physics analysis
Use LCG common software
Use widely Grid middleware
Further test of the computing model
~ same amount of data as for DC1
ATLAS DC3 : End 2004 - Begin 2005
5 times more data than for DC2
ATLAS DC4 : End 2005 - Begin 2006
2 times more data than for DC3
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
18
Spring02: CPU Resources
Most Resources not at CERN
(CERN not even biggest Single Resource)
UFL 5%
Wisconsin
18%
UCSD 3%
Bristol 3%
RAL 6%
Caltech 4%
Moscow
10%
FNAL 8%
HIP 1%
INFN 18%
CERN 15%
LCG
25-Nov-2002
IN2P3 10%
IC 6%
DESY-Nov-02; H F Hoffmann/CERN
6 million events
~20 sites
19
LCG
State of play
We are still solving basic reliability & functionality problems
This is worrying as we still have a long way to go to get to a solid
service
At end 2002, a solid service in mid-2003 looks (surprisingly)
ambitious
HEP needs to limit divergence in developments.
Complexity adds cost
We have not yet addressed system level issues
How to manage and maintain the Grid as a system providing a
high-quality reliable service.
Few tools and treatment in current developments of problem
determination, error recovery, fault tolerance etc.
Some of the advanced functionality we will need is only being thought
about now
Comprehensive data management, SLA’s, reservation schemes,
interactive use.
Many many initiatives are underway and more coming.
How do we manage the complexity of all this ?
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
20
LCG
Establishing Priorities
We need to create a basic infrastructure that
works well.
LHC needs a systems architecture and high-quality
middleware – reliable and fault tolerant.
Tools for systems administration.
Focus on mainline physics requirements and robust
data handling.
Simple end-user tools that deal with the
complexity.
Need to look at the overall picture of what we
are trying to do and
focus resources on key priority developments
We must simplify and make the simple things work well.
It is easy to expand scope, much harder to contract it !
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
21
LCG
Summary on LCG
basic middleware partnership
science, computer scientists and software engineers
industry and scientific research
international collaboration
global grid infrastructure for science
built on a foundation of core nodes serving existing national and
global collaborations
advanced middleware research programme
complementary projects
with a focus on the emerging requirements of the LHC data
analysis community
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
22
What else is going on
The International Context with the examples
Europe and US
(Not mentioned here: UK: e-science, DE: FZK,
Nordu-Grid, . . . )
HEP related Grid projects
Through links between sister projects, there is the
potential for a truely global scientific applications grid
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
24
www.cern.ch/grid
Project start 01-01-01, duration 3 years, 21 partners
Deliverables: Middleware tried out with Particle Physics-, Earth
Observation-, Biomedical Applications
1st review successfully passed,production quality middleware asked by Expts
Applications - Distributed Fabrics - "Transparent" Grid Middleware
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
25
The benefits of building Grids:
an example from Astronomy
Crab Nebula viewed
At four different
wavelengths: X-ray,
optical, infrared,
radio.
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
27
The benefits of building Grids:
an example from Earth observation
From global weather information to
precision agriculture and emergency
response
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
28
Managing Large Scale Data
Hierarchical Storage and
Indexing
Highly Distributed Source
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
29
Berkeley workshop (12-13 Nov 2002)
EU perspectives on Research on Grids
Kyriakos Baxevanidis
European Commission, DG INFSO
[email protected]
EU level effort complements National efforts
EU
(ISTProgramme: the
“flagship”)
National
•Foster cohesion,
interoperability,
cross-fertilization of
knowledge,
economies of scale,
critical mass
•Increase value
•Multiply impact
Overview of EU level RTD-policy on Grids
• Committed to develop, deploy Grids
• Strong support to synergetic and
integrated approaches
• Strong support to international cooperation (EU with the other regions)
Major Infrastructure deployments (on-going)
Examples:
1. DATAGRID, CROSSGRID, DATATAG
•Infrastructure across
17 European States
•Cross-Atlantic link
Application
requirements:
•Computing > 20 TFlops/s
•Downloads > 0.5PBytes
(2.5
Gbit/s)
2. EUROGRID, GRIP
•Network speeds at
10Gbit/s
•Infrastructure across
6 European States
•Collaborations of more
than 2000 scientists
•Industrial design, simulations
• Links to National Grid
(as well as scientific applications)
infrastructures
•Globus - Unicore interface
Application requirements:
•Real-time, Resource brokerage,
Portals, Coupled applications
Grids in FP6: some important priorities
• From current prototype to production level systems
(industrial quality); promote commercial uptake
• Research on new concepts (Semantic Grids, Grids and the
Web, Mobile Grids, Ambient Intelligence Spaces)
• Strengthen middle-ware developments in Europe
(Academia-Industry collaboration, skills)
• Involve new User/Application communities
• Funding targeted more to big scale efforts (tens of millions
of EURO) - mobilize National, private funds
• Close ties with National efforts (National Grid Centres?)
• Strengthen International cooperation - standards
Synergetic work: at the core of the activity
GRIA
GRIP
EUROGRI
D
DAMIEN
AVO
G
R
I
D
S
T
A
R
T
EGSO
GRIDLAB
CROSSGRI
D
DATAGRID
DATATAG
Synergy with new Grid projects, EU National and
International efforts, GGF
EGEE :EU 6th Framework Programme
(Enabling Grids for E-science and industry in Europe)
EU and EU member states major
investment in Grid Technology
Several good prototype results
Next Step:
Leverage current and planned national
programmes
work closely with relevant industrial
Grid developers and NRNs
build on existing middeware and
expertise
create a general European Grid
production quality infrastructure
This can be achieved for a minimum of
€100m/4 years on top of the national
and regional initiatives
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
applications
EGEE
network
36
Blue Ribbon Panel on
Cyberinfrastructure
Presentation to MAGIC
Paul Messina
November 6, 2002
Cyberinfrastructure: the Middle Layer
Applications in science and
engineering research and
education
Cyberinfrastructure: hardware,
software, personnel, services,
institutions
Base-technology: computation,
storage, communication
Some roles of cyberinfrastructure
• Processing, storage, connectivity
– Performance, sharing, integration, etc
• Make it easy to develop and deploy new
applications
– Tools, services, application commonality
• Interoperability and extensibility enables future
collaboration across disciplines
• Best practices, models, expertise
• Greatest need is software and experienced
people
From Prime Minister Tony Blair’s Speech to the
Royal Society (23 May 2002)
•
What is particularly impressive is the way that scientists are now undaunted by important
complex phenomena. Pulling together the massive power available from
modern computers, the engineering capability to design and build
enormously complex automated instruments to collect new data, with
the weight of scientific understanding developed over the centuries,
the frontiers of science have moved into a detailed understanding of
complex phenomena ranging from the genome to our global climate.
Predictive climate modelling covers the period to the end of this century and beyond, with
our own Hadley Centre playing the leading role internationally.
•
The emerging field of e-science should transform this kind of work.
It's significant that the UK is the first country to develop a national escience Grid, which intends to make access to computing power,
scientific data repositories and experimental facilities as easy as the
Web makes access to information.
•
One of the pilot e-science projects is to develop a digital mammographic archive, together
with an intelligent medical decision support system for breast cancer diagnosis and
treatment. An individual hospital will not have supercomputing facilties, but through the
Futures: The Computing Continuum
Smart
Objects
Petabyte
Archives
National
Petascale
Systems
Terabit
Collaboratories
Networks
Responsive
Environments
Laboratory
Terascale
Systems
Building Up
Ubiquitous
Sensor/actuator
Networks
Contextual
Awareness
Ubiquitous Infosphere
Building Out
Science, Policy
and Education
Key Points About the Proposed Initiative
• There is grass roots vision and demand from broad
S&E research communities. Many needs will not be
met by commercial world.
• Scope is broad, systemic, strategic. A lot more than
supercomputing. Extreme science - not flops.
Potential to relax constraints of distance, time, and
disciplinary boundaries. New methods: computation,
visualization, collaboration, intelligent instruments,
data mining, etc.
• Opportunity to leverage significantly prior NSF and
other government investments. Potential large
opportunity cost for not acting soon.
• The initiative is intrinsically international: cooperation
and competition. Can’t assume US is in the lead.
Components of CI-enabled
science & engineering
A broad, systemic, strategic conceptualization
High-performance computing
for modeling, simulation, data
processing/mining
Humans
Individual &
Group Interfaces
& Visualization
Collaboration
Services
Instruments for
observation and
characterization.
Global
Connectivity
Physical World
Facilities for activation,
manipulation and
construction
Knowledge management
institutions for collection building
and curation of data, information,
literature, digital objects
Coordination (synergy) Matrix
Applications of information technology to science
and engineering research
Cyberinfrastructure in support of applications
Core technologies incorporated into
cyberinfrastructure
Research in
technologies,
systems, and
applications
Development
or acquisition
Operations in
support of end
users
Bottom-line Recommendations
• NSF leadership for the Nation of an INITIATIVE to
revolutionize science and engineering research
capitalizing on new computing and
communications opportunities.
– 21st Century Cyberinfrastructure includes
supercomputing massive storage, networking,
software, collaboration, visualization, and
human resources
– Current centers (NCSA, SDSC, PSC) and other
programs are a key resource for the
INITIATIVE.
– Budget estimate: incremental $1020 M/year
(continuing).
Basic Middleware Partnership
LCG
Design
Production
Development
25-Nov-2002
System Design
Group
Project management
Software Process
Standards
Middleware
I
Middleware
II
DESY-Nov-02; H F Hoffmann/CERN
Modelling
Integration,
test,
certification
End User
Tools
Product
Delivery
System
Tools
47
LCG
Basic Middleware Partnership
• industrial management, development
process –
s/w engineering focus
• few development centres –
clear division of responsibilities
System Design
Group
Design
Modelling
• including key technology owners
• close co-operation between US, European,
Asian(?) teams
Production
- design and implementation
• strong preference for a single project –
- one management
- one review process
- multiple funding sources
Development
Project management
Software Process
Standards
Middleware I
Middleware II
Integration,
test,
certification
End User
Tools
Product
Delivery
System Tools
• an international partnership of –
- computer science and software
engineering
- science and industry
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
48
LCG
The LHC Grid – the foundation of a Global
Science Grid
LHC has a real need
and a reasonable scale
and has mature global collaborations of scientists
establish the basic middleware for a global science grid
deploy a core grid infrastructure in Europe, America, Asia
learn how to provide a “production quality” service during LHC
preparation (data challenges)
exploit the LHC core infrastructure as the foundation for a
general science grid
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
49
Development of a Global Grid
Infrastructure for Science
infrastructure
science research
LCG
middleware
research
lhc applications
bio applications
other science
applications
computer science
science
science
science
…..
providing
solutions for
exploitation of
grids by scientists
Advanced
Middleware
requirements defined
by current projects
Hardening/Reworking
of basic
middleware
prototyped by
current projects
application
adaptation
application
adaptation
application
adaptation
application-specific
middleware
application-specific
middleware
application-specific
middleware
core grid infrastructure
•
•
•
•
•
few core centres (including the LCG Tier 1s)
operate the information service, catalogues, ..
coordination, operations centre, ..
call centre, user support, training, ..
converge with other core infrastructures (DTF,
UK e-science grid, ….)
other grid nodes for
physics,
biology, medicine, ….
middleware
engineering
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
50
LCG
Longer-term Requirements
emphasis so far has been on the basic infrastructure
- as a solid base for constructing the LHC analysis environment
building on this foundation we need a programme of
collaborative/complementary research projects on advanced
middleware
advanced collaborative environments (dynamic workspaces for
scientific analysis communities)
resource optimisation – computation, storage, network
data placement, clustering, migration strategies
grid enabled object management
autonomic management and operation of grid resources
target 2009 –
a computing environment for LHC analysis teams,
efficiently and effortlessly exploiting global scientific
computing resources,
the scientist is not aware that she is using a grid
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
51
CERN users to be connected by a computer Grid
637
70
4306
22
538
27
87
55
10
Europe:
267 institutes, 4603 users
Elsewhere: 208 institutes, 1632 users
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
52
Prototyping the Global Information Society
“Bright” World to offer easy, affordable participation to e-sciences,
publications, results, e-education to all interested countries
25-Nov-2002
DESY-Nov-02; H F Hoffmann/CERN
53