Transcript Fabric Management for CERN Experiments Past, Present, and Future Tim Smith CERN/IT Contents  The Fabric of CERN today  The new challenges of LHC.

Fabric Management
for CERN Experiments
Past, Present, and Future
Tim Smith CERN/IT
Contents
 The Fabric of CERN today
 The new challenges of LHC computing
 What has this got to do with the GRID
 Fabric Management solutions of tomorrow?
 The DataGRID Project
2000/11/03
Tim Smith: HEPiX @ JLab
2
Fabric Elements
 Functionalities







 Infrastructure
Batch and Interactive
Disk servers
Tape Servers + devices
Stage servers
Home directory servers
Application servers
Backup service
2000/11/03







Job Scheduler
Authentication
Authorisation
Monitoring
Alarms
Console managers
Networks
Tim Smith: HEPiX @ JLab
3
Multiplicity Scale
Fabric Technology at CERN
PC Farms
10000
1000
PC Farms
RISC Workstations
100
10
1
Scalable Systems
SP2 CS2
RISC Workstations
SMPs
SGI,DEC,HP,SUN
Mainframes
IBM Cray
89 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05
Year
2000/11/03
Tim Smith: HEPiX @ JLab
4
Architecture Considerations
 Physics applications have ideal data
parallelism
 mass of independent problems
 No message passing
 throughput rather than performance
 resilience rather than ultimate reliability
 Can build hierarchies of mass market
components
 High Throughput Computing
2000/11/03
Tim Smith: HEPiX @ JLab
5
Component Architecture
High capacity
backbone
switch
Application
Server
100/1000baseT switch
CPU
CPU
CPU
CPU
CPU
Disk Server
1000baseT switch
Tape
Server
Tape
Server
2000/11/03
Tape
Server
Tape
Server
Tim Smith: HEPiX @ JLab
6
Analysis Chain: Farms
event filter
(selection &
reconstruction)
detector
processed
data
event
summary
data
raw
data
event
reconstruction
batch
physics
analysis
analysis objects
(extracted by physics topic)
event
simulation
2000/11/03
Tim Smith: HEPiX @ JLab
interactive
physics
analysis
7
Multiplication !
tomog
tapes
pcsf
nomad
na49
na48
na45
mta
lxbatch
lxplus
lhcb
l3c
ion
eff
cms
ccf
atlas
alice
1200
1000
#CPUs
800
600
400
200
0
Jul-97
2000/11/03
Jan-98
Jul-98
Jan-99
Jul-99
Tim Smith: HEPiX @ JLab
Jan-00
8
PC Farms
2000/11/03
Tim Smith: HEPiX @ JLab
9
Shared Facilities
EFF Scheduling 2000
140
120
DELPHI
CMS
ALEPH
80
ATLAS
NA45
60
COMPASS
ALICE
Available
40
20
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Number of PCs
100
Week Number
2000/11/03
Tim Smith: HEPiX @ JLab
10
LHC Computing Challenge
 The scale will be different
 CPU
 Disk
 Tape
10k SI95
30TB
600TB
1M SI95
3PB
9PB
 The model will be different
 There are compelling reasons why some of the
farms and some of the capacity will not be
located at CERN
2000/11/03
Tim Smith: HEPiX @ JLab
11
Estimated DISK Capacity ay CERN
1800
1600
Estimated disk
storage capacity at
CERN
1400
Bad News: Tapes
< factor 2 reduction in 8 years
Significant fraction of cost
Non-LHC
1000
800
LHC
600
400
200
0
1998
Moore’s Law
1999
2000
2001
2002
2003
2004
2005
2006
year
Estimated CPU Capacity at CERN
Bad News: IO
1996: 4G @10MB/s
1TB – 2500MB/s
2000: 50G @ 20 MB/s
1TB – 400 MB/s
Estimated CPU
capacity at CERN
2,500
2,000
K SI95
TeraBytes
1200
1,500
1,000
NonLHC
~10K SI95
1200 processors
LHC
500
0
1998
2000/11/03
1999
2000
Tim Smith: HEPiX @ JLab
2001
2002
year
2003
2004
2005
2006
12
Regional Centres:
a Multi-Tier Model
CERN – Tier 0
IN2P3
Tier 1
FNAL
RAL
Uni n
Lab a
Tier2
Uni b
Department

Lab c


Desktop
2000/11/03
MONARC http://cern.ch/MONARC
Tim Smith: HEPiX @ JLab
13
More realistically:
a Grid Topology
CERN – Tier 0
IN2P3
Tier 1
FNAL
RAL
Uni n
Lab a
Tier2
Uni b
Department

Lab c


Desktop
2000/11/03
DataGRID http://cern.ch/grid
Tim Smith: HEPiX @ JLab
14
Can we build LHC farms?
 Positive predictions
 CPU and disk price/performance trends suggest that the raw
processing and disk storage capacities will be affordable, and
 raw data rates and volumes look manageable
 perhaps not today for ALICE
 Space, power and cooling issues?
 So probably yes… but can we manage them?
 Understand costs - 1 PC is cheap, but managing 10000 is not!
 Building and managing coherent systems from such large
numbers of boxes will be a challenge.
2000/11/03
Tim Smith: HEPiX @ JLab
15
Management Tasks I
 Supporting adaptability
 Configuration Management
 Machine / Service hierarchy
 Automated registration / insertion / removal
 Dynamic reassignment
 Automatic Software Installation and
Management (OS and applications)
 Version management
 Application dependencies
 Controlled (re)deployment
2000/11/03
Tim Smith: HEPiX @ JLab
16
Management Tasks II
 Controlling Quality of Service
 System Monitoring
 Orientation to the service NOT the machine
 Uniform access to diverse fabric elements
 Integrated with configuration (change) management
 Problem Management
 Identification of root causes (faults + performance)
 Correlate network / system / application data
 Highly automated
 Adaptive - Integrated with configuration
management
2000/11/03
Tim Smith: HEPiX @ JLab
17
Relevance to the GRID ?
 Scalable solutions needed in absence of
GRID !
 For the GRID to work it must be presented
with information and opportunities
 Coordinated and efficiently run centres
 Presentable as a guaranteed quality resource
 ‘GRID’ification : the interfaces
2000/11/03
Tim Smith: HEPiX @ JLab
18
Mgmt Tasks: A GRID centre
 GRID enable
 Support external requests: services
 Publication
 Coordinated + ‘map’able
 Security: Authentication / Authorisation
 Policies: Allocation / Priorities / Estimation / Cost
 Scheduling
 Reservation
 Change Management
 Guarantees
 Resource availability / QoS
2000/11/03
Tim Smith: HEPiX @ JLab
19
Existing Solutions ?
 The world outside is moving fast !!
 Dissimilar problems
 Virtual super computers (~200 nodes)
 MPI, latency, interconnect topology and bandwith
 Roadrunner, LosLobos, Cplant, Beowulf
 Similar problems
 ISPs / ASPs (~200 nodes)
 Clustering: high availability / mission critical
 The DataGRID : Fabric Management WP4
2000/11/03
Tim Smith: HEPiX @ JLab
20
WP4 Partners







CERN (CH)
ZIB (D)
KIP (D)
NIKHEF (NL)
INFN (I)
RAL (UK)
IN2P3 (Fr)
2000/11/03
Tim Smith
Alexander Reinefeld
Volker Lindenstruth
Kors Bos
Michele Michelotto
Andrew Sansum
Denis Linglin
Tim Smith: HEPiX @ JLab
21
Concluding Remarks
 Years of experience in exploiting
inexpensive mass market components
 But we need to marry these with
inexpensive highly scalable
management tools
 Build components back together as a
resource for the GRID
2000/11/03
Tim Smith: HEPiX @ JLab
22