Transcript Presentazione di PowerPoint

European AFS Workshop 2009
September 28th-30th 2009
Department of Computer Science and Automation/University Roma Tre
State-of-the-art Storage Solutions
...and more than that.
Fabrizio Magugliani
EMEA HPC Business Development and Sales
[email protected]
1
What does E4 Computer Engineering stand for ?
E4 = Engineering
4 (for)
Computing
E4 builds the solutions that accomplish the users’ requirements
2
Products and Services
Workstation
Server
(fluid-dynamics, video editing … )
(firewall, computing node, scientific apps …)
Storage
(from small DB up to big data requirements)
SAN – Storage Area Network
HPC Cluster , GPU Cluster, Interconnect
System config and optimization
Wide – Reliable – Advanced
3
Technology Partners
4
Customer References
5
www.e4company.com
[email protected]
Choosing the right computing node
Architecture Non Uniform Memory
Access
Architecture Uniform Memory
Access
(AMD)
(INTEL)
•Form factor: [1U,7U]
•Form factor: [1U,7U]
•Socket: [1,2,4,8]
•Socket: [1,2,4]
•Core: 4,6
•Core: 4,6
•Memory size
•Memory size
•Accelerators (GPUs)
•Accelerators (GPUs)
•Form factor:
•Workstation (graphic)
•Server rack-mount
•Blade
6
Choosing the right accelerator
7
Choosing and connecting the right accelerator
8
Choosing the right accelerator: Tesla S1070 Architecture
4GB
GDDR3
DRAM
4GB
GDDR3
DRAM
Tesla
GPU
Tesla
GPU
Power
Supply
Thermal
Management
PCIe x16
Gen2
System
Monitoring
Tesla
GPU
Tesla
GPU
4GB
GDDR3
DRAM
4GB
GDDR3
DRAM
Multiplexes
PCIe bus
between 2
GPUs
PCIe
x16
Gen2
Switch
PCIe
x16
Gen2
Switch
PCIe x16
Gen2
PCI Express
Cables to
Host
System(s)
Each 2 GPU
sub-system
can be
connected to
a different
host
Choosing the right accelerator:
performance
S M L XL
800.0
700.0
600.0
GFLOPS
500.0
400.0
300.0
200.0
100.0
0.0
1 GPU
2 GPU
800 GFLOPS on 16 GPUs
~ 99% Scaling
10
© NVIDIA Corporation 2008
4 GPU
8 GPU
16 GPU
Choosing the right interconnection technologies
• Gigabit Ethernet entry level on every solution. Ideal solution for codes with
low interprocess communication requirements
• InfiniBand DDR 20 + 20 Gb/s, integrable on motherboard (first cluster
InfiniBand 2005, Caspur)
• 10 Gb/s Ethernet
• Quadrics
• Myrinet
1 GbE
10 GbE
10 GbE RDMA
(Chelsio)
IB DDR
(InfiniHost)
IB QDR
(ConnectX)
Latency (microsecondi)
50
50
10
2,5
1,2
Bandwith (MB/s)
Bisectional Bandwith
(MB/S)
112
350
875
1500
3000
175
500
2900
5900
11
Interconnect
• Gigabit Ethernet: Ideal solution for applications requiring moderate bandwidth
among processes
•Infiniband DDR 20 + 20 Gb/s motherboard-based. Infinipath on HTX slot, tested
with latencies less than 2 microseconds.
• Myrinet, Quadrics
Choosing the right Storage
Performance
HPC storage
350-600 MB/s
per each
chassis
ETH interface
HPC storage
6 GB/s – FC,
IB interface
Storage space
of PB DataDirect
SAN - FC
Up to 1 GB/s
Disk Server-EHT
300-800 MB/s
HD section
200-400 MB/s
Storage type
13
Storage
Interface
• Disk subassembly
Ctrl RAID PCI-Ex
Performance
200MB/s
•Disk Server SATA /SAS
300 – 800 MB/s
ETH
•Storage SAS / SAS
FC
ETH
Up to 1 GB/s
•Ideal for HPC applications, ethernet i/f
500 MB/s per chassis
ETH
• Ideal for HPC applications, FC/IB i/f
InfiniBand
FC
Up to 3 GB/s
Storage Server
• high flexibility, low power consumption
solution engineered by E4 for high bandwidth
requirements.
•COTS-based (2 CPU INTEL Nehalem)
•RAM can be configured according to the
users’ requirements (up to 144GB DDR3)
•Controller SAS/SATA multi lane
•48 TB in 4U
• 1GbE (n via trunking), 10GbE, Infiniband
DDR/QDR
•374 units installed at CERN (Geneva), 70 in
several customers
HPC Storage Systems
Data Direct Network
• Interface: FC / IB
•Performance: up to
6GB/s
•560TB per Storage
System
•Ideal areas:
•Real time data
acquisition
•Simulation
•Biomedicine,
Genomics
•Oil & Gas
•Rich media
•Finance
PANASAS Cluster Storage
•Clustered storage
system based on
Panasas File System
•Parallel
•Asynchronous
•Object-based
•Snapshot
•Interface : 4x1GbE,
1x10GbE, IB (router)
•Performace (x shelf)
•500 – 600 MB/s up to
100s GB/S (sequential)
•20 TB per shelf,
200TB/rack, up to PBs
•SSD (optimal for
random I/O)
HPC Storage Systems
File Systems
• NFS
•lustre
•GPFS
•panasas
•AFS
Storage Area Network
•E4 is Qlogic Signature Partner
•Latest technology
•Based on high performance I/F Fibre Channel 4+ 4 Gb multipath
•HA
•Failover for mission critical applicationss (finance, biomedics..)
•Oracle RAC
System’s validation - Rigid quality procedure
•Reliability: basic requirement,
guaranteed by E4’s production cycle
•Selection of quality components
•Production process taken care of in
every detail
•Burn-in to prevent infantile mortality of
the component
•At least 72h accelerated stress test
in a room with high temperature
(35C)
•24h individual test of each subsystem
•48h simultaneous test of each subsystem
•OS installation to prevent HW/SW
incompatibility
19
Case Histories
20
Case History – Oracle RAC
http://www.oracle.com/global/it/customers/pdf/snapshot_gruppodarco_e4.pdf
21
Case History – INTEL cluster @ Enginsoft
May 2007 INTEL Infinicluster
•
•
•
•
•
96 computing nodes Intel
quad core 2,66GHz 4
TFLOPS
1,5 TB RAM
Interconnection: Infiniband
4x DDR 20Gbps
30 TB Storage FC
Application’s fields:
Computer Aided Engineering
22
Case History – CERN computing servers 1U
•
Server 1U with high computing capacity
•
Application’s field: Educational, academic research
•
Customer: CERN (Geneva), main national computing
and research centres
•
2005
•
415 nodes dual Xeon® 2,8Ghz
•
4,6 TFLOPS
•
2006
•
250 nodes Xeon® Woodcrest 3GHz
•
6 TFLOPS
•
2 TB RAM
•
System installed up to July ’08 : over 3000
units
23
Case History – AMD cluster @ CASPUR
2004 Cluster SLACS
June 2005 AMD Infinicluster
•
24 computing nodes
Opteron, 200GFLOPS
•
•
128GB RAM
•
Managed by CASPUR on
behalf of Sardinian
LAboratory for
Computational materials
Science, l'INFM (Istituto
Nazionale Fisica della
Materia)
•
•
•
24 computing nodes Opteron
dual core 2,4GHz,
460GFLOPS
192GB RAM
Interconnection Infiniband
Expanded at 64 nodes: 1,2
TFLOPS, 512GB RAM
24
Case History – CRS4 Cluster 96 core
February 2005
•
96 computing nodes
Opteron Dual core, 384
GFLOPS
•
192 GB RAM in total
•
Application’s fields :
•
•
•
environmental sciences
Renewable energy, fuel
cell
bioinformatics
25
Case History – Cluster HPC Myrinet 2005
Cluster HPC interconnection Myrinet
•
16 computing nodes dual Intel® Xeon® 3.2 GHz
•
High speed interconnetcion Myrinet
•
Storage SCSI to SATA 5 TB
•
Monitor KVM
•
2 switch Ethernet 24 ports layer 3
Application’s fields : Educational, research
Customer : ICAR CNR of Palermo
26
Case History – CNR/ICAR
Hybryd Cluster (CPU + GPU)
12 Computes Nodes
96 core - 24 CPU INTEL “Nehalem” 5520
GFLOPS(peak): 920
RAM: 288 GB
6 GPU server nVIDIA S1070
24 GPU TESLA
5760 core singola precisione
720 core doppia precisione
GLOPS(peak): 24000 (24 TFLOPS)
Case History – CNR/ICAR
Hybryd Cluster (CPU + GPU)
1 Front end Node
48-port Gb Ethernet Switch
24-port Infiniband 20Gb/s Switch
Hybrid cluster CPU/GPU – ICAR CNR Cosenza - ALEPH
Case History – CNR/ICAR
Case History – EPFL
E4: The right partner for HPC
Questions?
Feel free to contact me:
Fabrizio Magugliani
[email protected]
+39 346 9424605
Thank you!
E4 Computer Engineering SpA
Via Martiri della Liberta’ 66
42019 - Scandiano (RE), Italy
www.e4company.com
Switchboard: +39.0522.991811
35