Transcript Visible Human User Interface Team

Visible Human
Networking Update
Thomas Hacker
June 3, 2000
Outline
•
•
•
•
•
•
Latest News
U-M Network Connectivity
Existing CPC Network Topology
Proposed CPC Network Topology
Performance Improvement Program
NPACI Storage Resource Broker
Performance
• Goals
Network Status
• In the works …
– CPC Upgrade from OC-3 155Mbps to OC-12
622Mbps
– CPC Router Purchase
• OC-12 line installed from Media Union to
Arbor Lakes, needs some tuning to improve
performance
University of Michigan Network Topology
VBNS/Abilene Research
Commercial/Net
X
GW
VBNS
Abilene
NSI
Cleveland
POP
Quin
Mich NET
DOE
Mich
Giga PoP
DREN
MERIT BIN
OCR
UM BIN
FDDI
OC-12
300-450m
17m
FW
ITD
FDDI
CAEN
OC-12
1 2 3 4
5
CPC
X
FW
MCIT
Firewalls
Existing CPC High Performance Network Topology 5/2000
144 GB Local Disk
FWD SCSI
FWD SCSI
1 Processor 1 GB Mem
New ADSM/Tivoli
Robotic Tape Controller
IBM F-50
Switched
10 BaseT
SP2 Control
Workstation
(48)
66 MHz Nodes
Switchless
(Training Nodes)
SP2 Build/
Submit Node
100 Mb/sec FDDI
1 Processor 512 Mb Mem
Switched
10 BaseT
48 Switched
10 BaseT Lines
48 Switched
100 BaseT Lines
100 Mb/sec FDDI
4 EtherChannel Bonded
Switched 100 BaseT Lines
2 Processor 2 GB Mem
New ADSM/Tivoli Mass
Storage Sever
IBM F-50
NPACI AFS
Storage Server
IBM F-50
LVD
SCSI
144 GB
RAID 1
FWD SCSI
216 GB
Dual
Channel
RAID 1
IBM NPACI Globus
Research Server
CPC-CISCO 5500 mr
CPC CISCO 5500
(64) 166 MHz
Nodes (SDSC)
June 2000
IBM 3995 288 GB
CD-R Juke Box
STK
Timberwolf 9710
41 TB
4 GB, 16 Processors
100 Mb/sec FDDI
CISCO CDDI/FDDI
Concentrator
Switched 100 BaseT
(16) 166 MHz
Nodes (SDSC)
June 2000
SGI
Powerchallenge
100 Mb/sec FDDI
(48)
166 MHz Nodes
(Production 1 GB
Mem Nodes)
DIFF
SCSI
100 Mb/sec FDDI
UMMU-CIS7513.9
OC-3 ATM 155 Mb/Sec
Existing ADSM Robotic
Tape Controller
CPC IBM SP2 Complex
Existing ADSM Mass
Storage Server
OC-3 ATM 155 Mb/Sec
Ethernet Switch
Switched 100 BaseT
Switched 100
BaseT Lines
4 EtherChannel Bonded
Switched 100 BaseT Lines
2 Processor 2 GB Mem
New Tivoli/ADSM
Backend Server
+
Engineering Research
Host Backup
IBM H-50
LVD SCSI
100 GB ADSM
Data Vole
2 Ether-Channel
Bonded Switched
100 BaseT Lines
LVD
SCS
I
Switched
100 BaseT Line
200 GB
Backup/Archive
2 Processor 2 GB Mem
New Nodes SP2
Control Workstation
IBM F-50
LINUX Cluster Firewall
4 Processor 2 GB
Multi-Resident/
Resident AFS Mass
Storage Server
IBM F-50
48 port 10BaseT Hub
4 Processor 2 GB Mem
Network Performance
Measurement Station
IBM F-50
CPC
BEOWULF
LINUX
Cluster
2 Processor 1 GB Mem
New ADSM/Tivoli
Robotic Tape Controller
IBM F-50
SGI 12 GB, 16
Processors
SGI Origin 2000
UMMU-CIS 1010
Media Union
ATM Switch
CISCO Lightstream
1010
ATM OC-12
622 Mb/sec
32 10 BaseT Lines
Dual Ppro
200 MHz 128 Mb
RAM per node
32 Switched 100BaseT
Lines
CISCO 2900XL
10/100 Switch
CPC Beowulf Complex
UMMU-cis7513.9
Chry-CIS 1010
Chrysler ATM
Switch CISCO
Lightstream 1010
OC-12
To Abilene
622 Mb/sec
Proposed CPC High Performance Network Topology
UMVH
Production
Prototype
Compaq
Cluster
LVD
SCSI
200 GB
Backup/Archive
LVD
SCSI
144 GB RAID 1
CPC IBM SP2 Complex
SP2 Build/
Submit Node
(48)
166 MHz Nodes
(Production 1GB
Mem Nodes)
1000 Mb/sec
Gigabit Ethernet
Switched 100 BaseT
(16) 166 MHz
Nodes (SDSC)
June 2000
CISCO 5500
Ethernet
Switch
(64) 166 MHz
Nodes (SDSC)
June 2000
Nodes
1000 Mb/sec
Gigabit Ethernet
2 Processor 2 GB Mem
SDSC Nodes SP2
Control Workstation
IBM F-50
4 Processor 2 GB
Multi-Resident/
Resident AFS Mass
Storage Server
IBM F-50
• 588 DLT7000 Tape Cartridges
• 6 DLT7000 Tape Drives
IBM F-50
1000 Mb/sec
Gigabit Ethernet
Media
Union
ATM
Router
OC-12
622 Mb/sec
CISCO 6509
Router
1,000 Mb/sec
Gigabit Ethernet
OC-12
622 Mb/sec
Engineering
ATM Cloud
- or To ITDCooley ATM
Switch
OC-12
622 Mb/sec
Arbor
Lakes
ATM
Switch
Abilene
UMVH
SGI OCTANE
Webserver
1,000 Mb/sec
Gigabit Ethernet
48 port 10 baseT Hub
CPC
BEOWULF
LINUX
Cluster
4 Processor 2
GB Mem
Network
Performance
Measurement
Station
IBM F-50
SGI Origin
2000
SGI Power
Challenge
Server
OC-48
Gigabit Ethernet
32 Switched 100BaseT Lines
CPC Beowulf Complex
1000 Mb/sec
Gigabit Ethernet
Media Union
SGI Onyx2
OC-12
CISCO 2900XL 10/100 Switch
FW SCSI
Storage
Technology
Timberwolf 9710
DLT Tape Silo 41 TB
LINUX Cluster Firewall
Dual Ppro
200 Mhz 128 Mb
RAM per node
36GB
AFS Disk
1000 Mb/sec
Gigabit Ethernet
1000 Mb/sec
Gigabit Ethernet
(48)
66 MHz Nodes
Switchless
(Training Nodes)
1,000 Mb/sec
Gigabit Ethernet
Switched
10 BaseT
IBM F-50
SP2 Control
Workstation
1440 GB Sun
A1000 Disk
2 Processor
FWD SCSI
FWD
2 GB Mem
SCSI
FWD SCSI
New Tivoli/ADSM
Backend Server
ADSM/Tivoli
ADSM/Tivoli
+
Robotic Tape
Engineering Research Mass Storage
System (42 TB)
Controller
Host Backup
4 Processor 2 GB
2 Processor 1 GB
IBM H-50
NPACI AFS Server
IBM F-50
4 EtherChannel Bonded 100 Mb/sec
Switched Ethernet
Switched
10 BaseT
216 GB Dual Channel RAID 1
LVD
SCSI
100 GB ADSM
Data Vol
SGI 4 GB,
16 Processors
SGI Origin 2000
SGI 12 GB,
16 Processors
SGI Origin 2000
High Performance Networking Upgrade Program
Platform
Approach
Application:
System:
Network:
Training, ImBench
performance testing
User training & consulting to optimize
application utilization of network
Network performance testing & tuning
(Matt Mathis–PSC/NLAR TCP testrig)
Move from 10/100
Optimize facilities and operations use
by performance tuning & host network
adapter upgrade
Move from OC-3
G.E./ATM
OC-12
Upgrade physical infrastructure
Applications
Current Applications That Will Immediately Benefit From Upgrade Program
• NPACI Storage Resource Broker/ Reagan Moore – Data Intensive Computing Environment Thrust
Area
• University of Texas/University of Michigan SP2 system integration collaboration
AFS file system, joint job scheduling, Engineering Materials Database sharing
• Multi-resident AFS wide area mass storage file systems (AFS server that utilizes mass storage system)
• Job scheduling/moving large multi-gigabyte files between NPACI and National HPC
sites/Grid computing infrastructure (Globus, Legion, Linux Clusters)
• NPACI thrust areas: Engineering/Education, Outreach and Training/ ISR-Social Science
• Visible Human/NPACI Neurology Collaboration/
Brian Athey @ U-M; Human Cochlear Modeling, Ed Givelberg @ U-M; Mark Ellisman @ UCSD,
Application Data Transfer
1st Read
File System
Disk B/W
1st File System
Read
2nd File
System
Read
Virtual
Memory
Cached
File System
Read B/W
Dependent on
Write
File System File System
“dirty page” Policy
BW
Write Virtual
Memory B/W
Network Socket
TCP/IP
Measure TCP/IP
Loopback Network
B/W
Measure TCP/IP
Loopback Network
B/W
Network Interface Card
Network Interface Card
Router
Router
Link Characteristics
– Link B/W
– Link RTT
– Link Packet Loss
Router
Router Router
Router
Router
DataTransfer.ppt
Time for
Application to
Process Data
Application
Time for
Application to
Process Data
Application
TCP/IP B/W
bcopy
Memory Read
B/W
File System
• Bi-Directional B/W
• Route Stability “Bi-Directional”
• Latency (RTT)
• Path Length (Hops)
Internet
Characteristics
Networking Goals
• Get new router and tuned OC-12 connection
in place
• Move critical servers and HPC systems to
gigabit ethernet
• Develop system, network and internet
performance tuning toolset and expertise to
optimize use of resources