Characterizing NAS Benchmark Performance on Shared Heterogeneous Networks Jaspal Subhlok Shreenivasa Venkataramaiah Amitoj Singh University of Houston Heterogeneous Computing Workshop, April 15, 2002 Rice01, slide 1
Download ReportTranscript Characterizing NAS Benchmark Performance on Shared Heterogeneous Networks Jaspal Subhlok Shreenivasa Venkataramaiah Amitoj Singh University of Houston Heterogeneous Computing Workshop, April 15, 2002 Rice01, slide 1
Characterizing NAS Benchmark Performance on Shared Heterogeneous Networks Jaspal Subhlok Shreenivasa Venkataramaiah Amitoj Singh University of Houston Heterogeneous Computing Workshop, April 15, 2002
Rice01, slide 1
Mapping/Adapting Distributed Applications on Networks Pre Data Model Stream Sim 1 Sim 2 Vis Application ?
Network
Rice01, slide 2
Automatic node selection Select 4 nodes for execution : Choice is easy m-7 Congested route m-6 Busy nodes m-5 m-8 Compute nodes Routers m-1 m-2 m-4 selected nodes m-3
Rice01, slide 3
Automatic node selection Select 5 nodes: choice depends on application m-7 Congested route m-6 Busy nodes m-5 m-8 Compute nodes Routers m-1 m-2 m-4 selected nodes m-3
Rice01, slide 4
Mapping/Adapting Distributed Applications on Networks Pre Data Model Stream Sim 1 Sim 2 Vis ?
Application Network 1) Discover application characteristics and model performance in a shared heterogeneous environment 2) Discover network structure and available resources (e.g., NWS, REMOS) 3) Algorithms to map/remap applications to networks
Rice01, slide 5
Methodology for Building Application Performance Signature Performance signature = model to predict application execution time under given network conditions 1. Execute the application on a controlled testbed 2. Measure
system level activity
during execution
–
such as CPU, communication and memory usage 3. Analyze and discover
program level activity
(message sizes, sequences, synchronization waits) 4. Develop a performance signature
•
No access to source code/libraries assumed
Rice01, slide 6
Discovering application characteristics Executable Application Code Benchmarking on a controlled testbed and analysis Model as a Performance Signature ethernet switch (crossbar) 100 Mbps links 500MHz Pentium Duos
•
capture patterns of CPU loads and traffic during execution
Rice01, slide 7
Executable Application Code Results in this paper Benchmarking on a controlled testbed Measure performance with resource sharing ethernet switch (crossbar) 100 Mbps links 500MHz Pentium Duos
•
capture patterns of CPU loads and traffic during execution Demonstrate that measured resource usage on a testbed is a good predictor of performance on a shared network for NAS benchmarks
Rice01, slide 8
Experiment Procedure
• • • •
Resource utilization of NAS benchmarks measured on a dedicated testbed
– –
CPU probes based on “top” and “vmstat” utility Bandwidth using “iptraf”, “tcpdump”, SNMP queries Performance of NAS benchmark measured with competing loads and limited bandwidth
–
Employ dummynet and NISTnet to limit bandwidth All measurements presented are on 500MHz Pentium Duos, 100 Mbps network, TCP/IP, FreeBSD All results on Class A, MPI, NAS Benchmarks
Rice01, slide 9
Discovered Communication Structure of NAS Benchmarks 0 1 0 1 0 1 2
BT
3 0 1 2
CG
3 0 1 2 0
IS
3 1 2
LU
3 0 2
MG
3 1 2
EP
3 2
SP
3
Rice01, slide 10
140 120 100 80 60 40 20 0 Performance with competing computation loads All nodes are loaded Most busy node loaded Least busy node loaded
• Increase beyond 50% due to lack of coordinated (gang) scheduling and synchronization
EP BT CG IS LU MG SP
• Correlation between low CPU utilization and smaller increase in execution time (e.g. MG shows only ~60% CPU utilization) • Execution time is lower if least busy node has a competing load (20% difference in the busyness level for CG) Rice01, slide 11
Performance with Limited Bandwidth (reduced from 100 to 10Mbps) on one link 140 120 100 80 60 40 20 0 CG IS MG SP BT LU EP Close correlation between link utilization and performance with a shared or slow link 2 0 8 6 4 16 14 12 10
Rice01, slide 12
Performance with Limited Bandwidth (reduced from 100 to 10 Mbps) on all links 500 450 400 350 300 250 200 150 100 50 0 80 70 60 50 40 30 20 10 0 IS CG SP MG BT LU EP Close correlation between total network traffic and performance with all shared or slow links
Rice01, slide 13
Results and Conclusions (not the last slide)
•
Computation and communication patterns can be captured by passive, near non-intrusive, monitoring
•
Benchmarked resource usage pattern is a strong indicator of performance with sharing
–
strong correlation between application traffic and performance with low bandwidth links
–
CPU utilization during normal execution a good indicator of performance with node sharing Synchronization and timing effects were not dominant for NAS Benchnmarks
Rice01, slide 14
Discussion and Ongoing Work (the last slide)
•
Capture application level data exchange pattern from network probes (e.g. MPI message sequence, sizes)
–
slowdown different for different message sizes
•
Infer the main synchronization/waiting patterns
–
Impact of unbalanced execution and lack of gang scheduling
•
Capture impact of CPU scheduling policy for accurate prediction with sharing
–
Policies try to compensate for waits Goal is to build a quantitative “performance signature” to estimate execution time under any given network conditions, and use it in a resource management prototype system
Rice01, slide 15