Wisdom consists of knowing when to avoid perfection. Tuesday, September 26, 2006 Horowitz
Download
Report
Transcript Wisdom consists of knowing when to avoid perfection. Tuesday, September 26, 2006 Horowitz
Tuesday, September 26, 2006
Wisdom consists of knowing
when to avoid perfection.
- Horowitz
1
Quiz 2
Assignment 1
2
Hypercube: log p dimensions with two nodes in
each dimension
0-D
hypercube
3
Hypercube: log p dimensions with two nodes in
each dimension
0-D
hypercube
1-D
hypercube
4
Hypercube: log p dimensions with two nodes in
each dimension
1-D
hypercube
2-D
hypercube
5
Hypercube: log p dimensions with two nodes in
each dimension
2-D
hypercube
3-D
hypercube
6
Hypercube: log p dimensions with two nodes in
each dimension
3-D
hypercube
4-D
hypercube
Each node is
connected to
d=log p other
nodes
7
•Numbering
•Minimum
distance
between
nodes
8
Diameter: Maximum distance between any
two processing nodes in the network
Ring
2-D Mesh
Hypercube
9
Diameter: Maximum distance between any
two processing nodes in the network
Ring
• └p/2┘
2-D Mesh
• 2(√p -1) no-wraparound
• 2 └(√p /2) ┘ wraparound
Hypercube
• log p
10
Connectivity: Multiplicity of paths
Minimum arcs that need to be removed to disconnect
the network into two
Ring
• 2
2-D Mesh
• 2 no-wraparound
• 4 wraparound
Hypercube
• d=log p
11
Bisection width:
Minimum arcs that need to be removed to partition the
network into two equal halves
Ring
• 2
2-D Mesh
• √p no-wraparound
• 2√p wraparound
Hypercube
• p/2
12
13
Domain Decomposition
In this type of partitioning, the data associated
with a problem is decomposed. Each parallel
task then works on a portion of the data.
14
Domain Decomposition
15
Functional Decomposition
16
Signal processing
17
Climate modeling
.
18
Examples of decomposition and task
dependencies
19
Examples of decomposition and task dependencies.
20
Examples of decomposition and task dependencies.
21
Granularity
Fine vs. Coarse
Decomposition in large number of small tasks
vs. small number of large tasks.
Maximum degree of concurrency
Average degree of concurrency
Concurrency vs. Granularity?
22
Granularity
23
Granularity
Critical Path length:
Longest directed path between any pair of start
and finish nodes is critical path
Average degree of concurrency:
Ratio of total amount of work to the critical
path length
24
Granularity
•Another example
25
Granularity
Measure of the ratio of computation to
communication.
Fine-grain Parallelism:
Facilitates load balancing
Implies high communication overhead and less
opportunity for performance enhancement
Coarse-grain Parallelism:
High computation to communication ratio
Implies more opportunity for performance increase
Harder to load balance efficiently
26
Granularity
Example:
Domain decompositions for a problem
involving a three-dimensional grid.
27