Transcript Wisdom consists of knowing when to avoid perfection. Tuesday, September 26, 2006 Horowitz

Tuesday, September 26, 2006
Wisdom consists of knowing
when to avoid perfection.
- Horowitz
1
 Quiz 2
 Assignment 1
2
Hypercube: log p dimensions with two nodes in
each dimension
0-D
hypercube
3
Hypercube: log p dimensions with two nodes in
each dimension
0-D
hypercube
1-D
hypercube
4
Hypercube: log p dimensions with two nodes in
each dimension
1-D
hypercube
2-D
hypercube
5
Hypercube: log p dimensions with two nodes in
each dimension
2-D
hypercube
3-D
hypercube
6
Hypercube: log p dimensions with two nodes in
each dimension
3-D
hypercube
4-D
hypercube
Each node is
connected to
d=log p other
nodes
7
•Numbering
•Minimum
distance
between
nodes
8
 Diameter: Maximum distance between any
two processing nodes in the network



Ring
2-D Mesh
Hypercube
9
 Diameter: Maximum distance between any
two processing nodes in the network

Ring
• └p/2┘

2-D Mesh
• 2(√p -1) no-wraparound
• 2 └(√p /2) ┘ wraparound

Hypercube
• log p
10
 Connectivity: Multiplicity of paths

Minimum arcs that need to be removed to disconnect
the network into two
 Ring
• 2

2-D Mesh
• 2 no-wraparound
• 4 wraparound

Hypercube
• d=log p
11
 Bisection width:

Minimum arcs that need to be removed to partition the
network into two equal halves
 Ring
• 2

2-D Mesh
• √p no-wraparound
• 2√p wraparound

Hypercube
• p/2
12
13
Domain Decomposition
 In this type of partitioning, the data associated
with a problem is decomposed. Each parallel
task then works on a portion of the data.
14
Domain Decomposition
15
Functional Decomposition
16
Signal processing
17
Climate modeling
.
18
Examples of decomposition and task
dependencies
19
Examples of decomposition and task dependencies.
20
Examples of decomposition and task dependencies.
21
Granularity
 Fine vs. Coarse

Decomposition in large number of small tasks
vs. small number of large tasks.
 Maximum degree of concurrency
 Average degree of concurrency
 Concurrency vs. Granularity?
22
Granularity
23
Granularity
 Critical Path length:

Longest directed path between any pair of start
and finish nodes is critical path
 Average degree of concurrency:

Ratio of total amount of work to the critical
path length
24
Granularity
•Another example
25
Granularity
Measure of the ratio of computation to
communication.
 Fine-grain Parallelism:


Facilitates load balancing
Implies high communication overhead and less
opportunity for performance enhancement
 Coarse-grain Parallelism:



High computation to communication ratio
Implies more opportunity for performance increase
Harder to load balance efficiently
26
Granularity
 Example:

Domain decompositions for a problem
involving a three-dimensional grid.
27