CS 584 Lecture 23
Download
Report
Transcript CS 584 Lecture 23
CS 584
Assignment
Systems of Linear Equations
A linear equation in n variables has the form
a0x0 + a1x1 + … + an-1xn-1
= b
A set of linear equations is called a system.
A solution exists for a system iff the solution
satisfies all equations in the system.
Many scientific and engineering problems
take this form.
Solving Systems of Equations
a0,0x0 + a0,1x1 + … + a0,n-1xn-1
= b0
a1,0x0 + a1,1x1 + … + a1,n-1xn-1
= b1
an-1,0x0 + an-1,1x1 + … + an-1,n-1xn-1 = bn-1
Many such systems are large.
– Thousands of equations and unknowns
Solving Systems of Equations
A linear system of equations can be
represented in matrix form
a0,0
a0,1
… a0,n-1
x0
a1,0
a1,1
… a1,n-1
x1
an-1,0
an-1,1 … an-1,n-1
Ax = b
xn-1
b0
=
b1
bn-1
Solving Systems of Equations
Solving a system of linear equations is done
in two steps:
– Reduce the system to upper-triangular
– Use back-substitution to find solution
These steps are performed on the system in
matrix form.
– Gaussian Elimination, etc.
Solving Systems of Equations
Reduce the system to upper-triangular form
a0,0
a0,1
… a0,n-1
x0
0
a1,1
… a1,n-1
x1
0
0
… an-1,n-1
xn-1
Use back-substitution
b0
=
b1
bn-1
Reducing the System
Gaussian elimination systematically
eliminates variable x[k] from equations k+1
to n-1.
– Reduces the coefficients to zero
This is done by subtracting a appropriate
multiple of the kth equation from each of the
equations k+1 to n-1
Procedure GaussianElimination(A, b, y)
for k = 0 to n-1
/* Division Step */
for j = k + 1 to n - 1
A[k,j] = A[k,j] / A[k,k]
y[k] = b[k] / A[k,k]
A[k,k] = 1
/* Elimination Step */
for i = k + 1 to n - 1
for j = k + 1 to n - 1
A[i,j] = A[i,j] - A[i,k] * A[k,j]
b[i] = b[i] - A[i,k] * y[k]
A[i,k] = 0
endfor
endfor
end
Parallelizing Gaussian Elim.
Use domain decomposition
– Rowwise striping
Division step requires no communication
Elimination step requires a one-to-all
broadcast for each equation.
No agglomeration
Initially map one to to each processor
Communication Analysis
Consider the algorithm step by step
Division step requires no communication
Elimination step requires one-to-all bcast
– only bcast to other active processors
– only bcast active elements
Final computation requires no
communication.
Communication Analysis
One-to-all broadcast
– log2q communications
– q = n - k - 1 active processors
Message size
– q active processors
– q elements required
T = (ts + twq)log2q
Computation Analysis
Division step
– q divisions
Elimination step
– q multiplications and subtractions
Assuming equal time --> 3q operations
Computation Analysis
In each step, the active processor set is
reduced by one resulting in:
CompTime = k =0 n - k - 1
n -1
CompTime = 3n(n - 1) / 2
Can we do better?
Previous version is synchronous and
parallelism is reduced at each step.
Pipeline the algorithm
Run the resulting algorithm on a linear array
of processors.
Communication is nearest-neighbor
Results in O(n) steps of O(n) operations
Pipelined Gaussian Elim.
Basic assumption: A processor does not need to
wait until all processors have received a value to
proceed.
Algorithm
– If processor p has data for other processors,
send the data to processor p+1
– If processor p can do some computation using
the data it has, do it.
– Otherwise, wait to receive data from processor
p-1
Conclusion
Using a striped partitioning method, it is
natural to pipeline the Gaussian elimination
algorithm to achieve best performance.
Pipelined algorithms work best on a linear
array of processors.
– Or something that can be linearly mapped
Would it be better to block partition?
– How would it affect the algorithm?