Transcript Lecture 8
Numerical Computation Lecture 8: Matrix Algorithm Analysis United International College Review Presentation • 10 minutes for review presentation. Review • During our Last Class we covered: – Gauss-Jordan Method for finding Inverses Today • We will cover: – Operation count for Gaussian Elimination, LU Factorization – Accuracy of Matrix Methods – Readings: • Pav, section 3.4.1 • Moler, section 2.8 Operation Count for Gaussian Elimination • How many floating point operations (+-*/) are used by the Gaussian Elimination algorithm? • Definition: Flop = floating point operation. We will consider a division to be equivalent to a multiplication, and a subtraction equivalent to an addition. • Thus, 2/3 = 2*(1/3) will be considered a multiplication. • 2-3 = 2 + (-3) will be considered an addition. Operation Count for Gaussian Elimination • In Gaussian Elimination we use row operations to reduce • to Operation Count for Gaussian Elimination • Consider the number of flops needed to zero out the entries below the first pivot a11 . Operation Count for Gaussian Elimination • First a multiplier is computed for each row below the first row. This requires (n-1) multiplies. m = A(i,k)/A(k,k); • Then in each row below row 1 the algorithm performs n multiplies and n adds. (A(i,j) = A(i,j) - m*A(k,j);) • Thus, there is a total of (n-1) + (n-1)*2*n flops for this step of Gaussian Elimination. • For k=1 algorithm uses 2n2 –n -1 flops Operation Count for Gaussian Elimination • For k =2, we zero out the column below a22 . • There are (n-2) rows below this pivot, so this takes 2(n-1)2 –(n-1) -1 flops. • For k -3, we would have 2(n-2)2 –(n-2) -1 flops, and so on. • To complete Gaussian Elimination, it will take In flops, where Operation Count for Gaussian Elimination • Now, • So, In = (2/6)n(n+1)(2n+1) – (1/2)n(n+1) – n = [(1/3)(2n+1)-(1/2)]*n(n+1) – n = [(2/3)n – (1/6)] * n(n+1) - n = (2/3)n3 + (lower power terms in n) • Thus, the number of flops for Gaussian Elimination is O(n3). Operation Count for LU Factorization • In the algorithm for LU Factorization, we only do the calculations described above to compute L and U. This is because we save the multipliers (m) and store them to create L. • So, the number of flops to create L and U is O(n3). Operation Count for using LU to solve Ax = b • Once we have factored A into LU, we do the following to solve Ax = b: • Solve the two equations: • Lz = b • Ux = z • How many flops are needed to do this? Operation Count for using LU to solve Ax = b • To solve Lz=b we use forward substitution z= z1 = b1 , so we use 0 flops to find z1. z2 = b2 – l21 *z1 , so we use 2 flops to find z2 . z3 = b3 – l31 *z1 – l32 *z2 , so we use 4 flops to find z2 , and so on. Operation Count for using LU to solve Ax = b • To solve Lz=b we use forward substitution z= • Totally, 0+2+4+ … + 2*(n-1)= 2*(1+2+…+(n-1)) = 2*(1/2)*(n-1)(n) = n2 – n. • So, the number of flops for forward substitution is O(n2). Operation Count for using LU to solve Ax = b • To solve Ux=z we use backward substitution • A similar analysis to that of forward substitution shows that the number of flops for backward substitution is also O(n2). • Thus, the number of flops for using LU to solve Ax=b is O(n2). Summary of Two Methods • Gaussian Elimination requires O(n3) flops to solve the linear system Ax = b. • To factor A = LU requires O(n3) flops • Once we have factored A = LU, then, using L and U to solve Ax = b requires O(n2) flops. • Suppose we have to solve Ax = b for a given matrix A, but for many different b vectors. What is the most efficient way to do this? Accuracy of Matrix Methods • Our algorithms are used to find the solution x to the system Ax=b. • But, how close to the exact solution is the computed solution? • Let x* be the computed solution and x be the exact solution. Accuracy of Matrix Methods • Definition: The error e is defined to be e = x - x* • Definition: The residual r is defined to be r = b – Ax* • Note: These two quantities may be quite different! Accuracy of Matrix Methods • Consider the system: 0.780 0.563 x1 0.217 0.913 0.659 x2 0.254 • Using Gaussian Elimination with partial pivoting, we swap rows 1 and 2 0.913 0.659 x1 0.254 0.780 0.563 x2 0.217 Accuracy of Matrix Methods • Suppose we had a computer with just 3 digit accuracy. The first multiplier would be: • Subtracting 0.854*row 1 from row 2, we get Accuracy of Matrix Methods • Solving this using back substitution gives • So, Accuracy of Matrix Methods • The exact solution is • Thus, the error is 1.000 (0.443) 0.557 e x x* 1.000 1.000 2.000 • The error is bigger than the solution!! Accuracy of Matrix Methods • The residual is • The residual is very small, but the error is very large! Accuracy of Matrix Methods • Theorem (sort of): Gaussian elimination with partial pivoting is guaranteed to produce small residuals. • Why is the error in our example so large? Accuracy of Matrix Methods • If we did Gaussian Elimination with much higher accuracy (more than 3 digits) we would see that the row reduction produces: • This matrix is very close to being singular (why?) • The relationship between the size of the residual and the size of the error is determined in part by a quantity known as the condition number of the matrix, which is the subject of our next lecture.