Transcript PPT

Autar Kaw
Benjamin Rigsby
http://nm.MathForCollege.com
Transforming Numerical Methods Education for STEM Undergraduates
http://nm.MathForCollege.com
1. solve a set of simultaneous linear equations using Naïve
Gauss elimination,
2. learn the pitfalls of the Naïve Gauss elimination method,
3. understand the effect of round-off error when solving a
set of linear equations with the Naïve Gauss elimination
method,
4. learn how to modify the Naïve Gauss elimination
method to the Gaussian elimination with partial pivoting
method to avoid pitfalls of the former method,
5. find the determinant of a square matrix using Gaussian
elimination, and
6. understand the relationship between the determinant of
a coefficient matrix and the solution of simultaneous
linear equations.
A method to solve simultaneous linear equations
of the form [A][X]=[C]
Two steps
1. Forward Elimination
2. Back Substitution
The goal of forward elimination is to transform the coefficient
matrix into an upper triangular matrix
 25 5 1  x1  106.8 
 64 8 1  x   177.2 

  2 

144 12 1  x3  279.2
5
1   x1   106.8 
25
 0  4.8  1.56  x    96.21

  2 

0
0.7   x3   0.735 
 0
A set of n equations and n unknowns
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a21x1  a22 x2  a23 x3  ...  a2 n xn  b2
.
.
.
.
.
.
an1x1  an 2 x2  an3 x3  ...  ann xn  bn
(n-1) steps of forward elimination
Step 1
For Equation 2, divide Equation 1 by a11 and multiply
by a21.
 a21 
 a (a11x1  a12 x2  a13 x3  ...  a1n xn  b1 )
 11 
a21
a21
a21
a21x1 
a12 x2  ... 
a1n xn 
b1
a11
a11
a11
Subtract the result from Equation 2.
a21x1  a22 x2  a23 x3  ...  a2 n xn  b2
a21
a21
a21
− a21x1  a a12 x2  ...  a a1n xn  a b1
11
11
11
_________________________________________________


a21 
a21 
a21
 a22 
a12  x2  ...   a2 n 
a1n  xn  b2 
b1
a11 
a11 
a11


or
a x  ...  a x  b
'
22 2
'
2n n
'
2
Repeat this procedure for the remaining equations
to reduce the set of equations as
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
'
'
a32
x2  a33
x3  ...  a3' n xn  b3'
.
.
.
.
.
.
.
.
.
'
an' 2 x2  an' 3 x3  ...  ann
xn  bn'
End of Step 1
Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
"
a33
x3  ...  a3" n xn  b3"
.
.
.
.
.
.
"
an" 3 x3  ...  ann
xn  bn"
End of Step 2
At the end of (n-1) Forward Elimination steps, the system of
equations will look like
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
"
a33
x3  ...  a3"n xn  b3"
.
.
.
.
.
.
n 1 
 n 1
ann
xn  bn
End of Step (n-1)
a11 a12 a13
 0 a' a'
22
23

"
0 a33
0





 0
0
0
 a1n   x1   b1 
 a'2 n   x2   b2' 
  

"
"
 a3n   x3    b3 
  


      
(n1 )
(n-1 )




0 ann   xn  bn 
Solve each equation starting from the last equation
5
1   x1   106.8 
25
 0  4.8  1.56  x    96.21

  2 

0
0.7   x3   0.735 
 0
Example of a system of 3 equations
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
a x  a x  ...  a x  b
'
22 2
'
23 3
'
2n n
'
2
a x  ...  a x  b
"
33 3
"
n n
.
.
.
.
.
.
 n 1
"
3
n 1 
ann xn  bn
Start with the last equation because it has only one
unknown
( n 1)
n
( n 1)
nn
b
xn 
a
bn( n 1)
x n  ( n 1)
a nn
i 1
xi 
bi
i 1
i 1
 ai ,i 1 xi 1  ai ,i  2 xi  2  ...  ai ,n xn
i 1
aii
i 1
xi 
i 1
bi
n
i 1
  aij x j
j i 1
i 1
ii
a
for i  n  1,...,1
for i  n  1,...,1
THE END
http://numericalmethods.eng.usf.edu
For all resources on this topic such as digital audiovisual lectures,
primers, textbook chapters, multiple-choice tests, worksheets in
MATLAB, MATHEMATICA, MathCad and MAPLE, blogs,
related physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/gaussian_elimination.
html
The upward velocity of a rocket is given at three
different times
Table 1 Velocity vs. time data.
Time, t s  Velocity, v m/s 
5
106.8
8
12
177.2
279.2
The velocity data is approximated by a polynomial as:
vt   a1t 2  a 2 t  a3 ,
Find the velocity at t=6 seconds .
5  t  12.
Assume
vt   a1t 2  a2t  a3 , 5  t  12.
Results in a matrix template of the form:
t12
 2
t 2
t32

t1
t2
t3
1

1
1

 a1   v1 
 a   v 
 2  2

 a3 
 
 v3 

Using data from Table 1, the matrix becomes:
 25 5 1  a1  106.8 
 64 8 1 a   177.2 

  2 

144 12 1 a3  279.2
 25 5 1  a1  106.8 
 25 5 1  106.8 
 64
8 1 a2   177.2    64
8 1  177.2 

   



144 12 1  a3  279.2
144 12 1  279.2
1. Forward Elimination
2. Back Substitution
 25 5 1  a1  106.8 
 25 5 1  106.8 
 64
8 1 a2   177.2    64
8 1  177.2 

   



144 12 1  a3  279.2
144 12 1  279.2
1. Forward Elimination
2. Back Substitution
Forward Elimination
Number of steps of forward elimination is
(n-1)=(3-1)=2
 25 5 1  106.8 
 64 8 1  177.2 


144 12 1  279.2
25
Divide Equation 1 by 25 and
64
 2.56 .
multiply it by 64,
25
5 1  106.8 2.56  64 12.8 2.56  273.408
.
Subtract the result from
Equation 2
Substitute new equation for
Equation 2
64
 64
0
1 
2.56 
177.2
273.408
 4.8  1.56 
 96.208
8
12.8
5
1
 106.8 
 25
 0  4.8  1.56   96.208


144 12
1
 279.2 
5
1
 106.8 
 25
 0  4.8  1.56   96.208


144 12
1
 279.2 
25
Divide Equation 1 by 25 and
144
multiply it by 144,
 5.76 .
25
5 1  106.8 5.76  144 28.8 5.76  615.168
.
Subtract the result from
Equation 3
Substitute new equation
for Equation 3
144
 144
0
1 
5.76 
279.2
615.168
 16.8  4.76 
 335.968
12
28.8
5
1

106.8 
25
 0  4.8  1.56   96.208 


 0  16.8  4.76   335.968
5
1

106.8 
25
 0  4.8  1.56   96.208 


 0  16.8  4.76   335.968
0
Divide Equation 2 by −4.8
and multiply it by −16.8,
 16.8
 3.5.
 4.8
 4.8 1.56   96.208 3.5  0 16.8  5.46   336.728
Subtract the result from
Equation 3
Substitute new equation
for Equation 3
0
 0
0
 16.8  4.76 
335.968
 16.8  5.46   336.728
0
0.7 
0.76
5
1
 106.8 
25
 0  4.8  1.56   96.208


 0
0
0.7 
0.76 
Back Substitution
5
1
 106.8  25
5
1   a1   106.8 
25
 0  4.8  1.56   96.2   0  4.8  1.56 a    96.208

 
  2 

 0
0
0.7  0.7   0
0
0.7   a3   0.76 
Solving for a3
0.7a3  0.76
0.76
a3 
0.7
a3  1.08571
5
1   a1   106.8 
25
 0  4.8  1.56 a    96.208

  2 

 0
0
0.7   a3   0.76 
Solving for a2
 4.8a2  1.56a3  96.208
 96.208  1.56a3
a2 
 4.8
 96.208  1.56 1.08571
a2 
 4.8
a2  19.6905
5
1   a1   106.8 
25
 0  4.8  1.56 a    96.2

  2 

 0
0
0.7   a3   0.76 
Solving for a1
25a1  5a2  a3  106.8
106.8  5a2  a3
a1 
25
106.8  5 19.6905  1.08571

25
 0.290472
 25 5 1  a1  106.8 
 64
8 1 a2   177.2 

   

144 12 1  a3  279.2
 a1  0.290472
a    19.6905 
 2 

 a3   1.08571 
Solution
The solution vector is
 a1  0.290472
a    19.6905 
 2 

 a3   1.08571 
The polynomial that passes through the three data points is then:
vt   a1t 2  a2t  a3
 0.290472t 2  19.6905t  1.08571, 5  t  12
v6   0.2904726   19.69056   1.08571
 129.686 m/s .
2
THE END
http://numericalmethods.eng.usf.edu
Pitfall#1. Division by zero
10 x2  7 x3  3
6 x1  2 x2  3x3  11
5 x1  x2  5 x3  9
0 10  7  x1   3 
6 2
3   x2   11

   
5  1 5   x3   9 
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
5 x1  x2  5 x3  9
12 10  7  x1  15
6 5
3   x2   14

   
 5  1 5   x3   9 
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
24 x1  x2  5 x3  28
12 10  7  x1  15
6 5
3   x2   14 

   
24  1 5   x3  28
12 10  7  x1   15 
0
0
6.5  x2   6.5

   
12  21 19   x3   2
Division by zero is a possibility at any step of forward
elimination
Pitfall#2. Large Round-off Errors
15
10  x1   45 
 20
 3  2.249 7   x   1.751

  2 

1
3   x3   9 
 5
Exact Solution
 x1  1
 x   1
 2  
 x3  1
Pitfall#2. Large Round-off Errors
15
10  x1   45 
 20
 3  2.249 7   x   1.751

  2 

1
3   x3   9 
 5
 x1   0.9625 
 x    1.05 
 2 

 x3  0.999995
Solve it on a computer using
6 significant digits with chopping
Pitfall#2. Large Round-off Errors
15
10  x1   45 
 20
 3  2.249 7   x   1.751

  2 

1
3   x3   9 
 5
Solve it on a computer using
5 significant digits with chopping
 x1   0.625 
 x    1.5 
 2 

 x3  0.99995
Is there a way to reduce the round off error?
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error
THE END
http://numericalmethods.eng.usf.edu

Possible division by zero

Large round-off errors
Increase the number of significant digits
•
Decreases round-off error
• Does not avoid division by zero
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error
At the beginning of the kth step of forward elimination,
find the maximum of
akk , ak 1,k ,................, ank
If the maximum of the values is a pk in the p th row,
k  p  n, then switch rows p and k.
a11 a12
 0 a'
22

'
 0 a32




 0 a'n 2
a13 
'
a23 
'
a33

 
an' 3 an' 4
a1n   x1   b1 
' 
' 


a 2 n x2
b2
   
a3' n   x3   b3' 
   
      
'
  xn  bn' 
ann
6 14 5.1 3.7 6   x1   5 
0  7 6
1
2   x2   6

   
12 1 11  x3    8 
0 4

   
x
0
9
23
6
8
9
4

   
0  17 12 11 43  x5   3 
Which two rows would you switch?
6 14 5.1 3.7 6   x1   5 
0  17 12 11 43  x   3 

 2   
12 1 11  x3    8 
0 4

   
23 6
8   x4   9 
0 9
0  7 6
1
2   x5   6
Switched Rows
A method to solve simultaneous linear
equations of the form [A][X]=[C]
Two steps
1. Forward Elimination
2. Back Substitution
Same as naïve Gauss elimination method except
that we switch rows before each of the (n-1)
steps of forward elimination.
a11 a12
 0 a'
22

'
 0 a32



 0 a'n 2
a13 
'
a23 
'
a33 
 
'
'
an 3 an 4
a1n   x1   b1 
' 
' 


a 2 n x2
b2
   
'
'
a3n   x3   b3 
   
      
'
'




ann   xn  bn 
a11 a12 a13
 0 a' a'
22
23

"
0 a33
0




 0
0
0
 a1n   x1   b1 
'
' 




 a2 n x2
b2
  

"
"
 a3n   x3    b3 
  


      
(n1 )
(n-1 )




0 ann   xn  bn 
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
"
a33
x3  ...  an" xn  b3"
.
.
.
 n 1
.
.
.
n 1 
ann xn  bn
bn( n 1)
x n  ( n 1)
a nn
i 1
xi 
bi
n
i 1
  aij x j
j i 1
i 1
ii
a
for i  n  1,...,1
THE END
http://numericalmethods.eng.usf.edu
Solve the following set of equations by
Gaussian elimination with partial pivoting
5 1  a1  106.8 
 25
 64





8 1 a 2  177.2

   

144 12 1  a3  279.2
 25 5 1  a1  106.8 
 25 5 1  106.8 
 64 8 1 a   177.2   

64
8
1

177
.
2
2

   



144 12 1  a3  279.2
144 12 1  279.2
1. Forward Elimination
2. Back Substitution
Forward Elimination
Number of steps of forward elimination is
(n-1)=(3-1)=2
 Examine absolute values of first column, first row
and below.
25 , 64 , 144
 Largest absolute value is 144 and exists in row 3.
 Switch row 1 and row 3.
 25 5 1  106.8 
144 12 1  279.2
 64 8 1  177.2    64 8 1  177.2 




144 12 1  279.2
 25 5 1  106.8 
144 12 1  279.2
 64 8 1  177.2 


 25 5 1  106.8 
144
Divide Equation 1 by 144 and
multiply it by 64,
64
 0.4444 .
144
12 1  279.2 0.4444  63.99 5.333 0.4444  124.1
Subtract the result from
Equation 2
Substitute new equation
for Equation 2
64
 63.99
0
8
1  177.2
5.333 0.4444 
124.1
2.667 0.5556  53.10
12
1
 279.2
144
 0 2.667 0.5556  53.10


 25
5
1
 106.8 
12
1
 279.2
144
 0 2.667 0.5556  53.10 Divide Equation 1 by 144 and


25
 0.1736.
multiply
it
by
25,
 25
5
1
 106.8 
144
144
12 1  279.2 0.1736  25.00 2.083 0.1736  48.47
Subtract the result from
Equation 3
Substitute new equation for
Equation 3
25
 25
0
5
1  106.8
2.917
0.8264  58.33
2.083 0.1736  48.47
12
1
 279.2
144
 0 2.667 0.5556  53.10


 0 2.917 0.8264  58.33

Examine absolute values of second column, second
row and below.
2.667 , 2.917
 Largest absolute value is 144 and exists in row 3.
 Switch row 1 and row 3.
1
 279.2
1
 279.2
144 12
144 12
 0 2.667 0.5556  53.10   0 2.917 0.8264  58.33




 0 2.917 0.8264  58.33
 0 2.667 0.5556  53.10
12
1
 279.2
144
 0 2.917 0.8264  58.33


 0 2.667 0.5556  53.10
0
Divide Equation 2 by 2.917 and
multiply it by
2.667
2.667, 2.917  0.9143.
2.917 0.8264  58.33 0.9143  0 2.667 0.7556  53.33
Subtract the result from
Equation 3
0
 0
0
Substitute new equation for
Equation 3
12
1
 279.2 
144
 0 2.917 0.8264  58.33 


 0
0
 0.2   0.23
2.667 0.5556 
2.667 0.7556 
0
53.10
53.33
 0.2   0.23
Back Substitution
1
 279.2 
1 
144 12
144 12
 0 2.917 0.8264  58.33    0 2.917 0.8264




 0
 0
0
 0.2   0.23
0
 0.2 
Solving for a3
 0.2a3  0.23
 0.23
a3 
 0.2
 1.15
 a1   279.2 
a    58.33 
 2 

 a3   0.23
1 
144 12
 0 2.917 0.8264


0
 0.2 
 0
 a1   279.2 
a    58.33 
 2 

 a3   0.23
Solving for a2
2.917a2  0.8264a3  58.33
58.33  0.8264a3
a2 
2.917
58.33  0.8264 1.15

2.917
 19.67
1 
144 12
 0 2.917 0.8264


0
 0.2 
 0
 a1   279.2 
a    58.33 
 2 

 a3   0.23
Solving for a1
144a1  12a2  a3  279.2
279.2  12a2  a3
a1 
144
279.2  12 19.67  1.15

144
 0.2917
 25 5 1  a1  106.8 
 64 8 1 a   177.2 

  2 

144 12 1  a3  279.2
 a1  0.2917
a    19.67 
 2 

 a3   1.15 
Consider the system of equations
10 x1  7 x2  7
 3x1  2.099 x2  6 x3  3.901
5 x1  x2  5 x3  6
In matrix form
 7 0  x1 
 7 
 10
3.901
 3 2.099 6  x 


  2 = 
 6 
 5
 1 5  x 3 
Solve using Gaussian Elimination with Partial Pivoting using five significant
digits with chopping
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch row1
with row1.
Performing Forward Elimination
 7 0  x1   7 
 10
 3 2.099 6  x   3.901

 2  

 5
 1 5  x3   6 

7
0  x1   7 
10
 0  0.001 6  x   6.001

 2  

 0
2.5
5  x3   2.5 
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is switched
with row 3
Performing the row swap
7
0  x1   7 
10
 0  0.001 6  x   6.001

 2  

 0
2.5
5  x3   2.5 

7
0  x1   7 
10
0
  x    2.5 
2
.
5
5

 2  

 0  0.001 6  x3  6.001
Forward Elimination: Step 2
Performing the Forward Elimination results in:
0   x1   7 
10  7
 0 2.5
  x    2.5 
5

 2  

 0 0 6.002  x3  6.002
Back Substitution
Solving the equations through back substitution
0   x1   7 
10  7
 0 2.5
  x    2.5 
5

 2  

 0 0 6.002  x3  6.002
7  7 x 2  0 x3
x1 
0
10
2.5  5 x3
x2 
 1
2.5
6.002
x3 
1
6.002
Compare the calculated and exact solution.
The fact that they are equal is coincidence, but it does illustrate
the advantage of Partial Pivoting
 x1   0 
X  calculated   x2    1
 x3   1 
X  exact
 x1   0 
  x 2    1
 x3   1 
THE END
http://numericalmethods.eng.usf.edu
Determinant of a Square Matrix Using
Naïve Gauss Elimination Example
If a multiple of one row of [A]nxn is added or subtracted
to another row of [A]nxn to result in [B]nxn then
det(A)=det(B)
The determinant of an upper triangular matrix [A] n n is
a ii
given by det A   a11  a22  ...  aii  ...  ann 
i 1
n
Using forward elimination to transform [A]n n to a
upper triangular matrix, [U]n n .
Ann  U  nn
det  A  det U 
Using naïve Gaussian elimination find the
determinant of the following square matrix.
 25 5 1
 64 8 1


144 12 1
Forward Elimination
 25 5 1
Divide Equation 1 by 25 and
 64 8 1
64


multiply it by 64, 25  2.56 .
144 12 1
25 5 1 2.56  64 12.8 2.56
.
Subtract the result from
Equation 2
Substitute new equation for
Equation 2
64
 64
0
8
12.8
1
2.56
 4.8  1.56
5
1 
 25
 0  4.8  1.56


144 12
1 
5
1 
 25
Divide Equation 1 by 25 and
 0  4.8  1.56
144
 5.76 .
multiply it by 144,


25
144 12
1 
25 5 1 5.76  144 28.8 5.76
.
Subtract the result from
Equation 3
Substitute new equation for
Equation 3
144 12
 144 28.8
0  16.8
1
5.76
 4.76
5
1 
25
 0  4.8  1.56 


 0  16.8  4.76
5
1 
25
 0  4.8  1.56 


 0  16.8  4.76
0
Divide Equation 2 by −4.8
and multiply it by −16.8,
 16.8
 3.5 .
 4.8
 4.8  1.56  3.5  0  16.8  5.46
.
Subtract the result from
Equation 3
Substitute new equation for
Equation 3
0
 0
0
 16.8  4.76
 16.8  5.46
0
0.7
5
1 
25
 0  4.8  1.56


 0
0
0.7 
After forward elimination
5
1 
 25 5 1
25
 64 8 1   0  4.8  1.56




0
0.7 
144 12 1
 0
det A   u11  u22  u33
 25   4.8 0.7
 84.00

Forward Elimination

Back Substitution

Pitfalls

Improvements

Partial Pivoting

Determinant of a Matrix
For all resources on this topic such as digital audiovisual lectures,
primers, textbook chapters, multiple-choice tests, worksheets in
MATLAB, MATHEMATICA, MathCad and MAPLE, blogs,
related physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/gaussian_elimination.
html
THE END
http://numericalmethods.eng.usf.edu