Transcript LU Decomposition - MATH FOR COLLEGE

LU Decomposition
Industrial Engineering Majors
Authors: Autar Kaw
http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
7/17/2015
http://numericalmethods.eng.usf.edu
1
LU Decomposition
http://numericalmethods.eng.usf.edu
LU Decomposition
LU Decomposition is another method to solve a set of
simultaneous linear equations
Which is better, Gauss Elimination or LU Decomposition?
To answer this, a closer look at LU decomposition is
needed.
http://numericalmethods.eng.usf.edu
LU Decomposition
Method
For most non-singular matrix [A] that one could conduct Naïve Gauss
Elimination forward elimination steps, one can always write it as
[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix
http://numericalmethods.eng.usf.edu
How does LU Decomposition work?
If solving a set of linear equations
If [A] = [L][U] then
Multiply by
Which gives
Remember [L]-1[L] = [I] which leads to
Now, if [I][U] = [U] then
Now, let
Which ends with
and
[A][X] = [C]
[L][U][X] = [C]
[L]-1
[L]-1[L][U][X] = [L]-1[C]
[I][U][X] = [L]-1[C]
[U][X] = [L]-1[C]
[L]-1[C]=[Z]
[L][Z] = [C] (1)
[U][X] = [Z] (2)
http://numericalmethods.eng.usf.edu
LU Decomposition
How can this be used?
Given [A][X] = [C]
1. Decompose [A] into [L] and [U]
2. Solve [L][Z] = [C] for [Z]
3. Solve [U][X] = [Z] for [X]
http://numericalmethods.eng.usf.edu
When is LU Decomposition better
than Gaussian Elimination?
To solve [A][X] = [B]
Table. Time taken by methods
Gaussian Elimination
 8n 3
4n 
T 
 12 n 2  
3 
 3
LU Decomposition
 8n 3
4n 
T 
 12 n 2  
3 
 3
where T = clock cycle time and n = size of the matrix
So both methods are equally efficient.
http://numericalmethods.eng.usf.edu
To find inverse of [A]
Time taken by Gaussian Elimination
Time taken by LU Decomposition
 nCT |FE CT |BS 
 CT |LU  n  CT |FS  n  CT |BS
 8n 4
4n 2 
3

 T 
 12n 
3
3


 32n 3
20n 

 T 
 12n 2 
3 
 3
Table 1 Comparing computational times of finding inverse of a matrix using
LU decomposition and Gaussian elimination.
n
10
100
1000
10000
CT|inverse GE / CT|inverse LU
3.28
25.83
250.8
2501
http://numericalmethods.eng.usf.edu
Method: [A] Decompose to [L] and [U]
1
A  LU    21
 31
0
1
 32
0 u11
0  0
1  0
u12
u 22
0
u13 
u 23 
u 33 
[U] is the same as the coefficient matrix at the end of the forward
elimination step.
[L] is obtained using the multipliers that were used in the forward
elimination process
http://numericalmethods.eng.usf.edu
Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination
 25 5 1
 64 8 1


144 12 1
Step 1:
5
1 
 25
64
 2.56; Row2  Row12.56   0  4.8  1.56
25
144 12
1 
5
1 
25
144
 5.76; Row3  Row15.76   0  4.8  1.56
25
 0  16.8  4.76
http://numericalmethods.eng.usf.edu
Finding the [U] Matrix
5
1 
25
Matrix after Step 1:  0  4.8  1.56


 0  16.8  4.76
1 
25 5
 16.8
 3.5; Row3  Row23.5   0  4.8  1.56
Step 2:
 4.8
 0
0
0.7 
1 
25 5
U    0  4.8  1.56
 0
0
0.7 
http://numericalmethods.eng.usf.edu
Finding the [L] matrix
1

 21
 31
0
1
 32
0
0
1
Using the multipliers used during the Forward Elimination Procedure
a
64
 21  21 
 2.56
5 1
From the first step  25
a11 25
of forward
elimination
 64 8 1


144 12 1
 31 
a31 144

 5.76
a11
25
http://numericalmethods.eng.usf.edu
Finding the [L] Matrix
From the second
step of forward
elimination
5
1 
25
 0  4.8  1.56


 0  16.8  4.76
 32 
a32  16.8

 3.5
a22
 4.8
0 0
 1
L  2.56 1 0
5.76 3.5 1
http://numericalmethods.eng.usf.edu
Does [L][U] = [A]?
0 0 25 5
1 
 1
LU   2.56 1 0  0  4.8  1.56 
5.76 3.5 1  0
0
0.7 
?
http://numericalmethods.eng.usf.edu
Example: Production Optimization
To find the number of toys a company should manufacture per day to
optimally use their injection-molding machine and the assembly line,
one needs to solve the following set of equations. The unknowns are
the number of toys for boys, x1, number of toys for girls, x2, and the
number of unisexual toys, x3.
0.3333 0.1667 0.6667  x1   756 
0.1667 0.6667 0.3333  x   1260

 2  

 1.05  1.00 0.00   x3   0 
Find the values of x1, x2,and x3 using LU Decomposition.
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Use Forward Elimination to find the [U] matrix
Step 1
 0.3333 0.1667 0.6667
0.1667 0.6667 0.3333


 1.05
 1.00
0.00 
0.6667 
0.3333 0.1667
0.1667
 0.50015; Row2  Row10.50015   0
0.58332  0.00015002
0.3333
 1.05  1.00

0.00
0.6667 
0.3333 0.1667
1.05
 3.1503; Row3  Row13.1503   0
0.58332  0.00015002
0.3333
 0
 1.5252  2.1003 
http://numericalmethods.eng.usf.edu
Example: Production Optimization
This is the matrix
after the 1st step:
0.6667 
0.3333 0.1667
 0

0
.
58332

0
.
00015002


 0
 1.5252
 2.1003 
Step 2
0.6667 
0.3333 0.1667
 1.5252
 2.6146; Row3  Row2 2.6146   0
0.58332  0.00015002
0.58332
 0
0
 2.1007 
0.6667 
0.3333 0.1667
U    0 0.58332  0.00015002
 0
0
 2.1007 
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Use the multipliers from Forward Elimination
1

 21
 31
0
1
 32
0
0
1
From the 1st step of forward elimination
 0.3333 0.1667 0.6667
0.1667 0.6667 0.3333


 1.05
 1.00
0.00 
 21 
a 21 0.1667

 0.50015
a11 0.3333
a 31
1.05
 31 

 3.1503
a11 0.3333
http://numericalmethods.eng.usf.edu
Example: Production Optimization
From the 2nd step of forward elimination
0.6667 
0.3333 0.1667
 0

0
.
5833

0
.
0002


 0
 1.52516  2.1003
 32 
a 32  1.5252

 2.6146
a 22 0.58332
0
 1
L  0.50015 1
 3.1503  2.6146
0
0
1
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Does [L][U] = [A] ?
0
0 0.3333 0.1667
0.6667 
 1
LU   0.50015 1 0  0 0.58332  0.00015002  ?
 3.1503  2.6146 1  0
0
 2.1007 
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Set [L][Z] = [C]
Solve for [Z]
0
0  z1   756 
 1
0.50015
  z   1260
1
0

 2  

 3.1503  2.6146 1  z3   0 
z1  756
0.50015z1  z2  1260
3.1503z1   2.6146  z3  0
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Solve for [Z]
z1  756
z2  1260 0.50015z1
 1260 0.50015 756
 881.89
 z1   756 
Z   z 2    881.89 
 z 3   75.864
z3  0  3.1503z1   2.6146z2
 0  3.1503 756  2.6146 881.89
 75.864
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Set [U][X] = [Z]
0.6667 
0.3333 0.1667
 0

0
.
58332

0
.
00015002


 0
0
 2.1007 
 x1 
x  
 2
 x3 
 756 
 881.89 


 75.864
Solve for [X]
The 3 equations become
0.3333x1  0.1667x2  0.6667x3  756
0.5833x2   0.0002x3  881.89
 2.1007x3  75.864
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Solve for [X]
 75.864
 2.1007
 36.113
x3 
881.89   0.00015002x3
0.58332
881.89   0.00015002 36.113

0.58332
 1511.8
x2 
http://numericalmethods.eng.usf.edu
Example: Production Optimization
Solve for [X] cont.
756 0.1667x2  0.6667x3
0.3333
756 0.16671511.8  0.6667 36.113

0.3333
 1439.8
x1 
 x1  1439.8
 x   1511.9 
 2 

 x3  36.113
http://numericalmethods.eng.usf.edu
Example: Production Optimization
The solution
vector is
 x1  1439.8
 x   1511.9 
 2 

 x3  36.113
1440 toys for boys should be produced
1512 toys for girls should be produced
36 unisexual toys should be produced
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
The inverse [B] of a square matrix [A] is defined as
[A][B] = [I] = [B][A]
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
How can LU Decomposition be used to find the inverse?
Assume the first column of [B] to be [b11 b12 … bn1]T
Using this and the definition of matrix multiplication
First column of [B]
b11  1
b  0
A  21    
    
   
bn1  0
Second column of [B]
 b12  0
 b  1
A  22    
    
   
bn 2  0
The remaining columns in [B] can be found in the same manner
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Find the inverse of a square matrix [A]
 25 5 1
A   64 8 1
144 12 1
Using the decomposition procedure, the [L] and [U] matrices are found to be
1 
 1 0 0 25 5
A  LU   2.56 1 0  0  4.8  1.56
5.76 3.5 1  0
0
0.7 
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving for the each column of [B] requires two steps
1) Solve [L] [Z] = [C] for [Z]
2) Solve [U] [X] = [Z] for [X]
Step 1:
0 0  z1  1
 1
LZ   C   2.56 1 0  z2   0
5.76 3.5 1  z3  0
This generates the equations:
z1  1
2.56z1  z2  0
5.76z1  3.5z2  z3  0
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving for [Z]
z1  1
z 2  0  2.56z1
 0  2.561
 2.56
z3  0  5.76z1  3.5 z 2
 z1   1 
Z    z2    2.56
 z3   3.2 
 0  5.761  3.5 2.56
 3.2
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving [U][X] = [Z] for [X]
5
1  b11   1 
25
 0  4.8  1.56 b    2.56

  21  

 0
0
0.7  b31   3.2 
25b11  5b21  b31  1
 4.8b21  1.56b31  2.56
0.7b31  3.2
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Using Backward Substitution
3.2
 4.571
0.7
2.56  1.560b31
b21 
4.8
2.56  1.5604.571

 0.9524
4 . 8
1  5b21  b31
b11 
25
1  5 0.9524  4.571

 0.04762
25
b31 
So the first column of
the inverse of [A] is:
b11   0.04762
b    0.9524
 21  

b31   4.571 
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Repeating for the second and third columns of the inverse
Second Column
 25 5 1 b12  0
 64 8 1 b   1

  22   
144 12 1 b32  0
b12   0.08333
b    1.417 
 22  

b32    5.000 
Third Column
 25 5 1  b13  0
 64 8 1 b   0

  23   
144 12 1 b33  1
b13   0.03571
b    0.4643
 23  

b33   1.429 
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
The inverse of [A] is
 0.04762  0.08333 0.03571
A1   0.9524 1.417  0.4643
 4.571
 5.000
1.429 
To check your work do the following operation
[A][A]-1 = [I] = [A]-1[A]
http://numericalmethods.eng.usf.edu
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/lu_decomp
osition.html
THE END
http://numericalmethods.eng.usf.edu