LU Decomposition - MATH FOR COLLEGE
Download
Report
Transcript LU Decomposition - MATH FOR COLLEGE
LU Decomposition
Major: All Engineering Majors
Authors: Autar Kaw
http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
7/17/2015
http://numericalmethods.eng.usf.edu
1
LU Decomposition
http://numericalmethods.eng.usf.edu
LU Decomposition
LU Decomposition is another method to solve a set of
simultaneous linear equations
Which is better, Gauss Elimination or LU Decomposition?
To answer this, a closer look at LU decomposition is
needed.
http://numericalmethods.eng.usf.edu
LU Decomposition
Method
For most non-singular matrix [A] that one could conduct Naïve Gauss
Elimination forward elimination steps, one can always write it as
[A] = [L][U]
where
[L] = lower triangular matrix
[U] = upper triangular matrix
http://numericalmethods.eng.usf.edu
How does LU Decomposition work?
If solving a set of linear equations
If [A] = [L][U] then
Multiply by
Which gives
Remember [L]-1[L] = [I] which leads to
Now, if [I][U] = [U] then
Now, let
Which ends with
and
[A][X] = [C]
[L][U][X] = [C]
[L]-1
[L]-1[L][U][X] = [L]-1[C]
[I][U][X] = [L]-1[C]
[U][X] = [L]-1[C]
[L]-1[C]=[Z]
[L][Z] = [C] (1)
[U][X] = [Z] (2)
http://numericalmethods.eng.usf.edu
LU Decomposition
How can this be used?
Given [A][X] = [C]
1. Decompose [A] into [L] and [U]
2. Solve [L][Z] = [C] for [Z]
3. Solve [U][X] = [Z] for [X]
http://numericalmethods.eng.usf.edu
Is LU Decomposition better than
Gaussian Elimination?
Solve [A][X] = [B]
T = clock cycle time and nxn = size of the matrix
Forward Elimination
8n 3
32n
CT | FE T
8n 2
3
3
Back Substitution
CT | BS T 4n 2 12n
Decomposition to LU
8n 3
20n
CT | DE T
4n 2
3
3
Forward Substitution
CT | FS T 4n 2 4n
Back Substitution
CT | BS T 4n 2 12n
http://numericalmethods.eng.usf.edu
Is LU Decomposition better than
Gaussian Elimination?
To solve [A][X] = [B]
Time taken by methods
Gaussian Elimination
8n 3
4n
T
12 n 2
3
3
LU Decomposition
8n 3
4n
T
12 n 2
3
3
T = clock cycle time and nxn = size of the matrix
So both methods are equally efficient.
http://numericalmethods.eng.usf.edu
To find inverse of [A]
Time taken by Gaussian Elimination
Time taken by LU Decomposition
nCT |FE CT |BS
CT |LU n CT |FS n CT |BS
8n 4
4n 2
3
T
12n
3
3
32n 3
20n
T
12n 2
3
3
http://numericalmethods.eng.usf.edu
To find inverse of [A]
Time taken by Gaussian Elimination
8n 4
4n 2
3
T
12n
3
3
Time taken by LU Decomposition
32 n 3
20 n
T
12 n 2
3
3
Table 1 Comparing computational times of finding inverse of a matrix using
LU decomposition and Gaussian elimination.
n
10
100
CT|inverse GE / CT|inverse LU
3.288 25.84
1000
10000
250.8
2501
For large n, CT|inverse GE / CT|inverse LU ≈ n/4
http://numericalmethods.eng.usf.edu
Method: [A] Decomposes to [L] and [U]
1
A LU 21
31
0
1
32
0 u11
0 0
1 0
u12
u 22
0
u13
u 23
u 33
[U] is the same as the coefficient matrix at the end of the forward elimination step.
[L] is obtained using the multipliers that were used in the forward elimination process
http://numericalmethods.eng.usf.edu
Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination
25 5 1
64 8 1
144 12 1
Step 1:
5
1
25
64
2.56; Row2 Row12.56 0 4.8 1.56
25
144 12
1
5
1
25
144
5.76; Row3 Row15.76 0 4.8 1.56
25
0 16.8 4.76
http://numericalmethods.eng.usf.edu
Finding the [U] Matrix
5
1
25
Matrix after Step 1: 0 4.8 1.56
0 16.8 4.76
1
25 5
16.8
3.5; Row3 Row23.5 0 4.8 1.56
Step 2:
4.8
0
0
0.7
1
25 5
U 0 4.8 1.56
0
0
0.7
http://numericalmethods.eng.usf.edu
Finding the [L] matrix
1
21
31
0
1
32
0
0
1
Using the multipliers used during the Forward Elimination Procedure
a
64
21 21
2.56
5 1
From the first step 25
a11 25
of forward
elimination
64 8 1
144 12 1
31
a31 144
5.76
a11
25
http://numericalmethods.eng.usf.edu
Finding the [L] Matrix
From the second
step of forward
elimination
5
1
25
0 4.8 1.56
0 16.8 4.76
32
a32 16.8
3.5
a22
4.8
0 0
1
L 2.56 1 0
5.76 3.5 1
http://numericalmethods.eng.usf.edu
Does [L][U] = [A]?
0 0 25 5
1
1
LU 2.56 1 0 0 4.8 1.56
5.76 3.5 1 0
0
0.7
?
http://numericalmethods.eng.usf.edu
Using LU Decomposition to solve SLEs
Solve the following set of
linear equations using LU
Decomposition
25 5 1 x1 106.8
64 8 1 x 177.2
2
144 12 1 x3 279.2
Using the procedure for finding the [L] and [U] matrices
0 0 25
5
1
1
A LU 2.56 1 0 0 4.8 1.56
5.76 3.5 1 0
0
0.7
http://numericalmethods.eng.usf.edu
Example
Set [L][Z] = [C]
Solve for [Z]
0 0 z1 106.8
1
2.56 1 0 z 177.2
2
5.76 3.5 1 z 3 279.2
z1 10
2.56z1 z 2 177.2
5.76z1 3.5 z 2 z 3 279.2
http://numericalmethods.eng.usf.edu
Example
Complete the forward substitution to solve for [Z]
z1 106.8
z 2 177.2 2.56z1
177.2 2.56106.8
96.2
z3 279.2 5.76z1 3.5 z 2
279.2 5.76106.8 3.5 96.21
z1 106.8
Z z2 96.21
z3 0.735
0.735
http://numericalmethods.eng.usf.edu
Example
Set [U][X] = [Z]
Solve for [X]
1 x1 106.8
25 5
0 4.8 1.56 x 96.21
2
0
0
0.7 x3 0.735
The 3 equations become
25a1 5a2 a3 106.8
4.8a2 1.56a3 96.21
0.7a3 0.735
http://numericalmethods.eng.usf.edu
Example
From the
3rd
equation
Substituting in a3 and using the
second equation
0.7 a3 0.735
4.8a2 1.56a3 96.21
0.735
a3
0.7
a3 1.050
96.21 1.56a3
a2
4.8
96.21 1.561.050
a2
4.8
a2 19.70
http://numericalmethods.eng.usf.edu
Example
Substituting in a3 and a2 using
the first equation
25a1 5a2 a3 106.8
106.8 5a2 a3
25
106.8 519.70 1.050
25
0.2900
a1
Hence the Solution Vector is:
a1 0.2900
a 19.70
2
a3 1.050
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
The inverse [B] of a square matrix [A] is defined as
[A][B] = [I] = [B][A]
http://numericalmethods.eng.usf.edu
Finding the inverse of a square matrix
How can LU Decomposition be used to find the inverse?
Assume the first column of [B] to be [b11 b12 … bn1]T
Using this and the definition of matrix multiplication
First column of [B]
b11 1
b 0
A 21
bn1 0
Second column of [B]
b12 0
b 1
A 22
bn 2 0
The remaining columns in [B] can be found in the same manner
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Find the inverse of a square matrix [A]
25 5 1
A 64 8 1
144 12 1
Using the decomposition procedure, the [L] and [U] matrices are found to be
1
1 0 0 25 5
A LU 2.56 1 0 0 4.8 1.56
5.76 3.5 1 0
0
0.7
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving for the each column of [B] requires two steps
1) Solve [L] [Z] = [C] for [Z]
2) Solve [U] [X] = [Z] for [X]
Step 1:
0 0 z1 1
1
LZ C 2.56 1 0 z2 0
5.76 3.5 1 z3 0
This generates the equations:
z1 1
2.56z1 z2 0
5.76z1 3.5z2 z3 0
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving for [Z]
z1 1
z 2 0 2.56z1
0 2.561
2.56
z3 0 5.76z1 3.5 z 2
z1 1
Z z2 2.56
z3 3.2
0 5.761 3.5 2.56
3.2
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Solving [U][X] = [Z] for [X]
5
1 b11 1
25
0 4.8 1.56 b 2.56
21
0
0
0.7 b31 3.2
25b11 5b21 b31 1
4.8b21 1.56b31 2.56
0.7b31 3.2
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Using Backward Substitution
3.2
4.571
0.7
2.56 1.560b31
b21
4.8
2.56 1.5604.571
0.9524
4 . 8
1 5b21 b31
b11
25
1 5 0.9524 4.571
0.04762
25
b31
So the first column of
the inverse of [A] is:
b11 0.04762
b 0.9524
21
b31 4.571
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
Repeating for the second and third columns of the inverse
Second Column
25 5 1 b12 0
64 8 1 b 1
22
144 12 1 b32 0
b12 0.08333
b 1.417
22
b32 5.000
Third Column
25 5 1 b13 0
64 8 1 b 0
23
144 12 1 b33 1
b13 0.03571
b 0.4643
23
b33 1.429
http://numericalmethods.eng.usf.edu
Example: Inverse of a Matrix
The inverse of [A] is
0.04762 0.08333 0.03571
A1 0.9524 1.417 0.4643
4.571
5.000
1.429
To check your work do the following operation
[A][A]-1 = [I] = [A]-1[A]
http://numericalmethods.eng.usf.edu
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/lu_decomp
osition.html
THE END
http://numericalmethods.eng.usf.edu