Transcript Linear Algebra

資訊科學數學13 :
Solutions of Linear Systems
陳光琦助理教授 (Kuang-Chi Chen)
[email protected]
1
Linear Equations and Matrices
Solutions of Linear Systems of Equations
2
Solutions of Linear Systems of Equations
1.6 Solutions of Linear Systems of Equations
x1 
2 x4  4
x2 
x4  5
x3  3 x 4  6

4
1 0 0 2
0 1 0  1  5


6 
0 0 1 3
3
Row Echelon Form
• Definition – Row echelon form (r.e.f.)
An mn matrix A is said to be in row echelon form if
(a) All zero rows, if there are any, appear at the bottom
of the matrix
(b) The first nonzero entry from the left of a nonzero
row is a 1; a leading one of the row
(c) For each nonzero row, the leading one appears to the
right and below any leading one’s in preceding rows
4
Reduced Row Echelon Form
• Definition – Reduced row echelon form
An mn matrix A is said to be in reduced row echelon
form if
(a) A is in row echelon form
(b) If a column contains a leading one, then all other
entries in that column are zero
(列梯形式; 簡化之列梯形式)
5
Example 1 - in row echelon form
• E.g. 1
1
0
A
0

0
0
1
0
0
1
0

C  0

0
0
5 0 2 2
0
0
1
0
0
0 
0

1
1 0 3
0 0 1
4
7
0 0 0
0 0 0
0
0
0
0

B  0

0
0
4
8 
2 

0
0 
0 1 3 5
0 0 0 1
0 0 0 0
0 0 0 0
0 0 0 0
9
2 3 
1 2

0 1
0 0 
7
6
Example 2 – reduced row echelon form
• E.g. 2
1
0
D
0

0
0
1
0
0
0
0
1
0
0
0 
0

1
1
0

E  0

0
0
0 0 0 2
1 0 0
0 0 1
4
7
0 0 0
0 0 0
0
0
4
8 
2 

0
0 
1 2 0 0 1 
F  0 0 1 2 3 
0 0 0 0 0 
7
E.g. 2 – not reduced row echelon form
• E.g. 2 1 2 0
,
4
G  0 0 0 0 
0 0 1 3
1 0 3 4 
H  0 2 2 5 
0 0 1 2 
1
0
I 
0

0
1
0
J 
0

0
,
,
0 3
1 2
1 2
0 0
4

5
2

0
2 3
1 2
0 1
0 0
4

5
2

0
Nonzero element above leading 1 in row 2
8
Three Basic Types of Elementary
Row Operations
• Type 1 – Interchange
row i and row j are interchanged
• Type 2 – Multiply
row i = row i times c
• Type 3 – Add
Add d times row r of A to row s of A
row s = row s + d  row r
9
Example 3
• E.g. 3
0 0 1 2 
A   2 3 0  2


 3 3 6  9
 E2
0 0 1 2 


C  2 3 0 2


1 1 2  3
 3 3 6  9
B  2 3 0  2


0 0 1 2 
E1
⇒
E3
0 0 1 2
D   2 3 0  2


 1  3 6  5
10
Row Equivalence
• Definition – Row Equivalence
An mn matrix A is said to be row equivalence
to an mn matrix B if B can be obtained by
applying a finite sequence of elementary row
operations to the matrix A .
11
• E.g. 4
Example 4
 1 2 4 3
A  2 1 3 2


1  2 2 3
E3
⇒
1 2 4 3
B  4  3 7 8


1  2 2 3
E1
1 2 4 3
C  1  2 2 3


4  3 7 8
E2
⇒
 2 4 8 6
D  1  2 2 3


4  3 7 8
12
Theorem 1.5
• Theorem 1.5
Every mn matrix is row equivalent to a
matrix in row echelon form .
13
E.g. 5 - Procedure of Row Echelon Form
• E.g. 5
Step 1 – Find the pivotal column
0
0
A
2

2
2 3 4
0 2
3
2 5 2
0 6 9
1
4

4

7
Pivot column
Step 2 – Identify the pivot in the pivotal column
Pivot
14
(cont’d)
• E.g. 5
Step 3 – Interchange if necessary so that the pivot is in
the 1st row
2 2  5 2 4
0 0 2
3 4

A1  
0 2 3  4 1
pivot


2 0  6 9 7
Step 4 – Multiply so that the pivot equals to 1
1
0
A2  
0

2
1  25
0 2
2
4

2 3  4 1

0  6 9 7
1
3
15
(cont’d)
• E.g. 5
Step 5 – Make all entries in the pivot column, except
the entry where the pivot was located, equal to zero
1 1  25 1
0 0
2
3
A3  
0 2
3 4

0  2  1 7
2
4

1

3
16
(cont’d)
• E.g. 5
Step 6 – Ignore the first row and repeat
1 1  25 1
2
3
0 0
B
0 2
3 4

0  2  1 7
⇒ … … …
1 1
2
0 2
4
⇒ B1  
0 0
1


0  2
3
⇒
1
0
H 
0

0
 25 1 2
3  4 1
2 3 4

1 7 3
1  25
1 23
0
1
0
0
1 2
 2 12 

3
2
2

0 0
17
Example 6
• Example 6
2 3
A 

3
1


 1  2
A2  

2 3 

2 3 
A1  

1

2



1  2
A3  

0
7


18
Remark
• Remark
- There may be more than one matrix in row
echelon form that is row equivalent to a given
matrix.
- A matrix in row echelon form (r.e.f.) that is
row equivalent to A is called
“a row echelon form of A”.
19
Theorem 1.6
• Theorem 1.6
- Every mn matrix is row equivalent to a
unique matrix in reduced row echelon form.
20
Example 7 – r.e.f. to reduced r.e.f.
• E.g. 7
1
0
H 
0

0
1 
1 23
1
0
J2  
0

0
1 0
5
2
0
1
0
0
1 2
 2 12 

3
2
2

0 0
1
0
J1  
0

0
1  25 1 2 
1 0  174  25 

3
0 1
2
2

0 0 0 0
7
 25 

2

0
1
0
K
0

0
0 0
19
4
1 0  174
0 1
3
2
0 0
0
9
1 0  174
0 1
3
2
0 0
0

 25 

2

0
19
2
21
Theorem 1.7
• Theorem 1.7
Let Ax = b and Cx = d be two linear systems
each of m equations in n unknowns. If the
augmented matrices [A|b] and [C|d] of these
systems are row equivalent, then both linear
systems have the same solutions.
22
Corollary 1.1
• Corollary 1.1
If A and C are row equivalent mn matrices,
then the linear system Ax = 0 and Cx = 0 have
exactly the same solutions.
23
Gauss-Jordan Reduction Procedure
• The Gauss-Jordan reduction procedure
Step 1. Form the augmented matrix [A|b]
Step 2. Obtain the reduced row echelon form
[C|d] of the augmented matrix [A|b] by using
elementary row operations
Step 3. For each nonzero row of [C|d], solve the
corresponding equation.
(augmented matrix 擴增矩陣)
24
Gauss Elimination Procedure
• The Gauss elimination procedure
Step 1. Form the augmented matrix [A|b]
Step 2. Obtain a row echelon form [C|d] of
the augmented matrix [A|b] by using
elementary row operations
Step 3. Solving the linear system corresponding to
[C|d], by back substitution (後代入法).
25
• E.g. 8
Example 8
Solve the linear system by Gauss-Jordan reduction
x  2 y  3z  9
2x  y  z  8
3x
 z3
- Step 1
3 9
1 2
2  1 1 8


 3 0  1 3
26
(cont’d)
• E.g. 8 - Solve the linear system by Gauss-Jordan reduction
- Step 2
1 2 3 9 
2  1 1 8


 3 0  1 3
1 2 3 9 
0 1 1 2 


0 0 1 3
9 
1 2 3
0  5  5  10


0  6  10  24
9 
1 2 3
0 1 1
2 


0  6  10  24
1 2 3 9 
0 1 0  1


0 0 1 3 
1 2 0 0 
0 1 0  1


0 0 1 3 
1 2 3 9 
0 1 1
2 


0 0  4  12
1 0 0 2 
0 1 0  1


0 0 1 3 
27
(cont’d)
• E.g. 8 - Solve the linear system by Gauss-Jordan reduction
- Step 3
x
y
=2
= -1
z=3
28
Example 9
• Example 9
- Solve the linear system by Gauss-Jordan reduction
x + y + 2z – 5w = 3
2x + 5y – z – 9w = -3
2x + y – z + 3w = -11
x – 3y + 2z + 7w = -5
29
(cont’d)
• Example 9
- Step 1
- Step 2
2 5 3 
1 1
2 5  1  9  3 


2 1  1 3  11


1

3
2
7

5


1
0

0

0
0
1
0
0
0 2  5
0 3 2 

1 2 3 

0 0
0
30
(cont’d)
• E.g. 9 - Step 3
x
 2 w  5
y  3w  2
z  2w  3
x  5  2r
y  2  3r
z  3  2r
w r
x  5  2 w
y  2  3w
z  3  2w
leading variables
a free variable
31
Example 10
• Example 10
- Solve the linear system by Gauss-Jordan reduction
x1 + 2x2
– 3x4 + x5
=2
x1 + 2x2 + x3 – 3x4 + x5 + 2x6 = 3
x1 + 2x2
– 3x4 + 2x5 + x6 = 4
3x1 + 6x2 + x3 – 9x4 + 4x5 + 3x6 = 9
32
(cont’d)
• Example 10
- Step 1
3
3
3
9
1
1

1

3
2
2
2
6
0
1
0
1
1
0

0

0
2
0
0
0
0 3
1 0
0 0
0 0
1
1
2
4
0
2
1
3
2
3

4

9
- Step 2
0 1
0 2
1 1
0 0
0
1

2

0
33
• Example 10 - Step 3
x1  2 x2
 3x 4
x3
(cont’d)
 x6  0
x1  x6  3x4  2 x2
 2 x6  1
x3  1  2 x6
x5  x6  2
x5  2  x 6
x1  r  3s  2t
x2  t
leading variables
x3  1  2r
x4  s
free variables
x5  2  r
x6  r
34
Example 11
• Example 11
- Solve the linear system by Gauss elimination
x + 2y + 3z = 9
2x – y + z = 8
3x
– z =3
35
(cont’d)
• Example 11
- Step 1
- Step 2
1 2 3 9


 2 1 1 8 
 3 0 1 3
1 2 3 9 


0
1
1
2


0 0 1 3 
36
(cont’d)
• Example 11 - Step 3
x  2 y  3z  9
yz 2
z 3
- By back substitution
x2
y  1
z 3
37
Example 12
• Example 12
- Solve the linear system by Gauss elimination
x + 2y + 3z + 4w = 5
x + 3y + 5z + 7w = 11
x
– z – w = -6
38
• Example 12
- Step 1
- Step 2
(cont’d)
1 2 3 4 5 


1 3 5 7 11 
1 0 1 2 6 
1 0 1 2 0 


0 1 2 3 0 
0 0 0 0 1 
- Step 3 ⇒ 0x + 0y + 0z + 0w = 1 ⇒ No solutions !!
39
Consistent and Inconsistent
• Consistent and inconsistent
- Consistent: Linear systems with at least one
solution
- Inconsistent: Linear systems with no solutions
40
Homogeneous Systems
• A system of linear equations is said to be
homogeneous if all the constant terms are zeros.
a11x1 + a12x2 + … + a1nxn = 0
a21x1 + a22x2 + … + a2nxn = 0
…
am1x1 + am2x2 + … + amnxn = 0
⇒ Ax = 0
Thus, a homogeneous system always has the solution
x1 = x2 = … = xn = 0 → the trivial solution
41
Example 13
• Example 13
x  2 y  3z  0
x  3y  2z  0
2x  y  2z  0
 1 2 3 0


 1 3 2 0 
 2 1 2 0 
1 0 0 0 


0
1
0
0


0 0 1 0 
x y z 0
42
Example 14
• Example 14
x y zw0
xw0
x  2y  z  0
1 0 0 1 0 


0
1
0

1
0


0 0 1 1 0 
1 1 1 1 0 


1
0
0
1
0


1 2 1 0 0 
x  r
yr
z  r
wr
43
Theorem 1.8
• Theorem 1.8
A homogeneous system of m equations in n
unknowns has a non-trivial solution if m < n,
that is, if the number of unknowns exceeds the
number of equations.
namely, a homogeneous system has more
variables than equations has many solutions.
(a homogeneous system齊次系統; non-trivial solution 非零解)
44
Example 15 - A Homogeneous System
• E.g. 15
x  y  2 z  5w  3
2 x  5 y  z  9 w  3
2 x  y  z  3w  11
 2 w  5
x
y
 3w  2
z  2w  3
x  3 y  2 z  7 w  5
x  5  2r
y  2  3r
z  3  2r
w r
45
(cont’d)
• If let
x
 y
x 
z
 
 w
 5  2r   5  2r 
 2  3r   2   3r 
  

x
 3  2r   3   2r 

   

 r  0  r 
 5
 2r 
2
 3r 

x p    and xh  
3
 2r 
 


0
 r 
46
A Homogeneous System Example
x = xp + xh
xp is a particular solution to the given system
Axp = b , where b = [3 -3 -11 -5]T
xh is a solution to the associated
homogeneous system Axh = 0 .
47
Polynomial Interpolation
• Polynomial Interpolation
- The general form
y = an – 1xn – 1 + an – 2xn – 2 + … + a1x + a0
E.g. n = 3, y = a2x2 + a1x + a0
Given three distinct points (x1 , y1), (x2 , y2), (x3 , y3),
we have
y1 = a2x12 + a1x1 + a0
y2 = a2x22 + a1x2 + a0
y3 = a2x32 + a1x3 + a0
48
Example 16
• Example 16 - Find the quadratic polynomial that
interpolates the points (1, 3), (2, 4), (3, 7)
a2  a1  a0  3
4a2  2a1  a0  4
a 2  1 , a 1  2 , a 0  4
9a2  3a1  a0  7
y  x  2x  4
2
49
Example 17 – Temperature Distribution
T1 = (260 – 100 + T2 + T3 )/4
T2 = (T1 + 100 + 40 + T4 )/4
T3 = (60 + T1 + T4 + 0)/4
T4 = (T2 + T3 + 40 + 0)/4
⇒ A
or
or
or
or
4T1 – T2 – T3 = 160
-T1 + 4T2 – T4 = 140
-T1 + 4T3 – T4 = 60
-T2 – T3 + 4T4 = 40
 4 1 1 0 160 


1 4 0 1 140 

b 
 1 0 4 1 60 


 0 1 1 4 40 
⇒ T1 = 65, T2 = 60, T3 = 40, T4 = 35 .
50
Linear Equations and Matrices
The Inverse of A Matrix
51
The Inverse of A Matrix
• 1.7 The inverse of a matrix
Definition
- An nn matrix A is called nonsingular (or
invertible 可逆的) if there exists an nn
matrix B such that AB = BA = In .
- The matrix B is called the inverse of A
- If there exists no such matrix B, then A is
called singular (or noninvertible)
- A is also an inverse of B
52
Example 1
• Example 1
2 3
A 

2 2
 1 3/ 2
B

1

1


⇒ AB = BA = I2
- B is an inverse of A and A is nonsingular.
53
Theorem 1.9
• Theorem 1.9
An inverse of a matrix, if exists, is unique.
(proof)
Let B and C be inverses of A.
Then AB = BA = In, and AC = CA = In.
Thus, C(AB) = CIn
(CA)B = C
InB = C , i.e., B = C .
54
Example 2 - Find the Inverse
For the matrix A, find the inverse
If exists, let the inverse A-1 be
such that
1 2
A 

3
4


a b 
A 

c
d


1
1 2 a b 
1 0
AA  
 I2  




3
4
c
d
0
1

 



1
55
(cont’d)
 a  2c b  2d  1 0
3a  4c 3b  4d   0 1

 



a  2c  1
3a  4c  0
and
b  2d  0
3b  4d  1
1 
a b   2
1


A
 c d  3/ 2 1/ 2

 

1  1 2 1 0
 2
3/ 2 1/ 2 3 4  0 1

 
 

56
• Example 3
1 2
A 

2
4



Example 3
a b 
A 

c
d


1
1 2 a b 
1 0
AA  
 I2  




2
4
c
d
0
1

 



1

 a  2c b  2d  1 0
2a  4c 2b  4d   0 1

 


a  2c  1 and b  2d  0
⇒ No solution; singular
2b  4 d  1
2a  4c  0
57
Theorem 1.10
• Thm. 1.10 - Properties of an inverse
- If A is nonsingular, then A-1 is nonsingular
and (A-1)-1 = A ;
- If A and B are nonsingular matrices, then AB
is nonsingular and (AB)-1 = B-1 A-1 ;
- If A is a nonsingular matrix, then (AT)-1 = (A-1)T .
58
Example 4
• Example 4
1 2
A 

3
4


A
1
 2
 3
 2
1 
 12 
3

2

2 
1 T
(A )  
1
1

2

3

2
1
3



2 
T 1
T
⇒ A 
and ( A )  

1
1

2

2 4
59
Corollary1.2
• Corollary 1.2
- If A1 , A2 , … , Ar are nn nonsingular
matrices, then (A1 A2 … Ar) is nonsingular
and (A1 A2 … Ar)-1 = Ar-1 … A2-1 A1-1 .
60
Theorem 1.11
• Theorem 1.11
Suppose that A, B are nn matrices,
- If AB = In , then BA = In ;
- If BA = In , then AB = In .
61
The Way to Find A-1
• A practical method for finding A-1
Step 1. Form the 22n matrix [A | In] obtained by
adjoining the identity matrix In to the given matrix A
Step 2. Compute the reduced row echelon form of the
matrix obtained in Step 1 by using elementary row
operations
Step 3. Suppose that Step 2 has produced the matrix
[C | D] in reduced row echelon form:
• If C = In , then D = A-1 ;
• If C ≠ In , then C has a row of zeros and the matrix A
is singular .
62
Example 5 – Find the Inverse
• E.g. 5 – Find the inverse
1 1 1 1 0 0 
A I 3   0 2 3 0 1 0
5 5 1 0 0 1
1 1 1
A  0 2 3


5 5 1
1 1 1 1 0 0 
0 2 3 0 1 0


5 5 1 0 0 1
1 1 1 1 0 0 
 0 2 3 0 1 0


0 0  4  5 0 1
1 1 0  14 0 14 
0 1 0  15 1 3 
8
2
8 

0 0 1 45 0  14 
1 1 1 1 0 0 
 0 1 3 0 1 0
2
2


0 0  4  5 0 1
1 0 0 138  12  81 
0 1 0  15 1 3 
8
2
8 

0 0 1 54 0  14 
1 1 1 1 0 0 
0 1 3 0 1 0 
2
2


5
1
0 0 1 4 0  4 
 138  12  81 
A1   158 12 83 


 45 0  14 
63
Example 6 – Find the Inverse
• E.g. 6 - Find the inverse
1 2  3
A  1  2 1 


5  2  3
1 2  3 1 0 0
1  2 1 0 1 0


5  2  3 0 0 1
1 2  3 1 0 0
 0  4 4  1 1 0


0 0 0  2  3 1
1 2  3 1 0 0
A I 3   1  2 1 0 1 0
5  2  3 0 0 1
1 2  3 1 0 0 
0  4 4  1 1 0


5  2  3 0 0 1
1 2  3 1 0 0 
 0  4 4  1 1 0


0  12 12  5 0 1
The left-half matrix cannot have a
one in the (3, 3) location, the
reduced echelon form cannot be I3.
Thus A-1 does not exist.
64
Theorem 1.12 & 1.13
• Theorem 1.12
An nn matrix is nonsingular iff it is row
equivalence to In .
• Theorem 1.13
If A is an nn matrix, the homogeneous
system Ax = 0 has a nontrivial solution iff
A is singular.
65
Proof of Theorem 1.13
• Proof of Theorem 1.13
Suppose that A is nonsingular, then A-1 exists and
A-1(Ax) = A-1 0
(A-1A)x = 0
In x = 0
x = 0 ⇒ Ax = 0 has a trivial solution
(contradiction to a non-trivial solution, hence A must
be singular)
66
Example 8
• Example 8
Consider the homogeneous system Ax = 0, where A is
the matrix (A is nonsingular)
1 1 1
A  0 2 3


5 5 1
1 1 1 0 
0 2 3 0


5 5 1 0
Gauss-Jordan reduction
1 0 0 0 
0 1 0 0


0 0 1 0
The trivial solution x = 0
67
• Example 9
Example 9
- Consider the homogeneous system Ax = 0, where A
is the matrix (A is singular)
1 2  3
A  1  2 1 


5  2  3
1 2  3 0
1  2 1 0


5  2  3 0
1 0  1 0 
0 1  1 0


0 0 0 0
xr
yr
zr
68
Theorem 1.14
• Theorem 1.14
If A is an nn matrix, then A is nonsingular iff
the linear system Ax = b has a unique solution
for every n1 matrix b .
69
Summary
The Symmetry, Singularity,
Inverse of A Matrix
70
Some Special Matrix
4. A square matrix A is said to be antisymmetric if -AT
= A. (i) If A is square, prove that A + AT is
symmetric and A – AT is antisymmetric;
(ii) any square matrix A can be decomposed into the
sum of a symmetric matrix B and an antisymmetric
matrix C: A = B + C .
5. Given two symmetric matrices of the same size, A
and B, then a necessary and sufficient condition for
the product AB to be symmetric is that AB = BA.
71
Some Special Matrix
1. A square matrix A is said to be normal if AAT = ATA.
All symmetric matrices are normal ;
2. A square matrix A is said to be idempotent if A2 = A.
If A is idempotent then AT is also idempotent ;
3. A square matrix A is said to nilpotent if there is a
positive integer p such that Ap = O. The least integer
such that Ap = O is called the degree of nilpotency of
the matrix. If A is nilpotent, then AT is also nilpotent
with the same degree of nilpotency.
72
List of Nonsingular Equivalences
• Nonsingular equivalences
1. A is nonsingular ;
2. x = 0 is the only solution to Ax = 0 ;
3. A is row equivalence to In ;
4.The linear system Ax = b has a unique
solution for every n1matrix b .
73
Properties of Matrix Inverse
• Properties of Matrix Inverse
1. (A-1)-1 = A ;
2. (cA)-1 = (1/c)A-1 , where c is a nonzero scalar;
3. (AB)-1 = B-1A-1 ;
4. (An)-1 = (A-1)n ;
5. (AT)-1 = (A-1)T , where T : transpose.
74
Conditions of Matrix Inverse
• A matrix has no inverse, if
(i) two rows are equal;
(ii) two columns are equal; (Use the transpose)
(iii) it has a column of zeros.
75
The Inverse of 22 Matrix
•
a b 
d b 
1
-1
If A = 
, show that A =
.



(ad  bc) c a 
c d 
Note: The cancellation law doesn’t hold.
That is, AB = AC doesn’t imply that B = C .
Also, AB = O doesn’t imply that A = O or B = O.
However, if A is an invertible matrix, then
if AB = AC , then B = C ;
if AB = 0, then B = 0 .
76