Chapter 5 Orthogonality
Download
Report
Transcript Chapter 5 Orthogonality
5.1
Orthogonality
Definitions
A set of vectors is called an orthogonal set if all pairs of
distinct vectors in the set are orthogonal.
An orthonormal set is an orthogonal set of unit vectors.
An orthogonal (orthonormal) basis for a subspace W of
n
R is a basis for W that is an orthogonal (orthonormal) set.
An orthogonal matrix is a square matrix whose columns
form an orthonormal set.
Examples
1) Is the following set of vectors orthogonal? orthonormal?
a) 1
2
3 2 1
, 4 , 1
1 2
b) { e1 , e 2 ,..., e n }
2) Find an orthogonal basis and an orthonormal basis
n
for the subspace W of R
x
W y : x y 2 z 0
z
Theorems
All vectors in an orthogonal set are linearly independent.
Let {v1, v2,…, vk } be an orthogonal basis for a subspace
n
W of R and w be any vector in W. Then the unique
scalars c1 ,c2 , …, ck such that w = c1v1 + c2v2 + …+ ckvk
are given by
w vi
ci
for i 1,..., k
vi vi
Proof: To find ci we take the dot product with vi
w vi = (c1v1 + c2v2 + …+ ckvk ) vi
Examples
3) The orthogonal basis for the subspace W in previous
example is 1 1
1 , 1
0 1
Pick a vector in W and express it in terms of the vectors
in the basis.
4) Is the following matrix orthogonal?
3
A
1
2
2
4
1
1
1
2
0
B 1
0
0
0
1
1
0
0
cos
C
sin
sin
cos
If it is orthogonal, find its inverse and its transpose.
Theorems on Orthogonal Matrix
The following statements are equivalent for a matrix A :
1) A is orthogonal
-1
T
2) A = A
n
3) ||Av|| = ||v|| for every v in R
n
4) Av1∙ Av2 = v1∙ v2 for every v1 ,v2 in R
Let A be an orthogonal matrix. Then
1) its rows form an orthonormal set.
-1
2) A is also orthogonal.
3) |det(A)| = 1
4) |λ| = 1 where λ is an eigenvalue of A
5) If A and B are orthogonal matrices, then so is AB
5.2
Orthogonal Complements
and Orthogonal Projections
Orthogonal Complements
Recall: A normal vector n to a plane is orthogonal to
every vector in that plane. If the plane passes through
the origin, then it is a subspace W of R3 .
Also, span(n) is also a subspace of R3
Note that every vector in span(n) is orthogonal to
every vector in subspace W . Then span(n) is called
orthogonal complement of W.
Definition:
A vector v is said to be orthogonal to a subspace W of
n
R if it is orthogonal to all vectors in W.
The set of all vectors that are orthogonal to W is called
the orthogonal complement of W, denoted W ┴ . That is
W perp
W
{v R
n
: v w 0 w W}
http://www.math.tamu.edu/~yvorobet/MATH304-2011C/Lect3-02web.pdf
Example
3
1) Find the orthogonal complements for W of R .
1
a) W span 2
3
b) W plane with direction
c) (subspace
x
W y : x y 2 z 0
z
spanned
by) vectors
1
0
1 and 1
0
1
Theorems
n
Let W be a subspace of R .
n
┴
1) W is a subspace of R .
2) (W ┴)┴ = W
3) W ∩ W ┴ = {0}
4) If W = span(w1,w2,…,wk), then v is in W ┴ iff v∙wi = 0
for all i =1,…,k.
Let A be an m x n matrix. Then
(row(A))┴ = null(A)
and
(col(A))┴ = null(AT)
Proof?
Example
2) Use previous theorem to find the orthogonal complements
3
for W of R .
1
0
a) W plane w ith direction (subspace spann ed by) vectors 1 and 1
0
1
b)
3
2
W subspace spanned by vectors 0 ,
1
4
1
2
2 an d
0
1
3
2
6
2
5
Orthogonal Projections
u v
w 1 proj v u
v 2
u
w2
w1
u v
v
v v
v
w 2 = perp v u u - w 1
v
Let u and v be nonzero vectors.
w1 is called the vector component of u along v
(or projection of u onto v), and is denoted by projvu
w2 is called the vector component of u orthogonal to v
Orthogonal Projections
n
Let W be a subspace of R with an orthogonal basis
{u1, u2,…, uk }, the orthogonal projection of v onto W is
defined as:
projW v = proju v + proju v + … + proju v
1
2
k
The component of v orthogonal to W is the vector
perpW v = v – projw v
n
n
Let W be a subspace of R and v be any vector in R .
Then there are unique vectors w1 in W and w2 in W ┴
such that v = w1 + w2 .
Examples
3) Find the orthogonal projection of v = [ 1, -1, 2 ] onto W and
the component of v orthogonal to W.
1
a) W span 2
3
1
-1
b) W subspace spanned by 1 and
1
0
1
x
c) W y : x y 2 z 0
z
5.3
The Gram-Schmidt Process
And the QR Factorization
The Gram-Schmidt Process
Goal: To construct an orthogonal (orthonormal) basis for
n
any subspace of R .
We start with any basis {x1, x2,…, xk }, and “orthogonalize”
each vector vi in the basis one at a time by finding the
component of vi orthogonal to W = span(x1, x2,…, xi-1 ).
Let {x1, x2,…, xk } be a basis for a subspace W. Then
choose the following vectors:
v1 = x1,
v2 = x2 – projv x2
1
v3 = x3 – projv x3 – projv x3
1
2
… and so on
Then {v1, v2,…, vk } is orthogonal basis for W .
We can normalize each vector in the basis to form an
orthonormal basis.
Examples
1) Use the following basis to find an orthonormal basis for R
3 1
, ,
1 2
3
2) Find an orthogonal basis for R that contains the vector
1
1
1
2 ,
1
, 0
1
1
1
2
The QR Factorization
If A is an m x n matrix with linearly independent columns,
then A can be factored as A = QR where R is an invertible
upper triangular matrix and Q is an m x n orthogonal
n
matrix. In fact columns of Q form orthonormal basis for R
which can be constructed from columns of A by using
Gram-Schmidt process.
Note: Since Q is orthogonal, Q-1 = QT and we have R = QT A
Examples
3) Find a QR factorization for the following matrices.
3
A
1
1
2
1
A 2
1
-1
1
-1
- 1
0
1
5.4
Orthogonal Diagonalization
of Symmetric Matrices
Example
1) Diagonalize the matrix.
3
A
2
2
6
Recall:
A square matrix A is symmetric if AT = A.
A square matrix A is diagonalizable if there exists a
matrix P and a diagonal matrix D such that P-1AP = D.
Orthogonal Diagonalization
Definition:
A square matrix A is orthogonally diagonalizable if there
exists an orthogonal matrix Q and a diagonal matrix D
such that Q-1AQ = D.
Note that Q-1 = QT
Theorems
1. If A is orthogonally diagonalizable, then A is symmetric.
2. If A is a real symmetric matrix, then the eigenvalues of A
are real.
3. If A is a symmetric matrix, then any two eigenvectors
corresponding to distinct eigenvalues of A are orthogonal.
A square matrix A is orthogonally diagonalizable
if and only if it is symmetric.
Example
2) Orthogonally diagonalize the matrix
0
A 1
1
1
0
1
1
1
0
and write A in terms of matrices Q and D.
Theorem
If A is orthogonally diagonalizable, and QTAQ = D
then A can written as
A 1 q1 q1 2 q 2 q 2 ... n q n q n
T
T
T
where qi is the orthonormal column of Q, and λi is
the corresponding eigenvalue.
This fact will help us construct the matrix A given
eigenvalues and orthogonal eigenvectors.
Example
3) Find a 2 x 2 matrix that has eigenvalues 2 and 7, with
corresponding eigenvectors
v1
1
2
v
2
1
2
5.5
Applications
Quadratic Forms
A quadratic form in x and y :
a
T
2
2
ax by cxy x 1
2 c
c
x
b
1
2
A quadratic form in x,y and z:
a
2
2
2
ax by cz dxy exz fyz x T 12 d
12 e
where x is the variable (column) matrix.
1
2
d
b
1
2
f
e
1
f
x
2
c
1
2
Quadratic Forms
A quadratic form in n variables is a function
n
f : R R of the form:
f (x) x Ax
T
where A is a symmetric n x n matrix and x is in R
A is called the matrix associated with f.
z f ( x , y ) x y 8 xy
2
2
z f ( x, y ) 2 x 5 y
2
2
n
The Principal Axes Theorem
Every quadratic form can be diagonalized. In fact,
if A is a symmetric n x n matrix and if Q is an
orthogonal matrix so that QTAQ = D then the change
of variable x = Qy transforms the quadratic form into
x A x y D y 1 y1 2 y 2 ... n y n
T
T
2
2
2
Example: Find a change of variable that transforms the
Quadratic into one with no cross-product terms.
z f ( x , y ) x y 8 xy
2
2
z f ( x, y ) 2 x 5 y
2
2