Transcript Document

MA/CS 375
Fall 2002
Lecture 22
MA/CS 375 Fall 2002
1
Interlude on Norms
• We all know that the “size” of 2 is the
same as the size of -2, namely:
• ||2|| = sqrt(2*2) = 2
• ||-2|| = sqrt((-2)*(-2)) = 2
• The ||..|| notation is known as the norm
function.
MA/CS 375 Fall 2002
2
Norm of A Vector
• We can generalize the scalar norm to a
vector norm, however now there are
N
more choices for a vector x  :
x 1  x1  x2  x3  .....  x N
x 2  x12  x22  x32 .... x N2
x
MA/CS 375 Fall 2002

 max  x1 , x2 , x3 ,.... , x N

3
Norm of A Vector
Matlab command:
1) norm(x,1)
x 1  x1  x2  x3  .....  x N
2) norm(x,2)
x 2  x  x  x .... x
3) norm(x,inf)
x
MA/CS 375 Fall 2002
2
1

2
2
2
3
2
N
 max  x1 , x2 , x3 ,.... , x N

4
Norm of A Matrix
• We can generalize the vector norm to a
matrix norm, however now there are
M N
A

more choices for a matrix
:
iM
• 1-norm maximum absolute column sum
A 1  max  A ij
1 j  N
i 1
A 2  max Ax
x 1
2
jN
• infinity-norm, maximum absolute row sum
A
A
MA/CS 375 Fall 2002

F
 max  A ij
1i  M

j 1
iN jM
 A
2
ij
i 1 j 1
5
Norm of A Matrix
Matlab command:
iM
1) norm(A,1)
A 1  max  A ij
2) norm(A,2)
A 2  max Ax
1 j  N
i 1
x 1
2
jN
3) norm(A,inf)
4) norm(A,’fro’)
MA/CS 375 Fall 2002
A
A

F
 max  A ij
1i  M

j 1
iN jM
 A
2
ij
i 1 j 1
6
Matrix Norm in Action
• Theorem 5: (page 185 van Loan)
M N
ˆ
A

– If
is the stored version of A 
ˆ  A  E where E  M N and
then A
M N
E 1  A 1 eps
i.e. an error of order ||A||1eps occurs when
a real matrix is stored in floating point.
MA/CS 375 Fall 2002
7
Condition Number
• Recall that when we asked
Matlab to invert a matrix
which was almost singular:
we got this warning
MA/CS 375 Fall 2002
8
Definition of Condition Number
• rcond = 1/(condition number)
• We are going to use the 1-norm
definition of the condition number given
as:
1  A   A 1 A
MA/CS 375 Fall 2002
1
1
9
What’s With The
Condition Number ?
• Theorem 6: (page 235 van Loan)
– If A 
N N
is non-singular and:
Ax  b, where x, b  N
In addition if eps 1  A   1 then the stored
linear system: fl  A  xˆ  fl (b) is nonsingular
and also:
xˆ  x
x1
MA/CS 375 Fall 2002
1

xˆ 1 
  eps  1  A   1 

x1

10
In Plain English
• The theorem supposes that we can solve the Ax=b
problem without making any mistakes except for the
stored approximation of A and b then:
– If the condition number is small enough, then the
difference between the exact answer and the
calculated answer will be bounded above by:
const*condition#*eps
– Matlab value of eps is approximately 1e-16
– So if A has a condition number of 1 then we can
expect to solve Ax=b to 16 decimal places at best
– If A has a condition number of 1000 then in the worst
case we could make an error of 1e-13
– If A has a condition number of 10^16 then we could
make O(1) errors
MA/CS 375 Fall 2002
11
cond in Matlab
• cond(A,1) will return the 1-norm condition
number
• cond(A,2) will return the 2-norm condition
number
MA/CS 375 Fall 2002
12
Team Exercise
• Build
1  
A 1

 
 

1 

• For delta=1,0.1,0.01,…,1e-16 compute:
• cond(A,1)
• Plot a loglog graph of the
•x=delta, y=condition number
• Figure out what is going on
MA/CS 375 Fall 2002
13
Team Exercise (theory)
1  
A 1

 
 
1  
 A 1  1  1
2



1

 
 
1  

A 1  max abs column sum  1

1 
 

1


 max  1    ,   1   



1

for  small

MA/CS 375 Fall 2002
 

1 

1  

1
A 1  max abs column sum 2  1
1
 
 
 1
1 1 1
 max 
 3,  2
 
 


1
 3 for  small
 

1 






14
Team Exercise (theory)
 
 1    
1  

1

 A 1  max abs column sum 2  1
A 1  max abs column sum  1
1
 
1 

1 
 

 

 1
1 1 1 
1


 max 
 3,  2 
 max  1    ,   1   
 
 
 




1
1

for  small

for  small
3

1  A   A 1 A
MA/CS 375 Fall 2002
1
1

1

4
15
Team Exercise (theory)
1  A   A 1 A
1
1

1

4
Pretty big huh 
Did your results concur?
MA/CS 375 Fall 2002
16
Team Exercise (theory)
1  A   A 1 A
1
1

1

4
Pretty big huh 
Remark: remember how when we set delta = 2^(-11) and we
could actually get the exact inverse. Well in this case the
condition number is a little pessimistic as a guide !.
MA/CS 375 Fall 2002
17
Enough Theory !
MA/CS 375 Fall 2002
18
Interpolation
• Question:
– someone gives you a function
evaluated at 10 points, how does the the
function behave between those 10 points?
• Answer:
– guesses
MA/CS 375 Fall 2002
19
Interpolation
• Question:
– someone gives you a function
evaluated at 10 points, how does the the
function behave between those 10 points?
• Answer:
– there is no unique answer,
but we can make a good guess
MA/CS 375 Fall 2002
20
Polynomial Interpolation
• What’s the highest order unique
polynomial that you can fit through a
function evaluated at 1 point?
• That would be a constant function with
the same value as the given function
value.
MA/CS 375 Fall 2002
21
Polynomial Interpolation
• What’s the highest order unique
polynomial that you can fit through a
function evaluated at 2 points?
• That would be a linear function that
passes through the two values:
MA/CS 375 Fall 2002
22
Linear
Interpolation
MA/CS 375 Fall 2002
23
Linear
Interpolation
exp(x)
Samples
Linear
fit
MA/CS 375 Fall 2002
24
General Monomial Interpolation
• Given N function values at N distinct
points then there is one unique
polynomial which passes through these
N points and has order (N-1)
• Guess what – we can figure out the
coefficients of this polynomial by solving
a system of N equations 
MA/CS 375 Fall 2002
25
2nd Order Polynomial Fit
• we are going to use
a 2nd order approximation
• let’s make sure that the
approximation agrees
with the sample at:
• x1,x2 and x3
MA/CS 375 Fall 2002
f  x   a1  a2 x  a3 x
2
f  x1   a1  a2 x1  a x
2
3 1
f  x2   a1  a2 x2  a x
2
3 2
f  x3   a1  a2 x3  a x
2
3 3
26
2nd Order Polynomial Fit
We know:
• x1,x2 and x3
• f(x1),f(x2) and f(x3)
We do not know:
f  x1   a1  a2 x1  a x
2
3 1
f  x2   a1  a2 x2  a x
2
3 2
f  x3   a1  a2 x3  a x
2
3 3
• a1,a2 and a3
MA/CS 375 Fall 2002
27
2nd Order Polynomial Fit
We know:
• x1,x2 and x3
• f(x1),f(x2) and f(x3)
We do not know:
• a1,a2 and a3
MA/CS 375 Fall 2002
x

x

x

0
1
0
2
0
3
1
1
1
2
1
3
x
x
x
x   a1   f  x1  
  

x  a 2  f  x2 

  
x   a3   f  x3  
2
1
2
2
2
3
re-written as system
28
Finally A Reason To
Solve A System
So we can build the
following then: a = V\f
MA/CS 375 Fall 2002
x x

V x x
x x

 f  x1  


f  f  x2 


f x 
3 

0
1
0
2
0
3
1
1
1
2
1
3
x 

x 
x 
2
1
2
2
2
3
29
Expansion Coefficients
• Once we have the coefficients a1,a2,a3
then we are able to evaluate the
quadratic interpolating polynomial
anywhere.
f  x   a1  a2 x  a3 x
MA/CS 375 Fall 2002
2
30
Class Exercise
• Part 1:
– Build a function called vandermonde.m which accepts
a vector of x values and a polynomial order P
– In the function find N=length of x
– Function returns a matrix V which is Nx(P+1) and
whose entries are:
V(n,m) = (xn)(m-1)
• Part 2:
– Translate this pseudo-code to Matlab a script:
– for N=1:5:20
• build x
= set of N points in [-1,1]
• build f
= exp(x)
• build xfine = set of 10N points in [-1,1]
• build Vorig = vandermonde(N-1,x)
• build Vfine = vandermonde(N-1,xfine)
• build Finterp = Vfine*(Vorig\f);
• plot x,f and xfine,Finterp on the same graph
– end
MA/CS 375 Fall 2002
31
Next Lecture
• If you have a digital camera bring it in
• If not, bring in some jpeg, gif or tif pictures
(make sure they are appropriate)
• We will use them to do some multi-dimensional
interpolation
• Estimates for accuracy of polynomial interpolation
MA/CS 375 Fall 2002
32