Transcript week04-vectors.ppt

Vectors [and more on masks]
Vector space theory applies directly to
several image
processing/representation problems
MSU CSE 803 Stockman Fall 2009
Image as a sum of “basic images”
What if every person’s portrait photo could be expressed as a
sum of 20 special images?  We would only need 20
numbers to model any photo  sparse rep on our Smart
card.
MSU CSE 803 Stockman Fall 2009
Efaces
100 x 100 images of
faces are
approximated by a
subspace of only 4
100 x 100 “images”,
the mean image
plus a linear
combination of the 3
most important
“eigenimages”
MSU CSE 803 Stockman Fall 2009
The image as an expansion
MSU CSE 803 Stockman Fall 2009
Different bases, different
properties revealed
MSU CSE 803 Stockman Fall 2009
Fundamental expansion
MSU CSE 803 Stockman Fall 2009
Basis gives structural parts
MSU CSE 803 Stockman Fall 2009
Vector space review, part 1
MSU CSE 803 Stockman Fall 2009
Vector space review, Part 2
2
MSU CSE 803 Stockman Fall 2009
A space of images in a vector
space




M x N image of real intensity values has
dimension D = M x N
Can concatenate all M rows to interpret
an image as a D dimensional 1D vector
The vector space properties apply
The 2D structure of the image is NOT
lost
MSU CSE 803 Stockman Fall 2009
Orthonormal basis vectors help
MSU CSE 803 Stockman Fall 2009
Represent S = [10, 15, 20]
MSU CSE 803 Stockman Fall 2009
Projection of vector U onto V
MSU CSE 803 Stockman Fall 2009
Normalized dot product
Can now think about the angle between two
signals, two faces, two text documents, …
MSU CSE 803 Stockman Fall 2009
Every 2x2 neighborhood has
some constant, some edge, and
some line component
Confirm that
basis vectors
are orthonormal
MSU CSE 803 Stockman Fall 2009
Roberts basis cont.
If a neighborhood N has large dot product with a
basis vector (image), then N is similar to that
basis image.
MSU CSE 803 Stockman Fall 2009
Standard 3x3 image basis
Structureless and relatively useless!
MSU CSE 803 Stockman Fall 2009
Frie-Chen basis
Confirm that
bases vectors
are orthonormal
MSU CSE 803 Stockman Fall 2009
Structure from Frie-Chen expansion
Expand N using FrieChen basis
MSU CSE 803 Stockman Fall 2009
Sinusoids provide a good basis
MSU CSE 803 Stockman Fall 2009
Sinusoids also model well in images
MSU CSE 803 Stockman Fall 2009
Operations using the Fourier basis
MSU CSE 803 Stockman Fall 2009
A few properties of 1D sinusoids
They are orthogonal
Are they orthonormal?
MSU CSE 803 Stockman Fall 2009
F(x,y) as a sum of sinusoids
MSU CSE 803 Stockman Fall 2009
Spatial direction and
frequency in 2D
MSU CSE 803 Stockman Fall 2009
Continuous 2D Fourier Transform
To compute F(u,v) we do a dot product of our image
f(x,y) with a specific sinusoid with frequencies u and v
MSU CSE 803 Stockman Fall 2009
Power spectrum from FT
MSU CSE 803 Stockman Fall 2009
Examples from images
Done with HIPS in 1997
MSU CSE 803 Stockman Fall 2009
Descriptions of former spectra
MSU CSE 803 Stockman Fall 2009
Discrete Fourier Transform
Do N x N dot products and determine where the energy is.
High energy in parameters u and v means original image
has similarity to those sinusoids.
MSU CSE 803 Stockman Fall 2009
Bandpass filtering
MSU CSE 803 Stockman Fall 2009
Convolution of two functions in the spatial
domain is equivalent to pointwise
multiplication in the frequency domain
MSU CSE 803 Stockman Fall 2009
LOG or DOG filter
Laplacian of Gaussian
Approx
Difference of Gaussians
MSU CSE 803 Stockman Fall 2009
LOG filter properties
MSU CSE 803 Stockman Fall 2009
Mathematical model
MSU CSE 803 Stockman Fall 2009
1D model;
rotate to create 2D model
MSU CSE 803 Stockman Fall 2009
1D Gaussian and 1st derivative
MSU CSE 803 Stockman Fall 2009
2nd derivative; then all 3 curves
MSU CSE 803 Stockman Fall 2009
Laplacian of Gaussian as 3x3
MSU CSE 803 Stockman Fall 2009
G(x,y): Mexican hat filter
MSU CSE 803 Stockman Fall 2009
Convolving LOG with region
boundary creates a zero-crossing
Mask h(x,y)
Input f(x,y)
Output f(x,y) * h(x,y)
MSU CSE 803 Stockman Fall 2009
MSU CSE 803 Stockman Fall 2009
LOG relates to animal vision
MSU CSE 803 Stockman Fall 2009
1D EX.
Artificial Neural
Network (ANN) for
computing
g(x) = f(x) * h(x)
level 1 cells feed 3
level 2 cells
level 2 cells integrate 3
level 1 input cells
using weights [-1,2,-1]
MSU CSE 803 Stockman Fall 2009
Experience the Mach band effect
MSU CSE 803 Stockman Fall 2009
Simple model of a neuron
MSU CSE 803 Stockman Fall 2009
Output conditioning: threshold
versus smoother output signal
MSU CSE 803 Stockman Fall 2009
3D situation in the eye
Neuron c has +
input to neuron A
but - input to
neuron B.
Neuron d has +
input to neuron B
but – input to
neuron A.
Neuron b gives no
input to neuron B:
it is not in the
receptive field of B.
MSU CSE 803 Stockman Fall 2009
Receptive fields
MSU CSE 803 Stockman Fall 2009
Experiments with cats/monkeys





Stabilize/drug animal to stare
Place delicate probe in visual network
Move step edge across FOV
Probe shows response function when
the edge images to receptive field
Slightly moving the probe produces
similar signal when edge is nearby
MSU CSE 803 Stockman Fall 2009
Canny edge detector uses LOG
filter
MSU CSE 803 Stockman Fall 2009
Summary of LOG filter




Convenient filter shape
Boundaries detected as 0-crossings
Psychophysical evidence that animal
visual systems might work this way
(your testimony)
Physiological evidence that real NNs
work as the ANNs
MSU CSE 803 Stockman Fall 2009