Transcript Image Processing

1
Geometric Transformation
 Operations depend on pixel’s Coordinates.
 Context free.
 Independent of pixel values.
x  f x x, y   x'
y  f y x, y   y'
(x,y)
I(x,y)
I ( x, y)  I '  f x x, y , f y x, y 
(x’,y’)
I’(x’,y’)
• Example: Translation
x  f x x, y   x  3
y  f y x, y   y  1
I ' x  3, y 1  I ( x, y)
(x’,y’)
(x,y)
I(x,y)
I’(x’,y’)
Forward Mapping
 Forward mapping:
Source
x  f x x, y   x
y  f y x, y   y
Target
Forward
Mapping
Forward vs. Inverse Mapping
 Forward mapping:
Source
x  f x x, y   x
y  f y x, y   y
Target
Forward
mapping
 Problems with forward mapping due to sampling:
– Holes (some target pixels are not populated)
– Overlaps (some target pixels assigned few colors)
 Inverse mapping:
Source
x  f x1  x, y  x
y  f y1  x, y  y
Target
Inverse
Mapping
 Each target pixel assigned a single color.
 Color Interpolation is required.
• Example: Scaling along X
– Forward mapping:
x  2 x ; y  y
Target
Source
(0,0)
(0,0)
– Inverse mapping: x  x / 2 ; y  y
Source
Target
Interpolation
• What happens when a mapping function calculates a
fractional pixel address?
• Interpolation: generates a new pixel by analyzing the
surrounding pixels.
Interpolation
• Good interpolation techniques attempt to find an optimal
balance between three undesirable artifacts: edge halos,
blurring and aliasing.
x4 scaling
N.N
Bilinear
9
Bicubic
Nearest Neighbor Interpolation
• The assign value is taken from the pixel closest to the
generated location:





I x, y  I round f x1 x, y , round f y1 x, y
• Advantage:
– Fast
• Disadvantage:
– Jagged results
– Discontinues results
Original Image
Nearest N.
Interpolation
Original Image
Nearest N.
Interpolation
Bilinear Interpolation
• The assigned value is an intermediate value between the
four nearest pixels:
Linear Interpolation
ve
v
vw
xw
x
xe
x  xw
v  vw

xe  x w ve  v w
• Isolating v in the above equation:
v  ve  1   vw
where  
x  xw
xe  xw
Bilinear Interpolation
• The assign value is a weighted sum of the four
nearest pixels.
• Each weight is proportional to the distance from each
existing pixel.
NW
V
S
SW
N
NE
SE
t
s
S  SW  1  s   SE  s ; N  NW  1  s   NE  s
V  S  1  t   N  t
• The bilinear interpolation is the best fit low-degree
polynomial of the form:
v ( s, t ) 
1
i j
a
s
 ij t
i , j 0
• The pixel’s boundaries are C0 continuous
(continuous values across boundaries).
Bilinear example
s
s
z=15
t
15
z=7
10
v
t
5
0
2
0
z=2
z=3
12
1.5
1.5
1
10
Nearest N.
Interpolation
Bilinear
Interpolation
Nearest N.
Interpolation
Bilinear
Interpolation
Bicubic Interpolation
• The assign value is a weighted sum of the 4x4
nearest pixels:
v ( s, t ) 
3
i j
a
s
 ij t
i , j 0
How can we find the right coefficients?
• Denote the pixel values Vpq {p,q=0..3}
• The unknown coefficients are aij {i,j=0..3}
v pq 
3
i j
a
s
 ij t
for p, q  {0..3}, s, t [1,2]
i , j 0
• We have a linear system of
16 equations with 16
coefficients.
• The pixel’s boundaries are t
C1 continuous (continuous
derivatives across
boundaries).
21
s
N.N
Bilinear
Bicubic
N.N
Bilinear
Bicubic
Applying the Transformation
T = …… % 2x2 transformation matrix
[r,c] = size(img)
% create array of destination x,y coordinates
[X,Y]=meshgrid(1:c,1:r);
% calculate source coordinates
sourceCoor = inv(T) * [X(:) Y(:) ] ‘ ;
% calculate nearest neighbor interpolation
Xs = round(sourceCoor(1,:));
Ys = round(sourceCoor(2,:));
indx=find(Xs<1 | Xs>r); %out of range pixels
Xs(indx)=1; Ys(indx)=1;
indy=find(Ys<1 | Ys>c);
Xs(indy)=1; Ys(indy)=1;
%out of range pixels
% calculate new image
newImage = img((Xs-1).*r+Ys);
newImage(indx)=0; newImage(indx)=0;
newImage = reshape(newImage,r,c);
Types of linear 2D transformations
• Rigid (Euclidean) transformation:
– Translation + Rotation (distance preserving).
• Similarity transformation:
– Translation + Rotation + Uniform Scale (angle preserving).
• Affine transformation:
– Translation + Rotation + Scale + Shear (parallelism preserving).
• Projective transformation
– Cross-ratio preserving
• All above transformations are groups where Rigid 
Similarity  Affine  Projective
Homogeneous Coordinates
• Homogeneous Coordinates is a mapping from Rn to
Rn+1:
( x, y)  ( X , Y ,W )  (tx, ty, t )
• Note: (tx,ty,t) all correspond to the same nonhomogeneous point (x,y). E.g. (2,3,1)(6,9,3) (4,6,2).
• Inverse mapping:
X Y 
 X , Y ,W    ,   x, y
W W 
Some 2D Transformations
• Translation :
 X  1 0 t x   x   x  t x 
 Y    0 1 t   y    y  t 
y  
y
  

W  0 0 1   1   1 
• Affine transformation:
 X  a b t x   x 
 Y    c d t   y
y  
  
W   0 0 1   1 
• Projective transformation:
 X  a
 Y    c
  
W   e
b
d
f
tx  x
t y   y 
1   1 
Hierarchy of Linear 2D Transformations
Non Linear 2D Transformations
• Non linear transformations do not necessarily preserve
straight lines.
• Methods:
– Piecewise linear transformations
– Non linear parametric mapping
• Non linear mapping:
i j

x s, t    aij s t
i, j
i j

y s, t    bij s t
i, j
• Example: non linear (radial) lens distortions:
Transformation Estimation
• Let: x’=fx(x,y,px) ; y’=fy(x,y,py), where px and py are
vector of parameters.
• If the mappings are linear in px and py the parameters
can be estimated using linear regression.
• Example: Affine transformation
 X  a b t x   x 
 Y    c d t   y
y  
  
W   0 0 1   1 
• Alternative representation:
 x   x y 0 0 1 0 
   

 y   0 0 x y 0 1 
a
 
b
c
 
d 
t 
 x
t 
 y
• Given k points (P1,P2,..Pk) in 2D that have been
transformed to (P'1,P'2,..,P'k) by affine transformation:
– How many points uniquely define the affine (projective)
transformation?
– How can we find the affine transformation?
– What if we have more points?
– What can be done if points coordinates are inaccurate?
Mp  b

pˆ  mina Mp  b

pˆ  M M
T

1
T
M b

Image Warping and Morphing
• Image rectification.
• Key frame animation.
• Image Synthesis
– Facial expression
– Viewing positions
Image Metamorphosis.
Image Metamorphosis
Cross Dissolve (pixel operations)
Source
Image
Destination
Image
t
cross dissolve
I t   S 1  t   T t 
t  [0,1]
warp + dissolve
Warping + Cross Dissolve
• Warp source image towards intermediate image.
• Warp destination image towards intermediate image.
• Cross-dissolve the two images by taking the weighted average
at each pixel.
source
Cross-dissolve
time
destination
warping images
warp
Cross-dissolve
Cross-dissolve
warp
Image Metamorphosis
• Let S,T be the source and the target images
• Let G(p) be the transformation from S towards T, where
G(0)=I the identity transformation
• Let t[0,1] the time step to be synthesized
Algorithm:
1. Warp S towards T:
S t   Gtp S
2. Warp T toward S:
T t   G1  t  p T 
3. Cross dissolve:
1
I t   1  t S t   t  T t 
t
sourse
S(t)=G(tp){S}
I(t)=(1-t)S(t)+t(T(t))
target
T(t)=G((1-t)p)-1{T}
Feature Based Morphing
• Morph one shape into another shape
• Use local features to define the geometric warping
Q’
Q
P
P’
Q
Q’
P’
P
Q
Q’
P’
P
Q
Q’
P’
P
Q
Q’
P’
P
Q
Q’
P’
P
One Segment Warping
Q’
Q

R

u
v
R’


P
u’
v’
P’
Source Image
Dest Image
• [0,1] is the relative position along the segment (P’,Q’).
•  is the actual perpendicular distance to the segment.
• (u’,v’) is the local coordinates of the segment (P’,Q’):
– u’ is a unit vector parallel to Q’-P’
– v’ is the unit vector perpendicular to Q’-P’

Q  P
u 
Q  P
 uy 

v  u  
  ux 
Q’
R’


u’
v’
P’
• The point R’ is mapped into (,) :

R  P  u

Q  P
;   R  P  v
where
R'  P' Q' P' u' v'
Q’
Q

R

u
v

P
Source Image
R’

u’
v’
P’
Dest Image
Inverse Mapping:
R( ,  )  P   Q  P u  v
where (u,v) is the local coordinates of the segment (P,Q):

Q  P
u
QP
 uy 

v  u  
  ux 
Multiple Segment Warping
Q1
2
R1
1
Q2
R2
P1
Q2’
Q1’
’1
P2
R’
’2
P1’ P2’
• In multiple segment warping the point R’ is influenced by multiple
segments.
• The influence strength of each segments is proportional to:
– Segment length
– The distance from the point R’
• The influence of each segments is:
 Qi  Pi
Wi  
 a  i

p




b
• The value p[0,1] controls the influence of the line length.
• The value a is a small number avoiding division by zero.
• The value b determines how the relative weight diminish as the 
increases
• The final mapping is:
W R
R
W
k
k
k
k
k
Example:
Example images from:
http://www.cc.gatech.edu/classes/AY2001/cs4451_spring/projects/Seven/
For more details see:
Thaddeus Beier & Shawn Neely / Feature-Based Image Metamorphosis Siggraph '92
http://www.hammerhead.com/thad/morph.html
Another Example:
56
THE
END