Transcript Multiple View Geometry in Computer Vision

Multiple View Geometry
In Computer Vision
Math 607
Ron Sahoo
Department of Mathematics
University of Louisville
Basic Information
Instructor: Dr. Ron Sahoo
Office: NS 218
Tel: 852 - 2731 Fax: 852 - 7132
Email: [email protected]
Course Information
Class Room: NS 212F
Scheduled Time: 11:00 - 12:15
Office Hour: 1:30 - 2:30 pm
Prerequisites:
Vector Calculus
Linear Algebra
Abstract Algebra
Geometry
Target Audience
The target audience is:
(1)EE graduate students that are doing or
planning to do research in CVIP lab.
(2) Mathematics graduate students that
are interested in applications of modern
geometry and multi-linear algebra to
computer vision.
Course Objectives
The objectives of this course are:
(1)To understand projective geometry
underlying multiview image formation.
(2) To understand the general principles
of linear and non-linear parameter
estimation methods.
(3) To be able to apply the state of the art
methods for estimatind 2D and 3D
geometry from images.
Grading Scheme
Grades will be based on 2 tests and
Final Project.
The Final Project can be:
(1) On a significant literature research
project (you are welcome to propose
your own topic).
(2) A practical project in computer vision
consistent with the course.
Final Project
For the Final Project you will:
(1) Hand in a written proposal.
(2) Prepare one presentation on your topic
for the class.
(3) Hand in a type set final report (about
6-8 pages in the format and style of a
research report).
Changes to Syllabus
The instructor reserves the right to make
changes in the syllabus when necessary to
meet learning objectives, to compensate
for missed classes, or for similar reasons.
Projective 2D geometry
Lecture 1
Multiple View Geometry
In Computer Vision
Math 607
Projective 2D Geometry
• Points, lines & conics
• Transformations & invariants
• 1D projective geometry and
the Cross-ratio
Homogeneous coordinates
Homogeneous representation of lines
ax  by  c  0
a,b,cT
(ka) x  (kb) y  kc  0, k  0
a,b,cT ~ k a,b,cT
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
Homogeneous representation of points
T
T
x  x, y  on l  a,b,c if and only if ax  by  c  0
x,y,1a,b,cT  x,y,1 l  0
x, y,1T ~ k x, y,1T , k  0
The point x lies on the line l if and only if xTl=lTx=0
Homogeneous coordinates x1, x2 , x3 
T
Inhomogeneous coordinates x, y 
T
but only 2 DOF
Points from lines and vice-versa
Intersections of lines
The intersection of two lines l and l' is x  l l'
Line joining two points
The line through two points x and x' is l  x x'
Example
y 1
x 1
Ideal points and the line at infinity
Intersections of parallel lines
l  a, b, c and l'  a, b, c'
T
T
Example
l l'  b,a,0
T
b,a  tangent vector
a, b  normal direction
x 1 x  2
Ideal points
Line at infinity
x1, x2 ,0T
T
l  0,0,1
P 2  R 2  l
Note that in P2 there is no distinction
between ideal points and others
A model for the projective plane
exactly one line through two points
exaclty one point at intersection of two lines
Duality
x
l
x Tl  0
lT x  0
x  l l'
l  x x'
Duality of points and lines
Points and lines are dual to each other in
projective space:
If p and q are points then the cross
product L = p x q denotes the line
defined by the two points.
If L and M are lines, then the cross
product p = L x M denotes the point
where two lines intersect.
Duality principle
To any theorem of 2-dimensional
projective geometry there corresponds
a dual theorem, which may be derived
by interchanging the role of points and
lines in the original theorem
Projective Geometry: Use
- Camera models
- Modeling scene transformations
- Camera calibration
- Single view geometry
- Stereo analysis: two (or more) views
- 3D reconstruction
Conics
Curve described by 2nd-degree equation in the plane
ax2  bxy  cy2  dx  ey  f  0
or homogenized x  x1 x , y  x2 x
3
3
ax1  bx1x2  cx2  dx1x3  ex2 x3  fx32  0
2
2
or in matrix form
b / 2 d / 2
 a

e / 2 
x T C x  0 with C   b / 2 c
d / 2 e / 2
f 
5DOF:
a : b : c : d : e : f 
Five points define a conic
For each point the conic passes through
axi2  bxi yi  cyi2  dxi  eyi  f  0
or
x , x y , y , x , y , f c  0
2
i
i
i
2
i
i
i
stacking constraints yields
 x12
 2
 x2
 x32
 2
 x4
x2
 5
x1 y1
y12
x1
x2 y 2
y22
x2
x3 y3
y32
x3
x4 y 4
y42
x4
x5 y5
y52
x5
y1 1

y2 1
y3 1c  0

y4 1
y5 1
c  a, b, c, d , e, f 
T
Tangent lines to conics
The line l tangent to C at point x on C is given by l=Cx
x
l
C
Dual conics
A line tangent to the conic C satisfies
In general (C full rank):
lT C* l  0
C*  C1
Dual conics = line conics = conic envelopes
Degenerate conics
A conic is degenerate if matrix C is not of full rank
m
e.g. two lines (rank 2)
l
C  lmT  mlT
e.g. repeated line (rank 1)
C  llT
l
Degenerate line conics: 2 points (rank 2), double point (rank1)
Note that for degenerate conics
C   C
* *
Projective transformations
Definition:
A projectivity is an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the same line if and
only if h(x1),h(x2),h(x3) do.
Theorem:
A mapping h:P2P2 is a projectivity if and only if there
exist a non-singular 3x3 matrix H such that for any point
in P2 reprented by a vector x it is true that h(x)=Hx
Definition: Projective transformation
 x'1   h11
  
 x'2   h21
 x'  h
 3   31
h12
h22
h32
h13  x1 
 

h23  x2 
h33  x3 
or
x'  H x
8DOF
projectivity=collineation=projective transformation=homography
Mapping between planes
central projection may be expressed by x’=Hx
(application of theorem)
Removing projective distortion
select four points in a plane with know coordinates
h x  h12 y  h13
h x  h22 y  h23
x'
x'
x'  1  11
y'  2  21
x'3 h31 x  h32 y  h33
x'3 h31 x  h32 y  h33
x' h31 x  h32 y  h33   h11 x  h12 y  h13
(linear in hij)
y' h31 x  h32 y  h33   h21 x  h22 y  h23
(2 constraints/point, 8DOF  4 points needed)
Remark: no calibration at all necessary,
better ways to compute (see later)
More examples
Transformation of lines and conics
For a point transformation
x'  H x
Transformation for lines
l'  H -T l
Transformation for conics
C'  H-TC H-1
Transformation for dual conics
C'*  HC*HT
A hierarchy of transformations
Projective linear group
Affine group (last row (0,0,1))
Euclidean group (upper left 2x2 orthogonal)
Oriented Euclidean group (upper left 2x2 det 1)
Alternative, characterize transformation in terms of elements
or quantities that are preserved or invariant
e.g. Euclidean transformations leave distances unchanged
Class I: Isometries
(iso=same, metric=measure)
 x'   cos
  
 y '     sin 
1  0
  
 sin 
cos
0
t x  x 
 

t y  y 
1  1 
  1
orientation preserving:   1
orientation reversing:   1
 R t
x'  H E x   T  x
0 1
RTR  I
3DOF (1 rotation, 2 translation)
special cases: pure rotation, pure translation
Invariants: length, angle, area
Class II: Similarities
(isometry + scale)
 x'   s cos
  
 y '    s sin 
1  0
  
 s sin 
s cos
0
t x  x 
 

t y  y 
1  1 
sR t 
x' H S x   T  x
 0 1
RTR  I
4DOF (1 scale, 1 rotation, 2 translation)
also know as equi-form (shape preserving)
metric structure = structure up to similarity (in literature)
Invariants: ratios of length, angle, ratios of areas,
parallel lines
Class III: Affine transformations
 x'   a11
  
 y '   a21
1  0
  
t x  x 
 

a22 t y  y 
0 1  1 
 A t
x'  H A x   T  x
0 1
a12
A  R R  DR 
1
D
0
0
2 
6DOF (2 scale, 2 rotation, 2 translation)
non-isotropic scaling! (2DOF: scale ratio and orientation)
Invariants: parallel lines, ratios of parallel lengths,
ratios of areas
Class VI: Projective transformations
A
x' H P x   T
v
t
x

v
v  v1, v2 
T
8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)
Action non-homogeneous over the plane
Invariants: cross-ratio of four points on a line
(ratio of ratio)
Action of affinities and projectivities
on line at infinity
A
0 T

 x1    x1  
t     A  
 x2     x2  

v   

0
0

  
Line at infinity stays at infinity,
but points move along line
A
vT

 x1  
x  
t     A 1  
x  
 x2   

 2
v   

v
x

v
x
0
1
1
2
2


 
Line at infinity becomes finite,
allows to observe vanishing points, horizon,
Decomposition of projective
transformations
sR t   K
H  H S H AH P   T   T
 0 1 0
0  I
1 vT
0  A
 T

v  v
t
v
A  sRK  tvT
decomposition unique (if chosen s>0)
K
upper-triangular, det
Example:
1.707 0.586 1.0 
H  2.707 8.242 2.0
 1.0
2.0 1.0 
2 cos 45

H   2 sin 45
 0

 2 sin 45
2 cos 45
0
1.0  0.5 1 0 1 0 0

2.0  0 2 0 0 1 0
1   0 0 1 1 2 1
K 1
Overview transformations
Projective
8dof
Affine
6dof
Similarity
4dof
Euclidean
3dof
 h11 h12
h
 21 h22
 h31 h32
 a11 a12
a
 21 a22
 0
0
h13 
h23 
h33 
tx 
t y 
1 
 sr11 sr12 t x 
 sr

sr
t
22
y
 21
 0
0
1 
 r11 r12 t x 
r

r
t
21
22
y


 0 0 1 
Concurrency, collinearity,
order of contact (intersection,
tangency, inflection, etc.),
cross ratio
Parallellism, ratio of areas,
ratio of lengths on parallel
lines (e.g midpoints), linear
combinations of vectors
(centroids).
The line at infinity l∞
Ratios of lengths, angles.
The circular points I,J
lengths, areas.
Number of invariants?
The number of functional invariants is equal to, or greater than,
the number of degrees of freedom of the configuration less the
number of degrees of freedom of the transformation
e.g. configuration of 4 points in general position has 8 dof (2/pt)
and so 4 similarity, 2 affinity and zero projective invariants