Transcript Multiple View Geometry in Computer Vision

Projective 2D geometry
Appunti basati sulla parte iniziale del testo
di R.Hartley e A.Zisserman “Multiple view geometry”
Rielaborazione di materiale di M.Pollefeys
Projective 2D Geometry
• Points, lines & conics
• Transformations & invariants
• 1D projective geometry and
the Cross-ratio
Homogeneous coordinates
Homogeneous representation of lines
ax  by  c  0
a,b,cT
(ka) x  (kb) y  kc  0, k  0
a,b,cT ~ k a,b,cT
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
Homogeneous representation of points
T
T
x  x, y  on l  a,b,c if and only if ax  by  c  0
x,y,1a,b,cT  x,y,1 l  0
x, y,1T ~ k x, y,1T , k  0
The point x lies on the line l if and only if xTl=lTx=0
Homogeneous coordinates x1, x2 , x3 
T
Inhomogeneous coordinates x, y 
T
but only 2DOF
Points from lines and vice-versa
Intersections of lines
The intersection of two lines l and l' is x  l l'
Line joining two points
The line through two points x and x' is l  x x'
Line joining two points: parametric equation
A point on the line through two points x and x’
is y = x + q x’
Example
y 1
x 1
Ideal points and the line at infinity
Intersections of parallel lines
l  a, b, c and l'  a, b, c'
T
T
Example
l l'  b,a,0
T
b,a  tangent vector
a, b  normal direction
x 1 x  2
Ideal points
Line at infinity
x1, x2 ,0T
T
l  0,0,1
P 2  R 2  l
Note that in P2 there is no distinction
between ideal points and others
A model for the projective plane
exactly one line through two points
exaclty one point at intersection of two lines
Duality
x
l
x Tl  0
lT x  0
x  l l'
l  x x'
Duality principle:
To any theorem of 2-dimensional projective geometry
there corresponds a dual theorem, which may be
derived by interchanging the role of points and lines in
the original theorem
Conics
Curve described by 2nd-degree equation in the plane
ax2  bxy  cy2  dx  ey  f  0
or homogenized x  x1 x , y  x2 x
3
3
ax1  bx1x2  cx2  dx1x3  ex2 x3  fx32  0
2
2
or in matrix form
b / 2 d / 2
 a

e / 2 
x T C x  0 with C   b / 2 c
d / 2 e / 2
f 
5DOF:
a : b : c : d : e : f 
Five points define a conic
For each point the conic passes through
axi2  bxi yi  cyi2  dxi  eyi  f  0
or
x , x y , y , x , y , f c  0
2
i
i
i
2
i
i
i
stacking constraints yields
 x12
 2
 x2
 x32
 2
 x4
x2
 5
x1 y1
y12
x1
x2 y 2
y22
x2
x3 y3
y32
x3
x4 y 4
y42
x4
x5 y5
y52
x5
y1 1

y2 1
y3 1c  0

y4 1
y5 1
c  a, b, c, d , e, f 
T
Pole-polar relationship
The polar line l=Cx of the point x with respect to the
conic C intersects the conic in two points. The two
lines tangent to C at these points intersect at x
Polarity: cross ratio
Cross ratio of 4 colinear points yi = x + qi x’ (with i=1,..,4)
ratio of ratios
θ1 - θ 3
θ1 - θ 4
θ 2 - θ3
θ2 - θ4
Harmonic 4-tuple of colinear points: such that CR=-1
Chasles’ theorem
Conic = locus of constant cross-ratio towards 4 ref. points
A
B
C
X
D
Polarity: conjugacy
Given a point y and a conic C, a point z is conjugated of y wrt C
if the 4-tuple (y,z,x’ ,x’’ ) is harmonic,
where x’ and x’’ are the intersection points between C and line (y,z).
Intersection points (y+q z)TC (y +q z) = 0 is a 2-nd degree equation on q
yTCy + 2 q yTCz + q2 zTCz = 0 --> two solutions q’ and q’’.
Since y and z correspond to q  0 and q  , harmonic -> q’+ q’’ = 0
Hence the two solutions sum up to zero: yTCz = 0, and also zTCy = 0
 the point z is on the line Cy, which is called the polar line of y wrt C
the polar line Cy is the locus of the points conjugate of y wrt C
Conic C is the locus of points belonging to their own polar line yTCy = 0
Correlations and conjugate points
A correlation is an invertible mapping from
points of P2 to lines of P2. It is represented
by a 3x3 non-singular matrix A as l=Ax
Conjugate points
with respect to C
y Cx  0
T
(on each others polar)
Conjugate lines
with respect to C*
(through each others pole)
l C m0
T
*
Polarity and tangent lines
Consider a line through a point y, tangent to a conic C:
let us determine the point z conjugate of y wrt C:
since the intersection points coincide, then q’  q’’
 harmonic 4-tuple: also z coincides with the tangency point
 All points y on the tangent are conjugate to tangency point z wrt C
 polar line of a point z on the conic C is tangent to C through z
In addition: since there are two tangents to C through a point y not on C,
both tangency points are conjugate to y wrt C
polar line of y wrt C (=locus of the conjugate points)
= line through the two tangency points of lines through y
Tangent lines to conics
The line l tangent to C at point x on C is given by l=Cx
x
l
C
Dual conics
lT C* l  0
A line tangent to the conic C satisfies
In general (C full rank):
Line
C*  C1
in fact
-1
but since yT Cy
l is the polar line of y : y = C l ,
T -T
-1
-T
-1
lC CC l=0 C =C =C
*
Dual conics = line conics = conic envelopes
=0
Degenerate conics
A conic is degenerate if matrix C is not of full rank
m
e.g. two lines (rank 2)
l
C  lmT  mlT
e.g. repeated line (rank 1)
C  llT
l
Degenerate line conics: 2 points (rank 2), double point (rank1)
Note that for degenerate conics
C   C
* *
Projective transformations
Definition:
A projectivity is an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the same line if and
only if h(x1),h(x2),h(x3) do.
Theorem:
A mapping h:P2P2 is a projectivity if and only if there
exist a non-singular 3x3 matrix H such that for any point
in P2 represented by a vector x it is true that h(x)=Hx
Definition: Projective transformation
 x'1   h11
  
 x'2   h21
 x'  h
 3   31
h12
h22
h32
h13  x1 
 

h23  x2 
h33  x3 
or
x'  H x
8DOF
projectivity=collineation=projective transformation=homography
Mapping between planes
central projection may be expressed by x’=Hx
(application of theorem)
Removing projective distortion
select four points in a plane with known coordinates
x' 
x'1 h11 x  h12 y  h13

x'3 h31 x  h32 y  h33
y' 
x'2 h21 x  h22 y  h23

x'3 h31 x  h32 y  h33
x' h31 x  h32 y  h33   h11 x  h12 y  h13
(linear in hij)
y' h31 x  h32 y  h33   h21 x  h22 y  h23
(2 constraints/point, 8DOF  4 points needed)
Remark: no calibration at all necessary,
better ways to compute (see later)
More examples
Transformation of lines and conics
For a point transformation
x'  H x
Transformation for lines
l'  H -T l
Transformation for conics
C'  H-TC H-1
Transformation for dual conics
C'*  HC*HT
A hierarchy of transformations
Projective linear group
Affine group (last row (0,0,1))
Euclidean group (upper left 2x2 orthogonal)
Oriented Euclidean group (upper left 2x2 det 1)
Alternative, characterize transformation in terms of elements
or quantities that are preserved or invariant
e.g. Euclidean transformations leave distances unchanged
Class I: Isometries
(iso=same, metric=measure)
 x'   cosq
  
 y '     sin q
1  0
  
 sin q
cosq
0
t x  x 
 

t y  y 
1  1 
  1
orientation preserving:   1
orientation reversing:   1
 R t
x'  H E x   T  x
0 1
RTR  I
3DOF (1 rotation, 2 translation)
special cases: pure rotation, pure translation
Invariants: length, angle, area
Class II: Similarities
(isometry + scale)
 x'   s cosq
  
 y '    s sin q
1  0
  
 s sin q
s cosq
0
t x  x 
 

t y  y 
1  1 
sR t 
x' H S x   T  x
 0 1
RTR  I
4DOF (1 scale, 1 rotation, 2 translation)
also know as equi-form (shape preserving)
metric structure = structure up to similarity (in literature)
Invariants: ratios of length, angle, ratios of areas,
parallel lines
Class III: Affine transformations
 x'   a11
  
 y '   a21
1  0
  
a12
a22
0
t x  x 
 

t y  y 
1  1 
 A t
x'  H A x   T  x
0 1
A  UDVT  (UVT )(VDVT )
A  Rq R  DR 
where
1
D
0
0
2 
6DOF (2 scale, 2 rotation, 2 translation)
non-isotropic scaling! (2DOF: scale ratio and orientation)
Invariants: parallel lines, ratios of parallel
segment lengths, ratios of areas
Action of affinities and
projectivities
on line at infinity
A
0 T

 x1    x1  
t     A  
 x2     x2  

v   

0
0

  
Line at infinity stays at infinity,
but points move along line
A
vT

 x1  
x  
t     A 1  
x  
 x2   

 2
v   

v
x

v
x
0
1
1
2
2


 
Line at infinity becomes finite,
allows to observe vanishing points, horizon,
Class VI: Projective transformations
A
x' H P x   T
v
t
x

v
v  v1, v2 
T
8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)
Action: non-homogeneous over the plane
Invariants: cross-ratio of four points on a line
(ratio of ratio)
Decomposition of projective
transformations
sR t   K
H  H S H AH P   T   T
 0 1 0
0  I
1 vT
0  A
 T

v  v
t
v
A  sRK  tvT
decomposition unique (if chosen s>0)
K
upper-triangular, det
Example:
1.707 0.586 1.0 
H  2.707 8.242 2.0
 1.0
2.0 1.0 
2 cos 45

H   2 sin 45
 0

 2 sin 45
2 cos 45
0
1.0  0.5 1 0 1 0 0

2.0  0 2 0 0 1 0
1   0 0 1 1 2 1
K 1
Overview transformations
Projective
8dof
Affine
6dof
Similarity
4dof
Euclidean
3dof
 h11 h12
h
 21 h22
 h31 h32
 a11 a12
a
 21 a22
 0
0
h13 
h23 
h33 
tx 
t y 
1 
 sr11 sr12 t x 
 sr

sr
t
22
y
 21
 0
0
1 
 r11 r12 t x 
r

r
t
21
22
y


 0 0 1 
Concurrency, collinearity,
order of contact (intersection,
tangency, inflection, etc.),
cross ratio
Parallellism, ratio of areas,
ratio of lengths on parallel
lines (e.g midpoints), linear
combinations of vectors
(centroids).
The line at infinity l∞
Ratios of lengths, angles.
The circular points I,J
lengths, areas.
Number of invariants?
The number of functional invariants is equal to, or greater than,
the number of degrees of freedom of the configuration less the
number of degrees of freedom of the transformation
e.g. configuration of 4 points in general position has 8 dof (2/pt)
and so 4 similarity, 2 affinity and zero projective invariants
Projective geometry of 1D
x1, x2 T
x2  0
x'  H 22 x
3DOF (2x2-1)
The cross ratio
Crossx1 , x 2 , x 3 , x 4  
x1 , x 2 x 3 , x 4
x1 , x 3 x 2 , x 4
Invariant under projective transformations
 xi1
x i , x j  det 
 xi 2
x j1 
x j 2 
Recovering metric and affine
properties from images
• Parallelism
• Parallel length ratios
• Angles
• Length ratios
The line at infinity
l  H
 A
l   T T
 t A
T
A 
T
0
0 
 0   l 
1 
1
The line at infinity l is a fixed line under a projective
transformation H if and only if H is an affinity
Note: not fixed pointwise
Affine properties from images
projection
1 0
H ' P   0 1
l1 l2
0
0  H A
l3 
rectification
l'  l1 l2 l3  , l3  0
T
in fact, any point x on l’∞is mapped to a point at the ∞
Affine rectification
l∞
v1
l1
l2
l3
v2
v 2  l3  l 4
l4
v1  l1  l2
l  v1  v2
Distance ratios
d a, b : d b, c  a : b
0,1T , a,1T , a  b,1T
H
a , b, c
v' H1,0
T
The circular points
1
 
Ii
0
 
 s cosq
I  H S I   s sin q
 0
1
 
J  i
0
 
 s sin q
s cosq
0
t x  1 
1


 

iq
t y  i   se  i   I
0
1  0 
 
The circular points I, J are fixed points under the
projective transformation H iff H is a similarity
The circular points
“circular points”
l∞
x12  x22  dx1x3  ex2 x3  fx32  0
x3  0
x12  x22  0
I  1, i,0 
T
Intersection points between any circle and l∞
J  1,i,0 
Algebraically, encodes orthogonal directions
I  1,0,0  i0,1,0
T
T
T
Conic dual to the circular points
C*  IJT  JI T
1 0 0


*
C   0 1 0 
0 0 0
A line conic: set of lines through any of the circular points
C*  HS C*HTS
The dual conic C* is fixed conic under the

projective transformation H iff H is a similarity
Note: C* has 4DOF
l∞ is the null vector
Angles
Euclidean: l  l1 , l2 , l3 
m  m1, m2 , m3 
T
cos q 
l
2
1
l1m1  l2 m2
l
2
2
m
2
1
m
Projective: cos q 
2
2


l'
T
T
l
l T C* m
T

C* l m T C* m
l'T C* ' m'

C* ' l' m'T C* ' m'


in fact, e.g. lT C* m  l'T HC* HT m' l'T C* ' m'
lT C* ' m  0 (orthogonal)
Length ratios
d (b, c) sin 

d (a, c) sin 
Metric properties from images
C* '  H P H A H S C* H P H A H S 
T
 H P H A H S C* H TS H P H A 
T
 H P H A C* H P H A 
T
KK T
 T
v K
K T v
T 
v v
Rectifying transformation from SVD
1 0 0 
C* '  U 0 1 0 U T
0 0 0
HU
1 0 0 
Why C* '  U 0 1 0 U T ?


0 0 0
Normally: SVD (Singular Value Decomposition)
 a 0 0
C* '  V 0 b 0 U T
0 0 c 
with U and V orthogonal
But C* ' is symmetric  C* 'T  UDVT  VDUT  C* '
and SVD is unique 
Observation : H=U
UV
orthogonal (3x3): not a
P2
isometry
Metric from affine
Once the image has been affinely rectified
C '  HAC HA
*

K
*
C '   T
0
*

T
1 0 0 T
t
K

0 1 0  T


1
t

0 0 0
0 KK T
 T
1  0
0

0
Metric from affine
l1
l2
KK
l3 
 0
T
 m1 
0 
 m2   0
0 
 m3 
l1m1, l1m2  l2 m1, l2 m2 k
2
11
 k , k11k12 , k
2
12

2 T
22
0
Metric from projective
l1
l2
KK
l3  T
v K
T
 m1 
K v  
m  0
T  2 
v v  
 m3 
T
l1m1,0.5l1m2  l2 m1 , l2 m2 ,0.5l1m3  l3m1 ,0.5l2 m3  l3m2 , l3m3 c  0
Projective conic classification
C  UDUT
D  U 1CU-T
D  diags1, s2 , s3 diage1, e2 , e3 diags1, s2 , s3 
ei  1 or 0
Diagonal
Equation
(1,1,1)
(1,-1,0)
x 2  y 2  w2  0
x 2  y 2  w2  0
x2  y 2  0
x2  y 2  0
(1,0,0)
x2  0
(1,1,-1)
(1,1,0)
Conic type
improper conic
circle
single real point
two lines
single line
Affine conic classification
ellipse
parabola
hyperbola
Fixed points and lines
He  e
H
T
(eigenvectors H =fixed points)
(1=2  pointwise fixed line)
l   l (eigenvectors H-T =fixed lines)