Transcript Diapositiva 1 - Politecnico di Milano

Self-calibration
Outline
•
•
•
•
Introduction
Self-calibration
Dual Absolute Quadric
Critical Motion Sequences
Motivation
• Avoid explicit calibration procedure
– Complex procedure
– Need for calibration object
– Need to maintain calibration
Motivation
• Allow flexible acquisition
– No prior calibration necessary
– Possibility to vary intrinsics
– Use archive footage
Example
Projective ambiguity
Reconstruction from uncalibrated images
 projective ambiguity on reconstruction
m  P M  (PT 1)(T M)  P´M´
Stratification of geometry
Projective
15 DOF
Affine
Metric
7 DOF
12 DOF
plane at infinity absolute conic
parallelism
angles, rel.dist.
More general
More structure
Constraints ?
• Scene constraints
– Parallellism, vanishing points, horizon, ...
– Distances, positions, angles, ...
Unknown scene  no constraints
• Camera extrinsics constraints
–Pose, orientation, ...
Unknown camera motion  no constraints
• Camera intrinsics constraints
–Focal length, principal point, aspect ratio & skew
Perspective camera model too general
 some constraints
Euclidean projection matrix
Factorization of Euclidean projection matrix
P  K R T
 R T t
fx

Intrinsics: K 


Extrinsics:
R, t 
s
fy
ux 
uy 
1 
(camera geometry)
(camera motion)
Note: every projection matrix can be factorized,
but only meaningful for euclidean projection matrices
Constraints on intrinsic
parameters
 fx
K  
Constant 
s
fy
ux 
u y 
1 
e.g. fixed camera:
Known
e.g. rectangular pixels:
square pixels:
principal point known:
K1  K2  
s 0
f x  fy , s  0
w h 
ux ,uy    , 
 2 2
Self-calibration
Upgrade from projective structure to metric
structure using constraints on intrinsic
camera parameters
– Constant intrinsics
(Faugeras et al. ECCV´92, Hartley´93,
Triggs´97, Pollefeys et al. PAMI´98, ...)
– Some known intrinsics, others varying
(Heyden&Astrom CVPR´97, Pollefeys et al. ICCV´98,...)
– Constraints on intrincs and restricted motion
(e.g. pure translation, pure rotation, planar motion)
(Moons et al.´94, Hartley ´94, Armstrong ECCV´96, ...)
A counting argument
• To go from projective (15DOF) to metric (7DOF) at
least 8 constraints are needed
• Minimal sequence length should satisfy
n  # known   n  1  # fixed   8
• Independent of algorithm
• Assumes general motion (i.e. not critical)
Self-calibration:
conceptual algorithm
Given projective structure and motion {Pj,Mi},
then the metric structure and motion can be
obtained as {PjT-1,TMi}, with
T  arg min C K P1T 1 , K P2T 1 ,, K Pn T 1 
T
C K 1 , K 2 ,, K n  criterium expressing constraints
K (P) function extracting intrinsics
from projection matrix
Outline
•
•
•
•
Introduction
Self-calibration
Dual Absolute Quadric
Critical Motion Sequences
Conics & Quadrics
conics
m Cm  0
T
quadrics
l Cl0
T
C C
*
*
MTQM  0
 TQ *   0
Q *  Q 1
1
transformations
C  C´~ HTCH1
Q  Q´~ T  TQT 1
C*  C*´~ HC*HT
Q *  Q *´~ TQ *T T
projection
C* ~ PQ*PT
The Absolute Dual
Quadric (Triggs CVPR´97)
Degenerate dual quadric *
Encodes both absolute conic  and 
 *


for metric frame:
I
π  T
0
T
0
π0

0
Absolute Dual Quadric and
Self-calibration
Eliminate extrinsics from equation
Equivalent to projection of dual quadric
Abs.Dual Quadric also exists in projective world
KK T  PΩ*PT  (PT 1 )( TΩ*TT )( T TPT )
 P´Ω´* P´T
Transforming world so that Ω´  Ω
reduces ambiguity to metric
*
*
Absolute Dual Quadric
and Self-calibration
Projection equation:

i

ω  PiΩ P  K iK
T
i
Translate constraints on K
through projection equation
to constraints on *
*
T
i
*
Absolute conic = calibration object which is
always present but can only be observed
through constraints on the intrinsics
Constraints on *
 f x2  s 2  cx2

ω*   sf y  cx c y

cx

condition
sf y  cx c y
f y2  c 2y
constraint
cy
type
cx 

cy 
1 
#constraints
Zero skew
*
*
ω12
ω*33  ω13
ω*23
quadratic
m
Principal point
*
ω13
 ω*23  0
linear
2m
Zero skew (& p.p.)
ω 0
*
ω ω'*22  ω*22ω'11
linear
m
Fixed aspect ratio
(& p.p.& Skew)
Known aspect ratio
(& p.p.& Skew)
Focal length
(& p.p. & Skew)
*
12
*
11
quadratic
m-1
*
ω11
 ω*22
linear
m
*
ω*33  ω11
linear
m
Linear algorithm
(Pollefeys et al.,ICCV´98/IJCV´99)
Assume everything known, except focal length
 fˆ

*
ω  0
0

2
0
fˆ 2
0
0

0  P*PT
1
PΩ P   PΩ P 
PΩ P   0
PΩ P   0
PΩ P   0


T
11
T
22

T
12

T
13

T
23
Yields 4 constraint per image
Note that rank-3 constraint is not enforced
0
Linear algorithm revisited
(Pollefeys et al., ECCV‘02)
Weighted linear equations
1
 T
 T
2
ˆ



P
Ω
P

P
Ω
P 22  0
f

0 0
11
0 .2
1


 T
T
2
ˆ

P
Ω
P 12  0
KK   0 f
0
0.01
1
 T
0


P
Ω
P 13  0
0
1
0 .1


1
 T

P
Ω
P 23  0
0 .1
1
9
1
9
PΩ P   PΩ P 
PΩ P   PΩ P 


T
11


T
fy
c x  0  0.1
c y  0  0.1
0
33  0
33
22
assumptions
log( fˆˆ )  log(1)  log(3)
f
log( ˆx )  log(1)  log(1.1)
T
s0
T
Projective to metric
Compute T from
~
~ I
* T
-1~ - T
*
I  TΩT or T I T  Ω with I   T
using eigenvalue decomposition of
0
*
Ω

and then obtain metric reconstruction as
-1
PT and TM
0
0
Alternatives:
(Dual) image of absolute conic
• Equivalent to Absolute Dual Quadric
ω  PΩ P
*

ω*  Hω*HT
*

T
(H  H  ea)
• Practical when H can be computed first
– Pure rotation (Hartley’94, Agapito et al.’98,’99)
– Vanishing points, pure translations, modulus
constraint, …
Note that in the absence of skew the IAC
can be more practical than the DIAC!
 f x2  cx2

*
ω   c x c y
 cx
 f y2
1 
ω  2 2  0
fx f y
 f y2cx
cx c y
f y2  c y2
cy
0
f x2
 f x2c y
cx 

cy 
1


 f y2cx

 f x2c y

f x2 f y2  f y2cx2  f x2c y2 
Kruppa equations
e' ω e'
 e' H ω H e'  Fω*FT
T
*
 to

 epipolar
equations
*
  
geometry
Limit
Only 2 independent equations per pair
But independent of plane at infinity
T

T

Refinement
• Metric bundle adjustment
Enforce constraints or priors
on intrinsics during minimization
(this is „self-calibration“ for photogrammetrist)
Outline
•
•
•
•
Introduction
Self-calibration
Dual Absolute Quadric
Critical Motion Sequences
Critical motion sequences
(Sturm, CVPR´97, Kahl, ICCV´99, Pollefeys,PhD´99)
• Self-calibration depends on camera motion
• Motion sequence is not always general enough
• Critical Motion Sequences have more than one
potential absolute conic satisfying all constraints
• Possible to derive classification of CMS
Critical motion sequences:
constant
intrinsic
parameters
Most important cases for constant intrinsics
Critical motion type
ambiguity
pure translation
affine transformation (5DOF)
pure rotation
arbitrary position for  (3DOF)
orbital motion
proj.distortion along rot. axis (2DOF)
planar motion
scaling axis  plane (1DOF)
Note relation between critical motion sequences and
restricted motion algorithms
Critical motion sequences:
varying focal length
Most important cases for varying focal length
(other parameters known)
Critical motion type
ambiguity
pure rotation
arbitrary position for  (3DOF)
forward motion
proj.distortion along opt. axis (2DOF)
translation and
rot. about opt. axis
hyperbolic and/or
elliptic motion
scaling optical axis (1DOF)
one extra solution
Critical motion sequences:
algorithm dependent
Additional critical motion sequences can
exist for some specific algorithms
– when not all constraints are enforced
(e.g. not imposing rank 3 constraint)
– Kruppa equations/linear algorithm: fixating a point
Some spheres also project to
circles located in the image and
hence satisfy all the linear/kruppa
self-calibration constraints
Non-ambiguous new views for CMS
(Pollefeys,ICCV´01)
• restrict motion of virtual camera to CMS
• use (wrong) computed camera parameters