Diapositiva 1 - Politecnico di Milano
Download
Report
Transcript Diapositiva 1 - Politecnico di Milano
Self-calibration
Outline
•
•
•
•
Introduction
Self-calibration
Dual Absolute Quadric
Critical Motion Sequences
Motivation
• Avoid explicit calibration procedure
– Complex procedure
– Need for calibration object
– Need to maintain calibration
Motivation
• Allow flexible acquisition
– No prior calibration necessary
– Possibility to vary intrinsics
– Use archive footage
Example
Projective ambiguity
Reconstruction from uncalibrated images
projective ambiguity on reconstruction
m P M (PT 1)(T M) P´M´
Stratification of geometry
Projective
15 DOF
Affine
Metric
7 DOF
12 DOF
plane at infinity absolute conic
parallelism
angles, rel.dist.
More general
More structure
Constraints ?
• Scene constraints
– Parallellism, vanishing points, horizon, ...
– Distances, positions, angles, ...
Unknown scene no constraints
• Camera extrinsics constraints
–Pose, orientation, ...
Unknown camera motion no constraints
• Camera intrinsics constraints
–Focal length, principal point, aspect ratio & skew
Perspective camera model too general
some constraints
Euclidean projection matrix
Factorization of Euclidean projection matrix
P K R T
R T t
fx
Intrinsics: K
Extrinsics:
R, t
s
fy
ux
uy
1
(camera geometry)
(camera motion)
Note: every projection matrix can be factorized,
but only meaningful for euclidean projection matrices
Constraints on intrinsic
parameters
fx
K
Constant
s
fy
ux
u y
1
e.g. fixed camera:
Known
e.g. rectangular pixels:
square pixels:
principal point known:
K1 K2
s 0
f x fy , s 0
w h
ux ,uy ,
2 2
Self-calibration
Upgrade from projective structure to metric
structure using constraints on intrinsic
camera parameters
– Constant intrinsics
(Faugeras et al. ECCV´92, Hartley´93,
Triggs´97, Pollefeys et al. PAMI´98, ...)
– Some known intrinsics, others varying
(Heyden&Astrom CVPR´97, Pollefeys et al. ICCV´98,...)
– Constraints on intrincs and restricted motion
(e.g. pure translation, pure rotation, planar motion)
(Moons et al.´94, Hartley ´94, Armstrong ECCV´96, ...)
A counting argument
• To go from projective (15DOF) to metric (7DOF) at
least 8 constraints are needed
• Minimal sequence length should satisfy
n # known n 1 # fixed 8
• Independent of algorithm
• Assumes general motion (i.e. not critical)
Self-calibration:
conceptual algorithm
Given projective structure and motion {Pj,Mi},
then the metric structure and motion can be
obtained as {PjT-1,TMi}, with
T arg min C K P1T 1 , K P2T 1 ,, K Pn T 1
T
C K 1 , K 2 ,, K n criterium expressing constraints
K (P) function extracting intrinsics
from projection matrix
Outline
•
•
•
•
Introduction
Self-calibration
Dual Absolute Quadric
Critical Motion Sequences
Conics & Quadrics
conics
m Cm 0
T
quadrics
l Cl0
T
C C
*
*
MTQM 0
TQ * 0
Q * Q 1
1
transformations
C C´~ HTCH1
Q Q´~ T TQT 1
C* C*´~ HC*HT
Q * Q *´~ TQ *T T
projection
C* ~ PQ*PT
The Absolute Dual
Quadric (Triggs CVPR´97)
Degenerate dual quadric *
Encodes both absolute conic and
*
for metric frame:
I
π T
0
T
0
π0
0
Absolute Dual Quadric and
Self-calibration
Eliminate extrinsics from equation
Equivalent to projection of dual quadric
Abs.Dual Quadric also exists in projective world
KK T PΩ*PT (PT 1 )( TΩ*TT )( T TPT )
P´Ω´* P´T
Transforming world so that Ω´ Ω
reduces ambiguity to metric
*
*
Absolute Dual Quadric
and Self-calibration
Projection equation:
i
ω PiΩ P K iK
T
i
Translate constraints on K
through projection equation
to constraints on *
*
T
i
*
Absolute conic = calibration object which is
always present but can only be observed
through constraints on the intrinsics
Constraints on *
f x2 s 2 cx2
ω* sf y cx c y
cx
condition
sf y cx c y
f y2 c 2y
constraint
cy
type
cx
cy
1
#constraints
Zero skew
*
*
ω12
ω*33 ω13
ω*23
quadratic
m
Principal point
*
ω13
ω*23 0
linear
2m
Zero skew (& p.p.)
ω 0
*
ω ω'*22 ω*22ω'11
linear
m
Fixed aspect ratio
(& p.p.& Skew)
Known aspect ratio
(& p.p.& Skew)
Focal length
(& p.p. & Skew)
*
12
*
11
quadratic
m-1
*
ω11
ω*22
linear
m
*
ω*33 ω11
linear
m
Linear algorithm
(Pollefeys et al.,ICCV´98/IJCV´99)
Assume everything known, except focal length
fˆ
*
ω 0
0
2
0
fˆ 2
0
0
0 P*PT
1
PΩ P PΩ P
PΩ P 0
PΩ P 0
PΩ P 0
T
11
T
22
T
12
T
13
T
23
Yields 4 constraint per image
Note that rank-3 constraint is not enforced
0
Linear algorithm revisited
(Pollefeys et al., ECCV‘02)
Weighted linear equations
1
T
T
2
ˆ
P
Ω
P
P
Ω
P 22 0
f
0 0
11
0 .2
1
T
T
2
ˆ
P
Ω
P 12 0
KK 0 f
0
0.01
1
T
0
P
Ω
P 13 0
0
1
0 .1
1
T
P
Ω
P 23 0
0 .1
1
9
1
9
PΩ P PΩ P
PΩ P PΩ P
T
11
T
fy
c x 0 0.1
c y 0 0.1
0
33 0
33
22
assumptions
log( fˆˆ ) log(1) log(3)
f
log( ˆx ) log(1) log(1.1)
T
s0
T
Projective to metric
Compute T from
~
~ I
* T
-1~ - T
*
I TΩT or T I T Ω with I T
using eigenvalue decomposition of
0
*
Ω
and then obtain metric reconstruction as
-1
PT and TM
0
0
Alternatives:
(Dual) image of absolute conic
• Equivalent to Absolute Dual Quadric
ω PΩ P
*
ω* Hω*HT
*
T
(H H ea)
• Practical when H can be computed first
– Pure rotation (Hartley’94, Agapito et al.’98,’99)
– Vanishing points, pure translations, modulus
constraint, …
Note that in the absence of skew the IAC
can be more practical than the DIAC!
f x2 cx2
*
ω c x c y
cx
f y2
1
ω 2 2 0
fx f y
f y2cx
cx c y
f y2 c y2
cy
0
f x2
f x2c y
cx
cy
1
f y2cx
f x2c y
f x2 f y2 f y2cx2 f x2c y2
Kruppa equations
e' ω e'
e' H ω H e' Fω*FT
T
*
to
epipolar
equations
*
geometry
Limit
Only 2 independent equations per pair
But independent of plane at infinity
T
T
Refinement
• Metric bundle adjustment
Enforce constraints or priors
on intrinsics during minimization
(this is „self-calibration“ for photogrammetrist)
Outline
•
•
•
•
Introduction
Self-calibration
Dual Absolute Quadric
Critical Motion Sequences
Critical motion sequences
(Sturm, CVPR´97, Kahl, ICCV´99, Pollefeys,PhD´99)
• Self-calibration depends on camera motion
• Motion sequence is not always general enough
• Critical Motion Sequences have more than one
potential absolute conic satisfying all constraints
• Possible to derive classification of CMS
Critical motion sequences:
constant
intrinsic
parameters
Most important cases for constant intrinsics
Critical motion type
ambiguity
pure translation
affine transformation (5DOF)
pure rotation
arbitrary position for (3DOF)
orbital motion
proj.distortion along rot. axis (2DOF)
planar motion
scaling axis plane (1DOF)
Note relation between critical motion sequences and
restricted motion algorithms
Critical motion sequences:
varying focal length
Most important cases for varying focal length
(other parameters known)
Critical motion type
ambiguity
pure rotation
arbitrary position for (3DOF)
forward motion
proj.distortion along opt. axis (2DOF)
translation and
rot. about opt. axis
hyperbolic and/or
elliptic motion
scaling optical axis (1DOF)
one extra solution
Critical motion sequences:
algorithm dependent
Additional critical motion sequences can
exist for some specific algorithms
– when not all constraints are enforced
(e.g. not imposing rank 3 constraint)
– Kruppa equations/linear algorithm: fixating a point
Some spheres also project to
circles located in the image and
hence satisfy all the linear/kruppa
self-calibration constraints
Non-ambiguous new views for CMS
(Pollefeys,ICCV´01)
• restrict motion of virtual camera to CMS
• use (wrong) computed camera parameters