Transcript week11-frames.ppt

CV: 3D sensing and calibration
Coordinate system changes;
perspective transformation;
Stereo and structured light
MSU CSE 803 Fall 2008 Stockman
1
roadmap





using multiple cameras
using structured light projector
3D transformations
general perspective transformation
justification of 3x4 camera model
MSU CSE 803 Fall 2008 Stockman
2
Four Coordinate frames
W: world,
C,D: cameras,
M: object model
Need to relate all
to each other.
MSU CSE 803 Fall 2008 Stockman
3
Can we recognize?




Is there some object M
That can be placed in
some location
That will create the two
images that are
observed?
Discover/compute what
object and what pose
MSU CSE 803 Fall 2008 Stockman
4
Need to relate frames to compute



relate camera to world using rotations
and translations
project world point into real image
using projection
scale image point in real image plane
to get pixel array coordinates
MSU CSE 803 Fall 2008 Stockman
5
Stereo configuration
2 corresponding image
points enable the
intersection of 2 rays in W
MSU CSE 803 Fall 2008 Stockman
6
Stereo computation
MSU CSE 803 Fall 2008 Stockman
7
Math for stereo computations




need to calibrate both cameras to W so
that rays in x,y,z reference same space
need to have corresponding points
find point of closest approach of the
two rays
(rays are too far apart  point
correspondence error or crude
calibration)
MSU CSE 803 Fall 2008 Stockman
8
Replace camera with projector
Can calibrate a projector to W
easily. Correspondence now
means identifying marks.
MSU CSE 803 Fall 2008 Stockman
9
Advantages/disadvantages of
structured light
MSU CSE 803 Fall 2008 Stockman
10
Grid projected on objects
All grid
intersects are
integral
MSU CSE 803 Fall 2008 Stockman
11
Computing surface normals
Surface normals
have been
computed and
then added to the
image (augmented
reality)
MSU CSE 803 Fall 2008 Stockman
12
Relating coordinate frames




need to relate camera frame to world
need to rotate, translate, and scale
coordinate systems
need to project world points to the
image plane
all the above are modeled using 4x4
matrices and 1x4 points in
homogeneous coordinates
MSU CSE 803 Fall 2008 Stockman
13
Translation of 3D point P
Point in
frame 2
Point in 3D
parameters
MSU CSE 803 Fall 2008 Stockman
Point in
frame 1
14
Scaling 3D point P
MSU CSE 803 Fall 2008 Stockman
15
Rotation of P about the X-axis
MSU CSE 803 Fall 2008 Stockman
16
Rotate P about the Y-axis
MSU CSE 803 Fall 2008 Stockman
17
Rotate P about the Z-axis
Looks same as 2D rotation omitting row, col 4
MSU CSE 803 Fall 2008 Stockman
18
Arbitrary rotation has
orthonormal rows and columns
MSU CSE 803 Fall 2008 Stockman
19
Equivalent rotation form


every rotation or combination of
rotations is equivalent to ONE rotation
of angle theta about a single axis or
rotation
the axis of rotation and angle of
rotation can be determined if three
points are known before and after
MSU CSE 803 Fall 2008 Stockman
20
Computing angle-axis form
B
A*
O
w
A
To rotate A to A*, the axis
of rotation is OB that is
perpendicular to OA and
OA*
Compute OB = OA x OA*
(unit vectors)
Compute w from OA o OA*
(unit vectors)
See frame software
MSU CSE 803 Fall 2008 Stockman
21
Example: camera relative to world
MSU CSE 803 Fall 2008 Stockman
22
exercise




verify that the 3 x 3 rotation matrix is
orthonormal by checking 6 dot products
invert the 3 x 3 rotation matrix
invert the 4 x 4 matrix
verify that the new matrix transforms
points correctly from C to W
MSU CSE 803 Fall 2008 Stockman
23
Transformation “calculus”:
notation accounts for transforms
Destination
frame W
W
M
Origin frame M
T
Denotes
transformation
T transforms points
from model frame to
world frame. (Notation
from John Craig, 1986)
MSU CSE 803 Fall 2008 Stockman
24
Apply transformations to points
Point in model
coordinates
W
P =
Point in world
coordinates
W
M
T
M
P
Transformation from model
to world coordinates
(instance transformation)
MSU CSE 803 Fall 2008 Stockman
25
Matrix algebra enables composition




Let M and N be 4 x 4 matrices and let P be a
4 x 1 point
M(NP) = (MN)P
we can transform P using N and then
transform that by M, or we can multiply
matrices M and N and then apply that to
point P
matrix multiplication is associative (but not
commutative)
MSU CSE 803 Fall 2008 Stockman
26
Composing transformations
Projection parms.
Parameters:
rotation and
translation
cancel
A
W
T =
A
C
T(p)
C
W
T (w)
Two transformations are composed to get one
transformation that maps points from the world frame
to the frame A
MSU CSE 803 Fall 2008 Stockman
27
Use of transformation calculus
Alignment of two congruent triangles
(ABC and DEF) via coordinate
transformation;
Fundamental operation
MSU CSE 803 Fall 2008 Stockman
28
Align corresponding vertices at
origin and align first side
0) starting configurations
in 3D
1) Translate vertex to
origin (do both)
2) Rotate DE to X-axis
and AB to X-axis
MSU CSE 803 Fall 2008 Stockman
29
Alignment: last steps
3) Rotate point F
about X-axis into
XY-plane. Rotate
point C in a similar
manner.
Both triangles should
now overlap in 3D
MSU CSE 803 Fall 2008 Stockman
30
Here’s what was done
Every transformation we applied is invertible.
So, we get the inverses to apply to one side.
This is
W
M
T
MSU CSE 803 Fall 2008 Stockman
31
Align two congruent triangles in
3D space
MSU CSE 803 Fall 2008 Stockman
32
What use is it?





fundamental step in aligning a 3D model to observed
3D points
might be lucky to have 3 corners of a rectangular
box
could have any matching 3D feature points
once aligned, other 3D points could be transformed
the same way
See /user/stockman/Software/RST/Frames/frame.cpp
and look for function align_triangles
MSU CSE 803 Fall 2008 Stockman
33
Deriving form of the camera matrix
We have already described what the
camera matrix does and what form it
has; we now go through the steps to
justify it
MSU CSE 803 Fall 2008 Stockman
34
Viewing model points M
What’s in front of the camera?
MSU CSE 803 Fall 2008 Stockman
35
Math for the steps
MSU CSE 803 Fall 2008 Stockman
36
Perspective transformation: camera
origin at the center of projection
This transformation
uses same units in 3D
as in the image plane
MSU CSE 803 Fall 2008 Stockman
37
Perspective projection: camera
origin in the real image plane
MSU CSE 803 Fall 2008 Stockman
38
Rigid transformation for change
of coordinate frame
MSU CSE 803 Fall 2008 Stockman
39
Relate camera frame to world frame
MSU CSE 803 Fall 2008 Stockman
40
Change scene units to pixels
To get into XV or GIMP
image coordinates!
MSU CSE 803 Fall 2008 Stockman
41
Final result






a 3x4 camera matrix
maps 2D image pixels to 3D rays
maps 3D rays to 2D image pixels
obtain matrix via calibration (easy)
obtain matrix via reasoning (hard)
do camera calibration exercise
MSU CSE 803 Fall 2008 Stockman
42