Transcript week13-SfM.ppt

Structure from Motion
A moving camera/computer
computes the 3D structure of the
scene and its own motion
CSE 803 Fall 2008 Stockman
1
Sensing 3D scene structure via
a moving camera
We now have two views
over time/space
compared to stereo
which has multiple
views at the same time.
CSE 803 Fall 2008 Stockman
2
Assumptions for now



The scene is rigid.
The scene may move or the camera
may move giving a sequence of 2 or
more 2D images
Corresponding 2D image points (Pi, Pj)
are available across the images
CSE 803 Fall 2008 Stockman
3
What can be computed


The 3D coordinates of the scene points
The motion of the camera
Camera sees
many frames
of 2D points
Rigid scene with
many 3D
interest points
From Jabara, Azarbayejani, Pentland
CSE 803 Fall 2008 Stockman
4
From 2D point correspondences,
compute 3D points and TR
CSE 803 Fall 2008 Stockman
5
applications


We can compute a 3D model of a
landmark from a video
We can compute the trajectory of the
sensor relative to the 3D object points
CSE 803 Fall 2008 Stockman
6
Use only 2D correspondences,
SfM can compute 3D jig pts
… up to one scale factor.
CSE 803 Fall 2008 Stockman
7
http://www1.cs.columbia.edu/~je
bara/htmlpapers/SFM/sfm.html
Jabara, Azarbayejani, Pentland
a) Two video frames with
corresponding 2D
interest points. 3D
points can be computed
from SfM method.
b) Some edges detected
from 2D gradients.
c) Texture mapping from
2D frames onto 3D
polyhedral model.
d) 3D model can be
viewed arbitrarily!
CSE 803 Fall 2008 Stockman
8
Virtual museums




Much work, and software, from about 10
years ago.
3D models, including shape and texture can
be made of famous places (Notre Dame, Taj
Mahal, Titanic, etc.) and made available to
those who cannot travel to see the real
landmark.
Theoretically, only quality video is required.
Usually, some handwork is needed.
CSE 803 Fall 2008 Stockman
9
SfM methods





Typically require careful mathematics
EX: from 5 matched points, get 10
equations to estimate 10 unknowns;
also a more popular 8 pt linear method
Methods must consider effects of noise
See Faugeras et al and ********
Methods can run in real time
CSE 803 Fall 2008 Stockman
10
Special mathematics



Epipolar geometry
Fundamental matrix: computed from a
pair of cameras
Essential matrix: specialization of
fundamental matrix when calibration is
available
CSE 803 Fall 2008 Stockman
11
Finishing the course: 2 options


HW7 on stereo
computation (Sat 6
Dec) (60%/7)
Final Exam (25%):
Monday 8 Dec
12:45-2:45


SfM report (6 Dec)
and demonstration.
Find existing code
and get it to work
on CSE 803 images
(60%/7 + 10%)
Final Exam (15%)
limited problems
CSE 803 Fall 2008 Stockman
12