Transcript Corner Detection - University of Delaware

Going Back a little
• Cameras.ppt
Computer Vision : CISC 4/689
Applications of RANSAC:
Solution for affine parameters
• Affine transform of [x,y] to [u,v]:
• Rewrite to solve for transform parameters:
Computer Vision : CISC 4/689
Assignment
• Program-1
• info-Link
• Data
Note:
You can generate, bring-in, your own images from www, as long as:
For n+1 levels, image must be Mr£2n+1 rows by Mc£2n+1 cols
Mr and Mc are any +ve integers
Sunday 10pm
Computer Vision : CISC 4/689
Another app. : Automatic Homography H
Estimation
– Homographies describe image transformation of...
• General scene when camera motion is rotation about camera center
• Planar surfaces under general camera motion
•
How to get correct correspondences without human intervention?
from Hartley & Zisserman
Computer Vision : CISC 4/689
Computing a Homography
Lets Side-track
• 8 degrees of freedom in 3 x 3 matrix H, so at least n
points are sufficient to determine it
= 4 pairs of 2-D
• Use same basic algorithm for P (aka Direct Linear Transformation, or DLT)
to compute H
– Now stacked matrix A is 2n x 9 vs. 2n x 12 for camera matrix P estimation
because all points are 2-D
• 3 collinear points in either image is a degenerate configuration preventing a
unique solution
Computer Vision : CISC 4/689
Estimating H: DLT Algorithm
• x0i = Hxi is an equation involving homogeneous vectors, so Hxi and
x0i need only be in the same direction, not strictly equal
• We can specify “same directionality” by using a cross product formulation:
• See Hartley & Zisserman, Chapter 3.1-3.1.1 (linked on course page) for
details
Computer Vision : CISC 4/689
Texture Mapping
•
Needed for nice display when applying transformations (like a homography H) to a
whole image
•
Simple approach: Iterate over source image coordinates and apply x0
get destination pixel location
= Hx to
– Problem: Some destination pixels may not be “hit”, leaving holes
•
Easy solution: Iterate over destination image and apply inverse transform x
x0
– Round off H-1 x0 to address “nearest” source pixel value
– This ensures every destination pixel is filled in
Computer Vision : CISC 4/689
= H-1
Automatic H Estimation:
Feature Extraction
•
Find features in pair of images using corner detection—e.g., eigenvalue threshold of:
from Hartley & Zisserman
Computer Vision : CISC 4/689
~500 features found
Automatic H Estimation:
Finding Feature Matches
• Best match over threshold within square search window (here §300
pixels) using SSD or normalized cross-correlation
from Hartley & Zisserman
Computer Vision : CISC 4/689
Automatic H Estimation:
Finding Feature Matches
• Best match over threshold within square search window (here §300
pixels) using SSD or normalized cross-correlation
from Hartley & Zisserman
Computer Vision : CISC 4/689
Automatic H Estimation:
Initial Match Hypotheses
from Hartley & Zisserman
268 matched features (over SSD threshold) in left image
pointing to locations of corresponding right image features
Computer Vision : CISC 4/689
Automatic H Estimation:
Applying RANSAC
• Sampling
– Size: Recall that 4 correspondences suffice to define homography, so sample size
s=4
– Choice
• Pick SSD threshold conservatively to minimize bad matches
• Disregard degenerate configurations
• Ensure points have good spatial distribution over image
• Distance measure
– Obvious choice is symmetric transfer error:
Computer Vision : CISC 4/689
Automatic H Estimation:
Outliers & Inliers after RANSAC
• 43 samples used with t = 1.25 pixels
from Hartley & Zisserman
117 outliers (²
Computer Vision : CISC 4/689
= 0.44)
151 inliers
A Short Review of Camera Calibration
Computer Vision : CISC 4/689
Pinhole Camera Terminology
Image plane
Optical axis
Principal point/
image center
Focal length
Camera center/
pinhole
Camera point
Image point
Computer Vision : CISC 4/689
Calibration
• Slides (calibration.ppt)
Computer Vision : CISC 4/689
Calibration and Pose estimation example
• Recover intrinsic and extrinsic parameters of camera by
using calibration board.
• 3D points are given, can find 2D image coordinates for the
corresponding 3D points.
• Assume world is located at the folded lower corner,
principal point is center of the image, fold is 90 degrees,
• Total length and width of board is 9in by 9in.
Next 8 slides, courtesy UCF.
Computer Vision : CISC 4/689
Matlab code
•
Matlab
• fx = 1.5031
• fy =1.2773
• Rc =
-0.0201 -0.2000 -0.9796
0.2198 0.9588 -0.1797
0.9752 -0.2189 0.0247
• Tc = (29.0725, -2.8850, 53.4196)
• camera position:( -51.0289, 19.7118 26.6985)
Computer Vision : CISC 4/689
Multi-View Geometry
Relates
• 3D World Points
• Camera Centers
• Camera Orientations
Computer Vision : CISC 4/689
Multi-View Geometry
Relates
• 3D World Points
• Camera Centers
• Camera Orientations
• Camera Intrinsic Parameters
• Image Points
Computer Vision : CISC 4/689
Stereo
scene point
image plane
optical center
Computer Vision : CISC 4/689
Stereo
• Basic Principle: Triangulation
– Gives reconstruction as intersection of two rays
– Requires
• calibration
• point correspondence
Computer Vision : CISC 4/689
Stereo Constraints
p’ ?
p
Given p in left image, where can the corresponding point p’
in right image be?
Computer Vision : CISC 4/689
Stereo Constraints
M
Image plane
Y1
Epipolar Line
p
p’
Y2
Z1
O1
X2
X1
Focal plane
O2
Epipole
Computer Vision : CISC 4/689
Z2
Stereo
• The geometric information that relates two different
viewpoints of the same scene is entirely contained in a
mathematical construct known as fundamental matrix.
• The geometry of two different images of the same scene is
called the epipolar geometry.
Computer Vision : CISC 4/689
Stereo/Two-View Geometry
•
•
The relationship of two views of a
scene taken from different camera
positions to one another
Interpretations
– “Stereo vision” generally means
two synchronized cameras or eyes
capturing images
– Could also be two sequential views
from the same camera in motion
• Assuming a static scene
http://www-sop.inria.fr/robotvis/personnel/sbougnou/Meta3DViewer/EpipolarGeo
Computer Vision : CISC 4/689
3D from two-views
There are two ways of extracting 3D from a pair of images.
• Classical method, called Calibrated route, we need to calibrate both
cameras (or viewpoints) w.r.t some world coordinate system. i.e,
calculate the so-called epipolar geometry by extracting the essential
matrix of the system.
• Second method, called uncalibrated route, a quantity known as
fundamental matrix is calculated from image correspondences, and this
is then used to determine the 3D.
Either way, principle of binocular vision is triangulation. Given a single
image, the 3D location of any visible object point must lie on the
straight line that passes through COP and image point (see fig.).
Intersection of two such lines from two views is triangulation.
Computer Vision : CISC 4/689
Mapping Points between Images
• What is the relationship between the images x, x’ of the
scene point X in two views?
• Intuitively, it depends on:
– The rigid transformation between cameras (derivable from the
camera matrices P, P’)
– The scene structure (i.e., the depth of X)
• Parallax: Closer points appear to move more
Computer Vision : CISC 4/689
Example: Two-View Geometry
x3
x2
x’3
x1
x’2
x’1
courtesy of F. Dellaert
Is there a transformation relating the points
Computer Vision : CISC 4/689
xi to x’i ?
Epipolar Geometry
• Baseline: Line joining camera centers C, C’
• Epipolar plane ¦: Defined by baseline and scene point X
Computerbaseline
Vision : CISC 4/689
from Hartley
& Zisserman
Epipolar Lines
•
Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes
•
Epipoles e, e’: Where baseline intersects image planes
– Equivalently, the image in one view of the other camera center.
C’
C
Computer Vision : CISC 4/689
from Hartley
& Zisserman
Epipolar Pencil
•
As position of X varies, epipolar planes “rotate” about the baseline (like a book
with pages)
– This set of planes is called the epipolar pencil
•
Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines
Computer Vision : CISC 4/689
from Hartley
& Zisserman
Epipolar Constraint
•
•
•
Camera center C and image point define ray in 3-D space that projects to epipolar line l’ in other
view (since it’s on the epipolar plane)
3-D point X on this ray, so image of X in other view x’ must be on l’
In other words, the epipolar geometry defines a mapping
in the other
x ! l’, of points in one image to lines
x’
C’
C
from Hartley
& Zisserman
Computer Vision : CISC 4/689
Example: Epipolar Lines for Converging
Cameras
Left view
Right view
Intersection of epipolar lines = Epipole !
Indicates direction of other camera
Computer Vision : CISC 4/689
from Hartley
& Zisserman
Special Case:
Translation Parallel to Image Plane
Note that epipolar lines are parallel and corresponding points lie on correspondVision : CISC 4/689
ing epipolar lines (the latter is Computer
true for
all kinds of camera motions)
From Geometry to Algebra
P
p
p’
O’
O
Computer Vision : CISC 4/689
Courtesy, UCF
From Geometry to Algebra
P
p
p’
O’
O
Computer Vision : CISC 4/689
Linear Constraint:
Should be able to express as matrix
multiplication.
Computer Vision : CISC 4/689