OpenCV Introduction
Download
Report
Transcript OpenCV Introduction
OpenCV Introduction
Hang Xiao
Oct 26, 2012
History
1999 Jan : lanched by Intel, real time machine vision library for UI, optimized code for
intel
2000 Jun : OpenCV alpha 3。
2000 Dec : OpenCV beta 1 for linux
2006 : the first 1.0 version supports Mac OS
2008 mid : obtain corporate support from Willow Garage
2009 Sep : OpenCV 1.2(beta2.0
2009 Oct : Version 2.0 released。
2010 Dec : OpenCV 2.2。
2011 Aug : OpenCV 2.3。
2012 Apr : OpenCV 2.4.
Overview
Goals
Develop a universal toolbox for research and development in the field
of Computer Vision
Algorithms
More than 350 algorithms, 500 API
Programming language
C/C++, C#, Ch , Python, Ruby, Matlab, and Java (using JavaCV)
OS support
Windows, Android, Maemo, FreeBSD, OpenBSD, iOS, Linux and Mac
OS.
Licence
BSDlisence, free for commercial and non-commmercial
Overview - Applications
2D and 3D feature toolkits
Egomotion estimation
Facial recognition system
Gesture recognition
Human–computer interaction (HCI)
Mobile robotics
Motion understanding
Object identification
Segmentation and Recognition
Stereopsis Stereo vision: depth perception from 2 cameras
Structure from motion (SFM)Motion tracking
Overview - A statistical machine
learning library
Boosting (meta-algorithm)
Decision tree learning
Gradient boosting trees
Expectation-maximization algorithm
k-nearest neighbor algorithm
Naive Bayes classifier
Artificial neural networks
Random forest
Support vector machine (SVM)
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Image Analysis
Thresholds
Statistics
Pyramids
Morphology
Distance transform
Flood fill
Feature detection
Contours retrieving
Image Thresholding
Fixed threshold;
Adaptive
threshold;
Image Thresholding Examples
Source picture
Fixed threshold
Adaptive threshold
Statistics
min, max, mean value, standard
deviation over the image
Norms C, L1, L2
Multidimensional histograms
Spatial moments up to order 3 (central,
normalized, Hu)
Multidimensional Histograms
Histogram operations
calculation, normalization, comparison, back project
Histograms types:
Dense histograms
Signatures (balanced tree)
EMD algorithm
The EMD computes the distance between two distributions, which
are represented by signatures.
The signatures are sets of weighted features that capture the
distributions. The features can be of any type and in any number of
dimensions, and are defined by the user.
The EMD is defined as the minimum amount of work needed to
change one signature into the other
EMD – a method for the
histograms comparison
pi P, 1 i P , q j Q, 1 j Q , two historam s
EMD( P, Q)
f
i, j
ij
d ( pi , q j )
f
,
ij
i, j
f ij weightcoefficients,
d ( pi , q j ) the distancebetween the elem ents pi and q j .
Image Pyramids
Gaussian and
Laplacian pyramids
Image
segmentation by
pyramids
Image Pyramids
Gaussian and Laplacian
Pyramid-based color
segmentation
On still pictures
And on movies
Morphological Operations
Two basic morphology operations using structuring element:
erosion
dilation
More complex morphology operations:
opening : erosion + dilation
closing : dilation + erosion
morphological gradient : the difference between the dilation and the erosion of an
image
top hat : the difference between an input image and its opening
black hat : the difference between the closing and its input image
Opening
Closing
Top Hat
Morphological Gradient
Black Hat
Morphological Operations Examples
Morphology - applying Min-Max. Filters and its combinations
Image
I
Erosion IB
Closing I•B= (IB)B Grad(I)= (IB)-(IB)
Dilatation IB
Opening IoB= (IB)B
TopHat(I)= I - (IB) BlackHat(I)= (IB) - I
Distance Transform
Calculate the distance for all non-feature points to
the closest feature point
Two-pass algorithm, 3x3 and 5x5 masks, various
metrics predefined
Flood Filling
Simple
Gradient
Feature Detection
Fixed filters (Sobel operator, Laplacian);
Optimal filter kernels with floating point
coefficients (first, second derivatives, Laplacian)
Special feature detection (corners)
Canny operator
Hough transform (find lines and line segments)
Gradient runs
Canny Edge Detector
Hough Transform
Detects lines in a binary image
•Standard Hough
Transform
•Probabilistic Hough
Transform
Another Sample of the
Hough Transform Using
Source picture
Result
Contour Retrieving
The contour representation:
Chain code (Freeman code)
Polygonal representation
Initial Point
Chain code for the curve:
34445670007654443
Contour representation
Hierarchical representation of
contours
Image Boundary
(W1)
(B2)
(W2)
(B3)
(W3)
(B4)
(W5)
(W6)
Contours Examples
Source Picture
(300x600 = 180000 pts total)
Retrieved Contours
(<1800 pts total)
After Approximation
(<180 pts total)
And it is rather fast: ~70 FPS for 640x480 on complex scenes
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Structural Analysis
Contours processing
Approximation
Hierarchical representation
Shape characteristics
Matching
Geometry
Contour properties
Fitting with primitives
PGH: pair-wise geometrical histogram for the
contour.
Contour Processing
Approximation:
RLE algorithm (chain code)
Teh-Chin approximation (polygonal)
Douglas-Peucker approximation (polygonal);
Contour moments (central and normalized up to order 3)
Hierarchical representation of contours
Matching of contours
Hierarchical Representation
of Contours
A contour is represented with a binary tree
Given the binary tree, the contour can be retrieved with arbitrary precision
The binary tree is quasi invariant to translations, rotations and scaling
Contours matching
Matching based on hierarchical
representation of contours
Geometry
Properties of contours: (perimeter, area,
convex hull, convexity defects, rectangle of
minimum area)
Fitting: (2D line, 3D line, circle, ellipse)
Pair-wise geometrical histogram
Pair-wise geometrical
histogram (PGH)
p (i , j )
f PGH [ E r (1), E r ( 2), E r ( N ), Ec (1), Ec ( 2), Ec ( M )] T ,
E r (i )
j p(i, j) / p(i, j),
j
Ec ( j )
j
i p(i, j) / p(i, j).
i
i
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Object Recognition
Eigen objects
Hidden Markov Models
Eigen Objects
u i {u1i , u2i , uni } R n , 1 i m, m n,
C {cij } cov ariance matrix,
cij
m
(uli ul ) (ulj ul ), ul 1 / m
l
eli 1 / i
ulk ,
k 1
vki (ulk ul ),
k
e i {e1i , e1i ,, eni }, i 1, m1 m eigen basis,
i and i { 1i , 2i ,, mi } eigen values and eigen vectors.
.
Eigen objects (continued)
Hidden Markov Model
Definitions
S {s1, s2 ,...,sN }
- The set of states
O {o1, o2 ,...,oM }
- The set of measurements
qt O
- The state at time t
A {aij \ aij P(qt s j | qt 1 si )}
B {bij \ bij P(oi | s j )}
{ i \ i P(q0 si )}
- The transition probability matrix
- The conditional probability
matrix
- The starting states distribution
Embedded
HMM for Face
Recognition
Model-
- Face ROI partition
Face recognition
using Hidden Markov Models
One person – one HMM
Stage 1 – Train every HMM
1
…
n
Stage 2 – Recognition
i
Pi
- probability
Choose max(Pi)
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
Motion Analysis and
Object Tracking
Background subtraction
Motion templates
Optical flow
Active contours
Estimators
Background Subtraction
Background model (normal distribution)
Background statistics functions:
Average
Standard deviation
Running average
Motion Templates
Object silhouette
Motion history images
Motion history gradients
Motion segmentation algorithm
silhouette
MHI
MHG
Motion Segmentation
Algorithm
Two-pass algorithm labeling all motion segments
Motion Templates Example
Motion templates allow
to retrieve the dynamic
characteristics of the
moving object
Optical Flow
Block matching technique
Horn & Schunck technique
Lucas & Kanade technique
Pyramidal LK algorithm
6DOF (6 degree of freedom) algorithm
Optical flow equations:
I ( x dx, y dy, t dt) I ( x, y , t );
I / t I / x ( dx / dt) I / y ( dy / dt);
G X b,
X (x, y ), G
I x2 , I x I y
, b
I x I y , I y2
I x
It
I y
Pyramidal Implementation of the
optical flow algorithm
Image Pyramid
Representation
J image
I image
Iterative Lucas – Kanade
Scheme
Location of point u on image uL=u/2L
Spatial gradient matrix
G
I x2 , I x I y
I x I y , I y2
Standard Lucas – Kanade scheme for
optical flow computation at level L dL
Generic Image
Guess for next pyramid level L – 1
g L1 2( g L d L )
Finally,
V U d
(L-1)-th Level
Image pyramid building
L-th Level
d d 0 g0
Optical flow computation
6DOF Algorithm
Parametrical optical flow equations:
X (s ).
I I / s I / X / s
N
i 1 ROI
N
I iT I i ds
i 1 ROI
I t I iT
i
Active Contours
Snake energy:
Internal energy:
External energy:
Two external energy types:
E Eint Eext
Eint Econt Ecurv
Eext Eimg Econ
Eimg I ,
Eimg grad( I ) ,
E Econt Ecurv Eimg min
Estimators
Kalman filter
ConDensation filter
Kalman object tracker
Outline
Image Analysis
Structural Analysis
Object Recognition
Motion Analysis and Object Tracking
3D Reconstruction
3D reconstruction
Camera Calibration
View Morphing
POSIT
Camera Calibration
Define intrinsic and extrinsic camera parameters.
Define Distortion parameters
p A[ RT ]P,
f x 0 cx
r11 r12 r13
t1
A 0 f y c y , R r21 r22 r23 , T t2 , P [ X , Y , Z ], p [u, v ]
0 0 1
r31 r32 r33
t3
u~ u (u c x ) [k1 r 2 k2 r 4 2 p1 y p2 ( r 2 / x 2 x )],
v~ v (v c ) [k r 2 k r 4 2 p x p ( r 2 / y 2 y )],
y
r2 x2 y2.
1
2
2
1
Camera Calibration
Now, camera calibration can be done by holding
checkerboard in front of the camera for a few seconds.
And after that you’ll get:
3D view of etalon
Un-distorted image
View Morphing
POSIT Algorithm
Perspective projection:
xi ( f / Zi ) X i , yi ( f / Zi ) Yi
Weak-perspective projection:
xi s X i , yi s Yi , s f / Z .
OpenCV Websites
http://opencv.org
OpenCV official webpage.
http://opencvlibrary.sourceforge.net/
OpenCV documentation and FAQs.
OpenCV Examples
adaptiveskindetector : detect skin area
fback_c : dense Franeback optical flow
contours : calculate contours on different levels
delaunay : delaunay triangle
find_obj : SURF Detector and Descriptor using either FLANN or brute force
matching on planar objects
morphology : open/close, erode/dilate
motempl :motion templates
mser_sample : Maximal Extremal Region interest point detector
polar_transforms : illustrates Linear-Polar and Log-Polar image transforms
pyramid_segmentation : color pyramid segmentation
Thanks !!!