Transcript Lecture 8 Pose estimation by Gaussian Process

EE4-62 MLCV
Lecture 15-16
Pose Estimation – Gaussian
Process
Tae-Kyun Kim
1
EE4-62 MLCV
MS KINECT
http://www.youtube.com/watch?v=p2qlHoxPioM
The KINECT body pose estimation is achieved by
randomised regression forest techniques.
2
EE4-62 MLCV
We learn pose estimation as a regression
problem, and Gaussian process as a cutting edge
regression method.
We see it through the case study (slide credits to):
Semi-supervised Multi-valued Regression,
Navaratnam, Fitzgibbon, Cipolla, ICCV07,
where practical challenges addressed are
1)multi-valued regression and 2)sparsity of data.
3
EE4-62 MLCV
A mapping function is learnt from the input image I to the pose vector θ , which
is taken as a continuous variable.
4
Given an image (top left) , e.g. a silhouette (top right) is obtained by background
subtraction techniques (http://en.wikipedia.org/wiki/Background_subtraction).
The estimated 3d pose is shown at two camera angels (bottom left and right).
5
EE4-62 MLCV
We attempt to map 2D image space to 3D pose space.
There is inherent ambiguity in pose estimation (as an example in the above).
6
EE4-62 MLCV
Eye tracking can be tackled as a regression problem, where
the input is an image I and the output is a eye location.
7
EE4-62 MLCV
Typical image processing steps:
Given an image, a silhouette is segmented.
A shape descriptor is applied to the silhouette to yield a finite
dimensional vector. (Belongie and Malik, Matching with Shape Contexts, 2000)
8
EE4-62 MLCV
The output is a vector of m joint angles.
9
EE4-62 MLCV
10
EE4-62 MLCV
11
EE4-62 MLCV
12
EE4-62 MLCV
13
EE4-62 MLCV
14
EE4-62 MLCV
EE4-62 MLCV
Gaussian Process
15
EE462 MLCV
Review of Gaussian density estimation
(lecture 1,2)
i.i.d.
(independent
identical
distributed)
We want to find the Gaussian parameters from the given data.
The problem is to find the parameters by maximising the
posterior probability
2
2
𝑝
𝐱
𝜇,
𝜎
𝑝
𝜇,
𝜎
𝑝 𝜇, 𝜎 2 𝐱 =
𝑝(𝐱)
16
EE462 MLCV
Review of Gaussian density estimation
(lecture 1,2)
2
2
𝑝
𝐱
𝜇,
𝜎
𝑝
𝜇,
𝜎
𝑝 𝜇, 𝜎 2 𝐱 =
𝑝(𝐱)
We do not have priors on the parameters and data, thus we
maximise the (log) likelihood function instead.
Maximum Likelihood (ML) vs Maximum A Posterior (MAP)
solutions:
P(X|Y)P(Y)
P(X|Y)
(e.g. Gaussian Process)
17
EE462 MLCV
Review of polynomial curve fitting
(lecture 1,2)
We want to fit a polynomial curve to given data pairs (x,t).
= 𝐰𝑇 𝐱
where
𝑤0
𝐰= 𝑤1
⋮
1
𝑥
𝐱= 𝑥 2
⋮
The objective ftn to minimise is
EE4-62 MLCV
Gaussian Processes
19
EE4-62 MLCV
20
EE4-62 MLCV
𝐲~𝑁(𝐲|𝟎, 𝐊)
21
EE4-62 MLCV
𝐲(𝐱)
𝐱
22
EE4-62 MLCV
Gaussian Processes for Regression
23
EE4-62 MLCV
24
EE4-62 MLCV
25
EE4-62 MLCV
26
EE4-62 MLCV
𝑡 =𝑦 𝑥 +𝜖
27
EE4-62 MLCV
𝑁×𝑁 𝑁×1
28
EE4-62 MLCV
29
30
EE4-62 MLCV
31
Gaussian Process Matlab Toolbox
http://www.lce.hut.fi/research/mm/gpstuff/i
nstall.shtml
(try demo_regression_robust.m,
demo_regression1.m)
32
EE4-62 MLCV
Learning Hyperparameters
33
34
EE4-62 MLCV
Automatic Relevance Determination
35
36
37
Back to the pose estimation
problem
38
39
EE4-62 MLCV
• It’ll never work…
–
is multivalued
–
and
live in high dimensions
40
EE4-62 MLCV
41
EE4-62 MLCV
42
EE4-62 MLCV
43
EE4-62 MLCV
44
EE4-62 MLCV
45
EE4-62 MLCV
46
EE4-62 MLCV
GMM : Gaussian Mixture Model
GPLVM : Gaussian Process Latent Variable Model
47
EE4-62 MLCV
𝑝 𝜃𝑧 =
𝑝(𝜃, 𝑧)
𝑝(𝑧)
𝑝 𝑧
= ∫ 𝑝 𝜃, 𝑧 𝑑𝜃
48
EE4-62 MLCV
49
EE4-62 MLCV
50
EE4-62 MLCV
51
EE4-62 MLCV
52
EE4-62 MLCV
53
EE4-62 MLCV
54
EE4-62 MLCV
55
EE4-62 MLCV
56
EE4-62 MLCV
57
EE4-62 MLCV
58
EE4-62 MLCV
59
EE4-62 MLCV
60
EE4-62 MLCV
t: a continuous
latent variable
𝑝 𝑧, 𝜃 = ∫ 𝑝 𝑧, 𝜃 𝑡 𝑝 𝑡 𝑑𝑡
61
EE4-62 MLCV
62
EE4-62 MLCV
63
EE4-62 MLCV
64
EE4-62 MLCV
65
EE4-62 MLCV
66
EE4-62 MLCV
67
EE4-62 MLCV
68
EE4-62 MLCV
69
EE4-62 MLCV
70
EE4-62 MLCV
Real-Time Human Pose Recognition in Parts
from Single Depth Images (J. Shotton et al, 2011)
• Key features
– Depth image as input
– Real-time by Random Forest, and Part-based
71
EE4-62 MLCV
Progressive Search Space Reduction for
Human Pose Estimation (Ferrari et al, 2008)
72
EE4-62 MLCV