Transcript PCA Extension

PCA Extension
By Jonash
Outline
• Robust PCA
• Generalized PCA
– Clustering points on a line
– Clustering lines on a plane
– Clustering hyperplanes in a space
Robust PCA
• Rrbust Principal Component Analysis for
Computer Vision
– Fernando De la Torre
– Mochael J. Black
• CS, Brown University
PCA is Least-Square Fit
PCA is Least-Square Fit
Robust Statistics
• Recover the best fit for the majority of
the data
• Detect and reject outliers
Robust PCA
Robust PCA
Robust PCA
• Training images
Robust PCA
• Naïve PCA
• Simply reject
• Robust PCA
RPCA
• In traditional PCA, we minimize
• Σni = 0 (di – B BT di)2 = Σni = 0 (di - Bci)2
• EM PCA Limσ -> 0(D = BC + σ2I)
E-step C = (BTB)-1BTD
M-step B = DCT(CCT)-1
di
BBTdi
BTdi = ci
B
RPCA
• Xu and Yuille [1995] tries to minimize
– Σni = 1 [ V i(di – B ci)2 + n(1-Vi) ]
– Hard to solve (continuous + discrete)
RPCA
• Gabriel and Zamir [1979] tries to minimize
– Σni = 1 Σdp = 1 [ wpi(dpi – Bci)2]
– Impratical for high dimension
– “Low rank approximation of matrices by least
squares with any choice of weights” 1979
RPCA
• Idea is to use a robust function ρ
– Geman-McClureρ (x,σ) = x2/(x2 + σ2)
– Σni = 1 Σdp = 1 ρ[ (dpi–μp –Σkj = 1 bpjcji), σp]
– Approximated by local quadratic function
– Use gradient descent
– The rest is nothing but heuristics
RPCA
Robust PCA - Experiment
• 256 training images (120x160)
• Obtain 20 RPCA basis
•
3 hrs on 900MHz Pentinum III in Matlab
Outline
• Robust PCA
• Generalized PCA
– Clustering points on a line
– Clustering lines on a plane
– Clustering hyperplanes in a space
Generalized PCA
• Generalized Principal Component Analysis
– Rene Vidal
– Yi Ma
– Shankar Sastry
• UC Berkley and UIUC
GPCA
GPCA Example 1
GPCA Example 2
GPCA Example 3
GPCA Goals
1. # of subspaces and their dimension
2. Basis for subspace
3. Segmentation of data
GPCA Ideas
• Union of subspaces = certain polynomials
Outline
• Robust PCA
• Generalized PCA
– Clustering points on a line
– Clustering lines on a plane
– Clustering hyperplanes in a space
GPCA 1D Case
GPCA 1D Case Cont’d
GPCA 1D Case Cont’d
MN = n+1
unknowns
To have a unique solution,
rank(Vn) = n = Mn- 1
GPCA 1D Example
• n = 2 groups
• pn(x) = ( x – μ1) ( x – μ2)
• No polynomial of degree 1
• Infinite polynomial of degree 3
• pn(x) = x2 + c1x + c2 => Polynomial factor
Outline
• Robust PCA
• Generalized PCA
– Clustering points on a line
– Clustering lines on a plane
– Clustering hyperplanes in a space
GPCA 2D Case
• L j = { X = [x, y]T: bj1x + bj2y = 0 }
• (b11x + b12y = 0) or (b21x + b22y = 0)…
GPCA 2D Case Cont’d
• (b11x + b12y = 0) or (b21x + b22y = 0)…
• Pn(x) = (b11x + b12y)…(bn1x + bn2y) = 0
= Σck xn-k yk
GPCA 2D Case Cont’d
• Take n = 2 for example…
• p2(x) = (b11x + b12y)(b21x + b22y)
• ▽p2(x) = (b21x + b22y)b1 + (b11x + b12y)b2
, bj = [bj1, bj2] T
• if x ~ L1, then ▽p2(x) ~ b1, otherwise ~ b2
GPCA 2D Case Cont’d
•
Given that {yj ε Lj}, the normal vector of
Lj is bj ~ ▽pn(yj)
•
3 things…
1. Determine “ n ” as min{ j: rank(Vj) = j }
2. Solve cn for Vncn = 0
3. Find normal vector bj
Outline
• Robust PCA
• Generalized PCA
– Clustering points on a line
– Clustering lines on a plane
– Clustering hyperplanes in a space
GPCA Hyperplanes
• Still assume d1 = … = dn = d = D – 1
• Sj = { bjTx = bj1x1 + bj2x2 + … + bjDxD = 0}
GPCA Hyperplanes
MN = C(D+n-1, D)
GPCA Hyperplanes
GPCA Hyperplanes
• Since we know n, we can solve for ck
• ck => bk by ▽pn(x)
• If we know yj on each Sj, finding bj will
be easy
GPCA Hyperplanes
• One point yj on each hyperplane Sj
• Consider a random line L = t * v + x0
• Obtain yj by intersecting L and Sj
– yj = tj * v + x 0
– Find roots tj by … Pn(t v + xo)
GPCA Hyperplanes
•
Summarize
1. We want to find n to solve for c
2. To get b (normal) for each S, find ▽pn(x)
3. To get label j, solve pn(yj = tj * v + x0) = 0
One More Thing
One More Thing
• Previously we assume d1 = … =dn= D – 1
• Actually we cannot assume that…
• Please read section 4.2 & 4.3 … by yourself
– Discuss how to recursively reduce dimension