Transcript ICCV-Meng-20140910

Matrix Factorization with
Unknown Noise
Deyu Meng
参考文献：
①Deyu Meng, Fernando De la Torre. Robust Matrix Factorization with Unknown
Noise. International Conference of Computer Vision (ICCV), 2013.
②Qian Zhao, Deyu Meng, Zongben Xu, Wangmeng Zuo, Lei Zhang. Robust principal
component analysis with complex noise, International Conference of Machine Learning
(ICML), 2014.
 Low-rank matrix factorization are widely used in computer vision.
Structure from Motion
(E.g.,Eriksson and Hengel ,2010)
Face Modeling
(E.g., Candes et al.,2012)
Photometric Stereo
(E.g., Zheng et al.,2012)
Background Subtraction
(E.g. Candes et al.,2012)
 Complete, clean data (or with Gaussian noise)
 SVD: Global solution
 Complete, clean data (or with Gaussian noise)
 SVD: Global solution
 There are always missing data
 There are always heavy and complex noise
L2 norm model
𝐖 ⊙ (𝐗 − 𝐔𝐕)
𝑭
Young diagram (CVPR, 2008)
 L2 Wiberg (IJCV, 2007)
 LM_S/LM_M (IJCV, 2008)
 SALS (CVIU, 2010)
 LRSDP (NIPS, 2010)
 Damped Wiberg (ICCV, 2011)
 Weighted SVD (Technometrics, 1979)
 WLRA (ICML, 2003)
 Damped Newton (CVPR, 2005)
 CWM (AAAI, 2013)
 Reg-ALM-L1 (CVPR, 2013)
Pros: smooth model, faster algorithm,
have global optimum for nonmissing data
Cons: not robust to heavy outliers
L2 norm model
𝐖 ⊙ (𝐗 − 𝐔𝐕)
𝑭
Young diagram (CVPR, 2008)
 L2 Wiberg (IJCV, 2007)
 LM_S/LM_M (IJCV, 2008)
 SALS (CVIU, 2010)
 LRSDP (NIPS, 2010)
 Damped Wiberg (ICCV, 2011)
 Weighted SVD (Technometrics, 1979)
 WLRA (ICML, 2003)
 Damped Newton (CVPR, 2005)
 CWM (AAAI, 2013)
 Reg-ALM-L1 (CVPR, 2013)
Pros: smooth model, faster algorithm,
have global optimum for nonmissing data
Cons: not robust to heavy outliers
L1 norm model
𝐖 ⊙ (𝐗 − 𝐔𝐕)
𝟏
 Torre&Black (ICCV, 2001)
 R1PCA (ICML, 2006)
 PCAL1 (PAMI, 2008)
 ALP/AQP (CVPR, 2005)
 L1Wiberg (CVPR, 2010, best paper award)
 RegL1ALM (CVPR, 2012)
Pros: robust to extreme outliers
Cons: non-smooth model, slow
algorithm, perform badly in
Gaussian noise data
 L2 model is optimal to Gaussian noise
 L1 model is optimal to Laplacian noise
 But real noise is generally neither Gaussian nor
Laplacian
Yale B faces:
…
Saturation and
shadow noise
Camera noise
We propose Mixture of Gaussian (MoG)
Universal approximation
property of MoG
Any continuous
distributions
MoG
(Maz’ya and Schmidt, 1996)
 E.g., a Laplace distribution can be equivalently expressed
as a scaled MoG
(Andrews and Mallows, 1974)
MLE Model
 Use EM algorithm to solve it!
 E Step:
 M Step:
𝐖 ⊙ (𝐗 − 𝐔𝐕)
𝟐
𝑭
Synthetic experiments
 Three noise cases
 Gaussian noise
 Sparse noise
 Mixture noise
 Six error measurements
What L2 and L1
methods optimize
Good measures to estimate
groundtruth subspace
Our method
L2 methods
L1 methods
Gaussian noise experiments
 MoG performs similar with L2 methods, better than L1 methods.
Sparse noise experiments
 MoG performs as good as the best L1 method, better than L2 methods.
Mixture noise experiments
 MoG performs better than all L2 and L1 competing methods
Why MoG is robust to outliers?
 L1 methods perform
well in outlier or heavy
noise cases since it is a
heavy-tail distribution.
 Through fitting the noise
as two Gaussians, the
obtained MoG distribution
is also heavy tailed.
Face modeling experiments
Explanation
Saturation and
shadow noise
Camera noise
Background Subtraction
Background Subtraction
Summary
 We propose a LRMF model with a Mixture of
Gaussians (MoG) noise
 The new method can well handle outliers like L1norm methods but using a more efficient way.
 The extracted noises are with certain physical
meanings
Thanks!