PowerPoint - People
Download
Report
Transcript PowerPoint - People
Joint Noise Level Estimation from Personal Photo Collections
YiChang
1*,2
Shih
1
Kwatra
Vivek
Troy
1Google Research
1
Chinen
Hui
2MIT CSAIL
1
Fang
Sergey
*Internship work at Google
Goal
Contributions
ο§ Given a set of face images from the same person, taken
under different lighting and cameras, estimate the noise
levels in each image
ο§ Key observation: given two noisy images, the noise
levels are correlated if they share the same underlying
2
2
image content, since π1 β π2 = π£ππ[π°π,π ] β π£ππ[π°π,π ]
Overview
Starting from a face image collection:
ο§ Preprocess: geometrically and photometrically align the
images with affine transform and color match
ο§ We formulate the estimation as maximizing the joint
probability distribution between all imagesβ noise levels
ο§ The joint distribution is conditioned on the pair-wise
2
2
relative noise levels {πππ |πππ β ππ β ππ }. We use a twostage optimization that first estimates {πππ }, then {ππ }
ο§ π°π = π°ππππ + π, i.i.d, zero mean. π = noise level β π π‘π[π]
ο§ This is difficult because we cannot decouple π from π°π
1
Ioffe
ο§ Two-stage optimization:
ο Estimating {πππ }: We take a patch-based method. We
first find the patch correspondence between π°π and π°π ,
β
then find the best estimated relative noise {πππ } from
the patch pairs.
β
2
2
β
ο With {πππ }, estimate {ππ } by constraining ππ β ππ = πππ
Pair-wise Relative Noise πππ Estimation
Results
Ground Truth Experiment and Comparison
ο§ The two faces are not perfectly aligned
ο§ We show one example below with estimated
noise levels and denoised result using BM3D +
our method for noise parameter
ο§ Add synthetic Gaussian noise with different parameters
ο§ Compare estimated noise levels and denoised result by BM3D
q
β’ πππ = exp(βπ
ππ π1π β π2π
2
Input
BM3D
Input
y=x Line (Ground Truth)
Liu et. al. (Mean)
Metric-Q (Mean)
Our Method (Mean)
BM3D
Mean PSNR (Metric-Q)
Mean PSNR (Our Method)
Mean PSNR(Best BM3D)
30
I2
β
π12
=
Ο=23.2
Ο=14.5
π,π πππ πππ
π,π πππ
), confidence that (π, π) is a true correspondence
26
40.00
22
18
14
10
6
ο§ More subjects
38.00
37.00
36.00
35.00
33.00
2
Ο=38.2
39.00
34.00
2
Ο=23.9
β’ For computational efficiency, we selected the best 5 π s for each π
41.00
Mean PSNR
I1
ο§ Compute pair-wise relative noise by aggregating πππ :
q
Estimated Noise Sigma
ο§ We break down the image into patches, and
estimate the patch-wise relative noise
levels πππ by πππ β π£ππ[π1π ] β π£ππ[π2π ]
p
7
12
0.00
17
Synthetic noise
10.00
15.00
20.00
Mean Noise Sigma
True Noise Sigma
Clean image
5.00
Our method
Metric-Q
Best BM3D
Absolute Noise Level Estimation with Global Optimization
ο§ We estimate {ππ } conditioning on
ο§
2
ππ
2
πβ π π€ππ ππ
β
{πππ }
2
ππ
=argmin
β
β
π€ππ : similarity between two faces
β 2
πππ
User Study
2
ππ
β
πππ
ο§ Solving a linear system
2
ππ
ο§ The system is under-determined, up to adding
a constant number.
- option 1: assign some images to be zero noise
- option 2: assuming the collection contains clean
images, assign the least noisy one to be zero. We use this one for evaluations
ο§ Based on BM3D denoised result, decide which
one is preferable
ο§ Ran on 71 images, each is evaluated by 3 users
Input image
Our method
Metric-Q
35%
24%
41%
Ο=46.3
Ο=35
Metric-Q
Ο
PSNR
19.25
22.33
20.31
34.65
27
34.53
23
34.65
Selected References
C. Liu, R. Szeliski, S. Kang, C. Zitnick, and W. Freeman. Automatic estimation and removal of
noise from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence,
30(2), 2008
X. Zhu and P. Milanfar. Automatic parameter selection for denoising algorithms using a noreference measure of image content. IEEE Transactions on Image Processing, 19(12), 2010.
Acknowledgements
Ours
Equally good
Ο=9.1
Ο=27
We thank MIT Graphics and Vision group for helpful discussion. We would like to thank the
volunteers who participated in the user study.