Transcript slides - University of Waterloo

Hashim Mir, Peter Xu, and Peter van Beek
University of Waterloo
• Digital cameras are equipped with contrast-based autofocus
• Contrast-detection autofocus makes use of a focus measure
• maps an image to a value that represents degree of focus of image
• Speed and accuracy of autofocus crucial to user satisfaction
• search for an image with maximal focus according to focus measure
• choice of focus measure is an important factor in performance of autofocus
February 4, 2014
Focus Measures for Digital Photography
2
• Previous studies on focus measures:
• small number of benchmarks images in their evaluation,
• directed at microscopy and not digital cameras, or
• based on ad hoc evaluation criteria
• Here:
• extensive empirical evaluation of focus measures for digital photography
• advocate using three standard statistical measures of performance—
precision, recall, and mean absolute error—as evaluation criteria
February 4, 2014
Focus Measures for Digital Photography
3
•
•
•
•
•
•
Introduction
Focus Measures
Related Work
Experimental Methodology
Experimental Results
Conclusion
February 4, 2014
Focus Measures for Digital Photography
4
• Many focus measures have been proposed in the literature
• Let f (x, y) be the luminance or grayscale at pixel (x, y) in an
image of size M × N pixels
• Example: squared gradient focus measure for an image
acquired when the lens is at position p is given by,
M 1N  2
 ( p) =
  ( f ( x , y  1) 
f ( x , y ))
2
x=0 y=0
February 4, 2014
Focus Measures for Digital Photography
5
(a)
February 4, 2014
(b)
Focus Measures for Digital Photography
6
(a)
February 4, 2014
(b)
Focus Measures for Digital Photography
7
• We evaluated more than 30 focus measures in our study
• The focus measures can be classified under six types:
1.
2.
3.
4.
5.
6.
first-order differentiation
second-order differentiation
image histogram
image statistics
correlation
data compression
February 4, 2014
Focus Measures for Digital Photography
8
• Example: squared gradient
• Example: focus measure based on 3×3 Sobel operators,
 1

2

  1
0
0
0
1

2

 1 
(horizontal)
February 4, 2014
1

0

  1
2
0
2
 1

0

 1 
(vertical)
Focus Measures for Digital Photography
9
• Example: focus measure based on 3×3 Laplacian operator,
1

1

  1
February 4, 2014
1
8
1
 1

1

 1 
Focus Measures for Digital Photography
10
• Example: variance focus measure is given by,
 ( p) =
1
MN
M 1N 1
  ( f ( x, y )   )
2
x=0 y=0
where µ is the mean luminance of the image
February 4, 2014
Focus Measures for Digital Photography
11
• Example: Vollath’s F4 focus measure is given by,
M  2 N 1
 ( p) =
  f ( x , y ) f ( x  1, y ) 
x=0 y=0
M  31 N  1
  f ( x , y ) f ( x  2, y )
x=0 y=0
February 4, 2014
Focus Measures for Digital Photography
12
setting
citation
conclusions
Groen, Young & Ligthart, 1985 squared gradient
Microscopy
applications
Firestone et al. 1991
vertical Brenner, variance
Santos et al. 1997
Vollath's F4, F5
Liu, Wang & Sun 2007
variance
Subbarao and Tyan 1998
Laplacian
Digital
Ng, Neow & Ang 2001
photography
Shih 2007
Yousefi et al. 2011
February 4, 2014
first-order 3×3 Sobel, variance
first-order 3×3 Sobel & Prewitt
Vollath's F4
Focus Measures for Digital Photography
13
• Step 1: Implemented a camera remote control application
• Step 2: Using app, gathered 25 sets of benchmark images
• cover a range of common photography settings
• 4,303 jpeg images (23 × 167 + 2 × 231, one for each focus position)
• Canon EOS 550D/Rebel T2i camera
February 4, 2014
Focus Measures for Digital Photography
14
• Step 3: Determined ground truth
• true lens position (or image) where an object is in best focus
• Step 4: Implemented more than 30 focus measures
• extensive survey of the literature
• Step 5: Given a benchmark set of images and a focus measure
• applied focus measure to each jpeg image in benchmark
• focus measure predicts where an object will be in best focus
• given predictions of focus measure and ground truth, calculated precision,
recall, and mean absolute error
February 4, 2014
Focus Measures for Digital Photography
15
ground truth
true
tp
(true positive)
fn
(false negative)
true
predicted
false
precision =
February 4, 2014
tp
tp  fp
false
fp
(false positive)
tn
(true negative)
recall =
Focus Measures for Digital Photography
tp
tp  fn
16
squared gradient
February 4, 2014
Laplacian
Focus Measures for Digital Photography
17
squared gradient
precision =
reca ll =
February 4, 2014
3
30
3
31
Laplacian
precision =
reca ll =
Focus Measures for Digital Photography
4
40
4
4 0
18
absolute error =
p  p
where
p is lens position of image in maximal focus
pis lens position that focus measure predicts
• MAE chosen over mean squared error
• MAE prefers errors that are occasionally large but small on average
• mean square error prefers errors that avoid large errors while still possibly
being quite unsatisfactory overall
February 4, 2014
Focus Measures for Digital Photography
19
Focus measure
Brenner
squared gradient
3 × 3 difference
3 × 3 Sobel
3 × 3 Scharr
3 × 3 Roberts
3 × 3 Prewitt
Gaussian
vertical sq. gradient
vertical Brenner
February 4, 2014
precision
100.00
100.00
98.00
98.00
98.00
98.00
98.00
95.00
91.33
91.33
recall
99.00
99.00
97.00
97.00
97.00
97.00
97.00
92.00
92.00
90.00
Focus Measures for Digital Photography
MAE
0.00
0.00
0.02
0.02
0.02
0.02
0.02
0.05
0.55
0.23
20
Focus measure
precision
Laplacian of Gaussian
98.00
5 × 5 vertical Sobel
96.67
5 × 5 Laplacian
96.67
5 × 5 cross Sobel
90.33
3 × 3 cross Sobel
89.67
3 × 3 vertical Sobel
88.73
3 × 3 Laplacian
85.00
5 × 5 horizontal Sobel
83.33
3 × 3 horizontal Sobel
73.27
February 4, 2014
recall
99.00
99.00
99.00
93.00
93.00
96.00
91.00
88.00
78.00
Focus Measures for Digital Photography
MAE
0.32
0.65
0.46
0.66
1.18
3.94
1.41
1.08
2.95
21
Focus measure
normalized variance
variance
threshold pixel count
threshold content
power
February 4, 2014
precision
43.57
32.33
4.24
2.13
0.53
recall
53.00
35.00
18.00
4.00
2.00
Focus Measures for Digital Photography
MAE
26.36
20.80
52.38
36.31
43.80
22
Focus measure
autocorrelation
Vollath's F4
Vollath's F5
February 4, 2014
precision
80.40
61.74
33.00
recall
99.00
87.00
35.00
Focus Measures for Digital Photography
MAE
26.87
24.60
19.00
23
• Results:
• some popular focus measures perform poorly when applied to
autofocusing in digital photography or to more representative sets of
images
• simple focus measures based on taking the first derivative of an image
perform exceedingly well in digital photography, although care must be
taken in which direction derivative is taken
April 8th 2013
Combinatorial Problems in Compiler
Optimization
24
• The speed and accuracy of a digital camera's contrast-based
autofocus algorithm are crucial to user satisfaction
• We advocate using three standard statistical measures of
performance—precision, recall, and mean absolute error—as
accuracy criteria
• Our experimental results indicate that two simple focus
measures based on taking first derivative of an image perform
exceedingly well in digital photography
February 4, 2014
Focus Measures for Digital Photography
25