Transcript Document 7643388

EE 7700
High Dynamic Range Imaging
References


Slides and papers by Debevec, Ward, Pattaniak, Nayar, Durand, et al…
http://people.csail.mit.edu/fredo/PUBLI/Siggraph2002/
Bahadir K. Gunturk
2
High Dynamic Range (HDR) Imaging

The range of luminances is more than 10^14 candela/m2
star light
10-6
moon light
10-2
office light
day light
100 101 102
104
search light
108
Range of human eye at an instant is around 10^4:1 (4log units)
Human eye can adapt to see much wider range.
Candela (cd) is the unit of luminous intensity (power emitted by a light source in a
particular direction, with wavelengths weighted by the sensitivity of the human eye.)
A common candle emits roughly 1 cd.
A 100 W incandescent lightbulb emits about 120 cd.
Bahadir K. Gunturk
3
Spectral Sensitivity of Human Visual
System: Luminosity Function
One candela is defined as the luminous intensity of a
monochromatic 540 THz light source that has a radiant intensity
of 1/683 watts per steradian, or about 1.464 mW/sr. The 540 THz
frequency corresponds to a wavelength of about 555 nm, which is
green light near the peak of the eye's response. A typical candle
produces very roughly one candela of luminous intensity.
Quantity
Luminance
Luminous flux
Illuminance
Derived SI Unit
Symbol
candela per square meter cd/m2
lumen
cd * sr = lm
lux
lm/m2 = lx
Photopic (black) and scotopic [1] (green) luminosity functions. The photopic includes the CIE 1931
standard [2] (solid), the Judd-Vos 1978 modified data [3] (dashed), and the Sharpe, Stockman,
Jagla & Jägle 2005 data [4] (dotted). The horizontal axis is wavelength in nm. (from Wikipedia)
Bahadir K. Gunturk
4
HDR

The range of radiances is more than 10^14 candela/m2
star light
10-6
moon light
10-2
office light
day light
100 101 102
104
search light
108
Range of Typical Displays:
from ~1 to ~100 cd/m2
0
Bahadir K. Gunturk
255
5
Sensitivity of Eye
Cone dominated
Gain
log Gain
rod
cone
-6
-4
Bahadir K. Gunturk
-2
0
log La
2
4
6
1000 cd/m^2
6
Sensitivity of Eye
Rod dominated
Gain
log Gain
rod
cone
0.04 cd/m^2
-6
-4
Bahadir K. Gunturk
-2
0
log La
2
4
6
7
Sensitivity of Eye
Bahadir K. Gunturk
8
HDR

The range of image capture devices is also low
Bahadir K. Gunturk
9
HDR

The range of image capture devices is also low
Bahadir K. Gunturk
10
HDR

HDR image rendered to be displayed on a LDR display.
Bahadir K. Gunturk
11
HDR Problems:
• How to capture an HDR image with LDR cameras?
• How to display an HDR image on LDR displays?
Bahadir K. Gunturk
12
• Capture multiple images with varying exposure.
• Combine them to produce an HDR image.
Bahadir K. Gunturk
13
Creating HDR from Multiple Pictures
Measured intensity, z
t1
t2
Irradiance, E (=total power per unit area)
t1
Bahadir K. Gunturk
t2
14
Creating HDR from Multiple Pictures
Measured intensity, z
t1
z1
t2
t1
t2
z2
z1 = t1 * E
z2 = t2 * E
E
Irradiance, E
Estimates:
Take a weighted sum of E1 and E2:
E1=z1/t1
w1
E2=z2/t2
w2
E=( w1*E1 + w2*E2 ) / (w1+w2)
E
Bahadir K. Gunturk
15
Creating HDR from Multiple Pictures
Measured intensity, z
t1
z1
t2
t1
t2
z2
z1 = t1 * E
z2 = t2 * E
E
Irradiance, E
Estimates:
Take a weighted sum of E1 and E2:
E1=z1/t1
w
E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))
E2=z2/t2
z
Bahadir K. Gunturk
255
16
Creating HDR from Multiple Pictures
Measured intensity, z
t1
z1
t2
t1
t2
z2
z1 = t1 * E
z2 = t2 * E
E
Irradiance, E
Estimates:
Take a weighted sum of E1 and E2:
E1=z1/t1
w
E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))
E2=z2/t2
z
Bahadir K. Gunturk
255
Question: If t1 and t2 are not given, how can we estimate them?
17
Creating HDR from Multiple Pictures
In general, the camera response is not linear.
f
z1 = f ( t1 * E )
z2 = f ( t2 * E )
t1
t2
g
E1= g (z1) / t1
E2= g (z2) / t2
z
w
w is sometimes chosen as the
derivative of f. (Mann)
E=( w(z1)*E1 + w(z2)*E2 ) / (w(z1)+w(z2))
z
Questions: How to estimate g and t?  One approach is based on polynomial model (Nayar).
Bahadir K. Gunturk
18
Radiometric Self Calibration
Irradiance
Intensity
Polynomial model
Exposure ratios:
Pixel
Image number
Cost function
Solve using
If exposure ratios are not known, solve iteratively
Bahadir K. Gunturk
19
Tone Mapping
Given an HDR image, how are we going to display it in an LDR display?
Bahadir K. Gunturk
20
Tone Mapping
Given an HDR image, how are we going to display it in an LDR display?
Linear
Nonlinear
Bahadir K. Gunturk
21
Durand
Dorsey
Bahadir K.&Gunturk
22
Durand
Dorsey
Bahadir K.&Gunturk
23
Durand
Dorsey
Bahadir K.&Gunturk
24
Durand
Dorsey
Bahadir K.&Gunturk
25
Durand
Dorsey
Bahadir K.&Gunturk
26
Durand & Dorsey
Durand
Dorsey
Bahadir K.&Gunturk
 Bilateral filter
27
Durand
Dorsey
Bahadir K.&Gunturk
28
Fattal et al in 1D
log
derivative
attenuate
2500:1
7.5:1
exp
Bahadir K. Gunturk
integrate
29
Reinhard et al.
L_white is the smallest luminance that will be mapped to pure white (1).
Set L_white = L_max to have no “burn-out”.
Bahadir K. Gunturk
30
Durand
Dorsey
Bahadir K.&Gunturk
31
Durand
Dorsey
Bahadir K.&Gunturk
32
Informal comparison
Gradient
Gradientdomain
domain
[Fattal
et al.]
Bahadir
K. Gunturket al.]
[Fattal
Bilateral
Bilateral
[Durand
[Durandetetal.]
al.]
Photographic
Photographic
[Reinhard
33al.]
[Reinhardetetal.]
Spatially Varying Exposures



Instead of capturing multiple pictures, allow different
amounts of light pass for different pixel positions.
Estimate the missing pixels.
Combine to obtain an HDR image.
100%
75%
50%
25%
Nayar
Bahadir K. Gunturk
34
Image Reconstruction: Interpolation
Bahadir K. Gunturk
35
Image Reconstruction: Aggregation
Bahadir K. Gunturk
36
HDR image examples
Bahadir K. Gunturk
37
HDR image examples
Bahadir K. Gunturk
38
HDR image examples
Bahadir K. Gunturk
39
The Bilateral Filter (BF)





The SUSAN filter, which is essentially the bilateral filter, was
used for corner/edge detection and denoising in [Smith &
Brady 97].
The BF was presented in [Tomasi & Manduchi 98].
[Elad 02] and [Barash 02] show that the BF is related to the
weighted least squares estimation and anisotropic diffusion.
Fast implementations/approximations have been proposed,
e.g., in [Paris & Durand 06].
In addition to image denoising, the BF is used in tone
mapping of HDR images, contrast enhancement, 3D mesh
smoothing, blocking artifact reduction, etc.
Bahadir K. Gunturk
40
Bilateral Filtering
Intensity (range)
proximity
Spatial (domain)
proximity
2
2
ˆI ( x)  1  e I ( y ) I ( x ) / 2 r2 e y  x / 2 d2 I ( y )
K y
Bahadir K. Gunturk
41
Bilateral Filtering
Input
Gaussian
 d  10
Bilateral
 d  10
 r  1.3
Bahadir K. Gunturk
42
What are the optimal values of the
parameters of the Bilateral Filter?
d  2
d  4
 r  10
MSE=49.8
MSE=50.9
MSE=30.3
MSE=43.4
MSE=42.5
MSE=71.5
MSE=100.0
 n  10
 r  30
 r  50
Bahadir K. Gunturk
43