Transcript What are Good Apertures for Defocus Deblurring? Changyin Zhou Shree K. Nayar Columbia University ICCP 2009, San Francisco.

What are Good Apertures for
Defocus Deblurring?
Changyin Zhou
Shree K. Nayar
Columbia University
ICCP 2009, San Francisco
Geometry of Defocus
Lens
Object
Aperture Pattern
Sensor
PSF
Focus Plane
Formulation of Defocus
In the Spatial Domain
Focused Image
PSF
Image Noise
Captured Image
Formulation of Defocus
In the Fourier Domain
Focused Image
PSF
Image Noise
Captured Image
Various Aperture Coding Techniques
Annular Pattern
[Welford 1960]
[Levin et al. 2007]
Wave-front Coding
[Dowski 1993]
MURA Pattern
[Gottesman 1989]
[Veeraraghavan
et al. 2007]
What are Good Apertures for
Defocus Deblurring?
Outline
- Background
- Evaluation Criterion for Aperture Patterns
- Pattern Optimization
- Experiments
How to Evaluate Aperture Patterns?
Natural image
Gaussian white noise
Captured Image
Focused Image
An Optimal
Deblurring Algorithm
Deblurred Image
Basic idea:
Evaluate K using expected quality of deblurred image
How to Evaluate Aperture Patterns?
Natural image
Gaussian white noise
Captured Image
Focused Image
An Optimal
Deblurring Algorithm
Deblurred Image
Linear system
The Optimal Linear Deblurring Algorithm
 For white noise and L2 distance, Weiner filter is optimal
 Optimize C by minimizing the expected recovery error
The Optimal Linear Deblurring Algorithm

log(A)
: Gaussian noise level
: Expected power spectrum of natural images
 A variant of Weiner deconvolution
1/f law
How to Evaluate Aperture Patterns?
Gaussian white noise
Natural image
Captured Image
Focused Image
An Optimal
Deblurring Algorithm
Deblurred Image
Linear system
Evaluation
Function:
Evaluation Criterion for Aperture Pattern
Noise Level
Frequency
R(K |
Prior Power Spectrum
Expected deblurring quality at frequency
) : Expected deblurring quality at noise level
.
Evaluate Patterns Using the Criterion
R Curve of Circular Pattern
R
2
R( K |  )  
2
2
|
K
|


/ A


Noise Level

Evaluate Patterns Using the Criterion
Curves of Relative R: R(K)/R(K0)
R( K ) / R( K0 )
Veeraraghavan Levin
Noise Level

Annular
Image(binary)
Image(gray)
Random
• Coded apertures help more when noise level is low.
• Circular aperture is better when noise level is high.
Outline
- Background
- Evaluation Criterion for Aperture Patterns
- Pattern Optimization
- Experiments
Pattern Optimization

Difficult to solve analytically
 Patterns
evaluated in the Fourier domain, but strictly
constrained in the spatial domain

.
Difficult to do brute force search
 For
binary patterns of resolution N x N, the number
of possible solutions is huge,
when N = 13, if evaluating one pattern takes 1 millisecond,
the brute force search requires 1045 yrs.
Pattern Optimization
Pattern Evolution in Genetic Algorithm:
13 x 13 binary patterns; 8 different noise levels
σ =0.0001 σ =0.001 σ =0.002 σ =0.005 σ =0.008 σ =0.01 σ =0.02 σ =0.03
2nd Run:
3nd Run:
Evaluate the Optimized Patterns
Evaluate the Optimized Patterns
Relative R curves of the Optimized Patterns
R( K ) / R( K0 )
Circular
Noise Level 
Image
Veeraraghavan
σ= 0.001
Outline
- Background
- Evaluation Criterion for Aperture Patterns
- Pattern Optimization
- Experiments
Implementation
Precision Laser
Photoplot (1 micron)
Implementation
Canon EF 50mm
f/1.8 Lens
Implementation
Image Pattern
Veeraraghavan
et al.’s Pattern
Levin et al.’s
Pattern
Circular Pattern
(wide open)
Our Optimized
Pattern
Comparison Experiments on a CZP Chart
Captured Images
Focused Image
Circular Pattern
Levin et al.’s Pattern
Veeraraghavan et al’s Pattern
Image Pattern
Our Optimized Pattern
Comparison Experiments on a CZP Chart
Deblurred Images
Focused Image
Circular Pattern
Levin et al.’s Pattern
Veeraraghavan et al’s Pattern
Image Pattern
Our Optimized Pattern
Shrek
Captured Image using the Optimized Pattern
Shrek
Captured Image using
the Optimized Pattern
Deblurring Result
On the Street
Captured Image Using the Optimized Pattern
On the Street
Captured Image
Using the Optimized Pattern
Deblurring Result
Traffic Scene
Captured Image Using the Optimized Pattern
Traffic Scene
Captured Image Using
the Optimized Pattern
Deblurring Result
Summary


Main Contributions

Aperture Evaluation Criterion for Defocus Deblurring

Aperture Pattern Optimization for Defocus Deblurring
Future Work

Optimize gray-level patterns

Apply Criterion to other PSF Engineering Problems

Extend Analysis to Account for Diffraction
Thank You!