Transcript Iterated Denoising for Image Recovery

Predicting Wavelet Coefficients
Over Edges Using Estimates
Based on Nonlinear
Approximants
Onur G. Guleryuz
[email protected]
Epson Palo Alto Laboratory
Palo Alto, CA
google: Onur Guleryuz
Overview
Topic: Wavelet compression of piecewise smooth signals with edges.
(piecewise sparse)
Benchmark scenario:
Piecewise smooth signal
Erase all high frequency wavelet coefficients
mse?
Predict erased data
Outline:
•Background and Problem Statement
•Formulation
More than what I am doing,
•Algorithm
it’s how I am doing it.
•Results
Notes
Q: What are edges?
(Vague and loose) A: Edges are localized singularities that separate
statistically uniform regions of a nonstationary process.
Caveats:
No amount of looking at one side
helps predict the other side.
This method is not:
• edge/singularity detection,
• convex (and therefore not POCS),
• solving inverse problems under additive noise (wavelet-vaguelette),
• an explicit edge/singularity model.
This method is:
• a systematic way of constructing adaptive linear estimators,
• an adaptive sparse reconstruction,
• based on sparse nonlinear approximants (non-convex by design),
• a model for non-edges (sparsity/predictable detection).
Wavelet Compression in 1-D and 2-D
1-D
Wavelets of compact support achieve
sparse decompositions
A. Cohen, I. Daubechies, O. G. Guleryuz, and M. T. Orchard, ``On the importance of combining wavelet-based nonlinear
approximation with coding strategies,'' IEEE Trans. Info. Theory}, vol. 48, no. 7, pp. 1895-1921, July 2002.
Too many wavelet coefficients over edges
2-D
(Need to reduce)
M. N. Do, P. L. Dragotti, R. Shukla, and M. Vetterli, ``On the compression of two-dimensional piecewise
smooth functions,'‘ Proc. IEEE Int. Conf. on Image Proc. ICIP ’01, Thessaloniki, Greece, 2001.
Current Approaches
“1”: Modeling higher order dependencies over edges in wavelet domain.
•F. Arandiga, A. Cohen, M. Doblas, and B. Matei, ``Edge Adapted Nonlinear Multiscale Transforms for Compact Image
Representation ,'‘ Proc. IEEE Int. Conf. Image Proc., Barcelona, Spain, 2003.
•H. F. Ates and M. T. Orchard, ``Nonlinear Modeling of Wavelet Coefficients around Edges,'‘
Proc. IEEE Int. Conf. Image Proc., Barcelona, Spain, 2003.
…
“2”: New Representations.
(Reduce by prediction)
•J. Starck, E. J. Candes, and D. L. Donoho, ``The Curvelet Transform for Image Denoising,'‘ IEEE Trans. on Image
Proc., vol. 11, pp. 670-684, 2002.
•M. Wakin, J. Romberg, C. Hyeokho, and R. Baraniuk, ``Rate-distortion optimized image compression using
wedgelets,'‘ Proc. IEEE Int. Conf. Image Proc. June 2002.
…
•P.L. Dragotti and M. Vetterli, ``Wavelet footprints: theory, algorithms, and applications,'‘ IEEE Trans. on Sig.
Proc., vol. 51, pp. 1306-1323, 2003.
(Don’t create too many)
Translation/rotation invariance is an issue.
Best linear representations are given by overcomplete transforms.
Q: What are Overcomplete
Transforms?
Example: Translation invariant, overcomplete transforms
•Spatial DCT tilings of an Image
1
G
image-wide,
orthonormal transform
G
2
…
…
Image arranged in a (Nx1) vector x, G
i
G
M
are (NxN)
Sparse Decompositions and
Overcomplete Transforms
No single orthonormal transform in the overcomplete set provides a
very sparse decomposition.
V 1  {4,2,...}
image
1
…
G
M
V
…
G
2
G
1
VM
sparse portions
1


G
1
I
nonsparse portions G   
1
G
 S
Issues with Overcomplete Trfs
im age G ... G  .
Compression angle:
1
M
.

.
Thresholding based Denoising:
sparse portions
G
1
V (T )
GI1
…
image
(x)
…
…
G
M
1
V M (T ) GIM
nonsparse portions
remove the insignificant coefficients and the
noise that they contain
[1  (GI1T GI1  ... GIMT GIM )] x
( DT  x)
DCC’02
http://eeweb.poly.edu/~onur
Onur G. Guleryuz, "Nonlinear Approximation Based Image Recovery Using Adaptive Sparse Reconstructions and
Iterated Denoising: Part I - Theory“, “Part II – Adaptive Algorithms,” IEEE Transactions on Image Processing, in
review.
 x0 
y 
 xˆ1 
 x0 
x 
 x1 
Fill missing information with initial values, T=T0.
Denoise image with hard-threshold T. ( DT  y)
Enforce available information.
T=T-dT
Nonlinear Approximation and
Nonconvex Image Models
Assume single transform
missing
sample
available
sample
Sample coordinates for a two sample signal
Recovery transform coordinates
Find the missing data to minimize
min y (G G  ... G G ) y
T
xˆ1
1T
I
1
I
MT
I
M
I
(V 1 (T )...V M (T ))
Underlying Estimation Method
yT (GI1T GI1  ...  GIMT GIM ) y
min
xˆ1
min
xˆ1
yT (1 DT0 ) y  y0  (V 1 (T0  dT)...V M (T0  dT))
(T1  T0  dT )
yT (1  DT1 ) y  y1

(V 1 (T2 )...V M (T2 ))
There is method to the denoise, denoise, …, denoise madness.
•No explicit statistical modeling.
•Systematic way of generating adaptive linear estimators.
•It doesn’t care about the nonsparse portions of transforms (must identify
sparse portions correctly)
•Sparse
predictable.
•Relationships to harmonic analysis.
Modeling “Non-Edges” (Sparse
Regions)
DCT1
edge
DCT2=shift(DCT1)
DCTM=…
smooth
smooth
min yT (GI1T GI1  ... GIMT GIM ) y
xˆ1
I don’t care how badly the transform I am using does over the edges.
I determine non-edges aggressively.
Algorithm
Fill missing information (high frequency wavelet coefficients)
with initial values (0), T=T0.
Denoise image with hard-threshold T. ( DT  y)
Enforce available information
(low frequency wavelet coefficients).
T=T-dT
I use DCTs and a simple but good denoising technique:
http://eeweb.poly.edu/~onur
Onur G. Guleryuz, ``Weighted Overcomplete Denoising,‘’ Proc. Asilomar Conference on Signals and Systems, Pacific
Grove, CA, Nov. 2003.
Test Images
Graphics
(512x512)
Bubbles
(512x512)
Pattern
(512x512)
Cross
(512x512)
I admit, you can do edge detection on this one
Teapot (960x1280)
Lena (512x512)
Implementation
1: l-level wavelet transform (l=1, l=2)
2: All high frequency coefficients set to zero (l=1 half resolution,
l=2 quarter resolution)
3: Predict missing information
4: Report PSNR=10log10(255*255/mse)
Results on Graphics
Graphics, l=1
30.48dB to 51dB
Graphics, l=2
27.15dB to 37.44dB
Results on Bubbles
Bubbles, l=1
33.10dB to 35.10dB
Bubbles, l=2
29.03dB to 30.14dB
Bubbles crop, l=1
Unproc.: 30.41dB
magnitude info.
location info
Predicted: 33.00dB
Bubbles crop, l=2
Unproc.: 26.92dB
Predicted: 28.20dB
Pattern crop, l=1
Holder exponent
extrapolation, step edge
assumption, edge
detection, etc., aren’t
going to work well here.
Unproc.: 25.94dB
still a jump
Predicted: 26.63dB
Cross crop, l=1
Holder exponent
extrapolation, step edge
assumption, edge
detection, etc., aren’t
going to work well here.
Unproc.: 18.52dB
Predicted: 18.78dB
PSNR over 3 and 5 pixel neighborhood of edges (l=1)
Graphics
Unprocessed
3 pixel neigh.
5 pixel neigh.
overall
18.23 dB
20.22 dB
30.48 dB
+21 dB
Bubbles
Pattern
Cross
+21 dB
Predicted
39.00 dB
41.00 dB
51.00 dB
U
24.61 dB
26.52 dB
33.10 dB
P
28.56 dB
30.29 dB
35.10 dB
U
20.46 dB
22.02 dB
27.04 dB
P
22.39 dB
23.83 dB
27.48 dB
U
16.88 dB
17.44 dB
18.72 dB
18.32 dB
18.52 dB
18.87 dB
P
+4 dB
+2 dB
+1.5 dB
+2 dB
+0.5 dB
+0 dB
Comments and Conclusion
• I will show a few more results.
• Around edges, magnitude and location distortions.
• Instead of trying to model many different types of edges, model non-edges
as sparse (same algorithm handles all varieties).
• Early work 1: Interpolation in pixel domain may give misleading PSNR
numbers for two reasons.
• Early work 2: Hemami’s group and Vetterli’s group have wavelet domain
results (based on Holder exponents), but not on same scale.
• You can implement this for your own transform/filter bank
(denoise, available info, reduce threshold, …).
Results on Teapot
Teapot, l=1
36.17dB to 41.81dB
Teapot, l=2
32.54dB to 35.93dB
Teapot crop, l=1
Unproc.: 28.38dB
Predicted: 34.78dB
Teapot crop, l=2
Unproc.: 25.10dB
Predicted: ??.??dB
Results on Lena
Lena, l=1
35.26dB to 35.65dB
Lena, l=2
29.58dB to 30.04dB
Lena crop, l=1
Unproc.: 34.42dB
Predicted: 35.03dB
Lena crop, l=2
Unproc.: 27.79dB
Predicted: 29.83dB