Transcript Linearized Bregman Iteration for Compressed Sensing

Redundancy, Sparsity and
Algorithm
Zuowei Shen
National University of Singapore
Outline


Frames
Algorithms



Compressed sensing
Frame-based image deblurring
Matrix completion
Outline


Frames
Algorithms



Compressed sensing
Frame-based image deblurring
Matrix completion
Wavelet Tight Frame
Unitary Extension Principle
Piecewise Linear Tight Frame
Piecewise Linear Tight Frame
Oblique Extension Principle

I. Daubechies, B, Han, A. Ron and Z. Shen, Framelets: MRAbased constructions of wavelet frames, Applied and
Computation Harmonic Analysis, 14, 1—46, 2003

C. K. Chui, W. He, J. Stöckler, Compactly supported tight and
sibling frames with maximum vanishing moments, Applied and
Computation Harmonic Analysis, 13, 224—262, 2002
Outline


Frames
Algorithms



Compressed sensing
Frame-based image deblurring
Matrix completion
Compressed Sensing
Consider Equation:
Challenges
Algorithm


Linearized Bregman
Uzawa
The iteration converges to:
J.-F. Cai, S. Osher and Z. Shen, Linearized Bregman Iterations for Compressed
Sensing, Math. Comp., to appear.
J.-F. Cai, S. Osher and Z. Shen, Convergence of the Linearized Bregman
Iteration for l1-norm Minimization, Math. Comp. To appear.
Forward-Backward Splitting (FBS)

The FBS method is popular for solving
1st order optimality condition

FBS iteration:

Introducing an intermediate variable, we obtain
Compare of Linearied Bregman and FBS
Linearized Bregman



FBS iteration
Replace
in the first equation of linearized
Bregman by , and adjust parameters properly.
We will get the FBS iteration.
Large thresholding required
to get a solution which
leads to a sparse solution.
For noisy data, stopping the
iteration early.


Not so large thresholding
required to get a solution,
which leads to a not so
sparse solution.
For noisy data, adjusting
the thresholding parameter.
Outline


Frames
Algorithms



Compressed sensing
Frame-based image deblurring
Matrix completion
In Terms of Frame
Frame-based image deblurring

Degradation model:

Solve it in a tight frame domain:
where F is a tight frame satisfies
J.-F. Cai, S. Osher and Z. Shen, Linearized Bregman Iterations for Frame-based
Image Deblurring, UCLA CAM Report (08-57).
Results
Blurred (Gaussian kernel),
Noisy (sigma=2) image.
Deblurred image.
11 steps of iteration
Frame-based image deblurring
Step2

Step 5
Step 8
Main features come in first, and details increase as iterations
going on. Hence,


A few iterations are enough.
Robust to noise
Comparison with FBS method
Daubechies, Teschke, Vese.
Blurred (Motion kernel),
Noisy (sigma=5) image.
Our method, 24
iterations, PSNR=20.8dB
FBS method, 1825
iterations, PSNR=19.4dB
Chan, Riemenschneider, Shen, Shen.
Blurred (High resolution),
Noisy (sigma=5) image.
Our method, 5 iterations,
PSNR=27.8dB
FBS method, 164
iterations, PSNR=27.4dB
Comparison with other methods
SNR Comparison
All the results other than ours are from
Lou, Zhang, Osher, Bertozzi, Image recovery via nonlocal operators, UCLA
CAM Report (08-35), 2007.
Blind deblurring result
Real image taking by a shaking camera
Blind deblurred image
Cai, Ji, Liu, Shen, Blind motion deblurring from a single image using sparse
approximation, preprint.
Outline


Frames
Algorithms



Compressed sensing
Frame-based image deblurring
Matrix completion
Matrix Completion

Let
be a low rank matrix,

be its sampled entries.
Matrix completion: recovering
Applications: machine learning, control, computer
vision.
Recovered exactly with high probability by solving




where
is the nuclear norm.
Candes, Recht, Exact Matrix Completion via Convex Optimization, UCLA
CAM Report (08-76), 2007.
Challenges



is huge and dense.
has of low rank.
Off-the-shelf algorithms, i.e. SDPT3, are not directly amenable to
large scale matrix.
Singular Value Thresholding (SVT) operator

Let



be the reduced (SVD) of
:
--- singular vectors
--- positive singular values
SVT operator: thresholds singular values, and keeps singular
vectors.
where
SVT algorithm

Iteration: Starting from

Important properties:

is sparse.

is of low rank.
 Linearized Bregman iteration or Uzawa’s algorithm.
J.-F. Cai, E. Candes and Z. Shen, A Singular Value Thresholding Algorithm for
Matrix Completion, UCLA CAM Report (08-77).
Convergence

The SVT algorithm converges to the unique solution of

When
, it tends to
J.-F. Cai, E. Candes and Z. Shen, A Singular Value Thresholding Algorithm for
Matrix Completion, UCLA CAM Report (08-77).
Numerical results


is generated by
, where
are
matrices
with i.i.d. entries with normal distribution.
The tolerance is
; In the table,
is the
degrees of freedom of , hence
is the oversampling ratio.
Rank profile
Rank of
versus the iteration count k
SVT algorithm with noise

For noisy data, stopping the iteration when the error is within the
noise level.
Thanks!
Web: http://www.math.nus.edu.sg/~matzuows