Transcript Parameter Expanded Variational Bayesian Methods Yuan (Alan

From Sparse Solutions of Systems of
Equations to Sparse Modeling of Signals
and Images
Alfred M. Bruckstein (Technion), David L. Donoho
(Stanford), Michael Elad (Technion)
SIAM REVIEW 2009
Presented by: Mingyuan Zhou
Duke University, ECE
June 11, 2009
Outline
•
•
•
•
Introduction
The sparsest solution of Ax = b
Variations on P0
Sparsity-seeking methods in signal
processing
• Processing of sparsely generated signals
• Applications in image processing
Introduction
• Under-determined linear system equation
• L2 norm
• L0 norm
How can uniqueness of a solution be claimed?
How to verify a candidate solution?
How to efficiently solve the problem (the exhaustive search is a NP-hard
problem)?
What kind of approximations will work and how accurate can those be?
Current achievements
• Conditions under which
has a unique solution
• Conditions under which
has the unique solution as
• Conditions under which the solution can be found by some “pursuit”
algorithm
• Less restrictive notions of sparsity, impact of noise, the behavior of
approximate solutions, and the properties of problem instances defined by
ensembles of random matrices…
The signal processing perspective
• JPEG, DCT
• JPEG-2000, DWT
• The sparsity of representation under given basis is key to many
important signal and image processing problems:
Image compression, Image denoising, image deblurring, speech
compression, audio compression…
Measuring sparsity
The sparsest solution of Ax = b
Uniqueness
• Uniqueness via the Spark
• Uniqueness via the Mutual Coherence
Pursuit Algorithms: Practice
• Greedy Algorithms
• Convex Relaxation Techniques
Pursuit Algorithms: Performance
• Greedy Algorithms
• Convex Relaxation Techniques
Variations on P0
Uncertainty Principles and Sparsity
From Exact to Approximate Solutions
• Relaxed constraint:
• Stability:
• Pursuit algorithms:
OMP
Iteratively reweighted least squares (IRLS)
Iterative thresholding
Stepwise algorithms: LARS and Homotopy
• Performance of pursuit algorithms
Beyond Coherence Arguments
• Empirical evidence: The column of A is drawn at random from a
Gaussian distribution,
Without noise
,
With noise
• Phase transitions in typical behavior:
• Phase transitions in typical behavior:
• Restricted isometry property (RIP):
The sparsest solution of Ax = b:
A summary
•
•
•
•
Uniqueness
Solvability
Approximate solutions
Beyond coherence
Sparsity-Seeking Methods in Signal
Processing
• Non-Gaussian Prior
• Combined representation
Processing of Sparsely Generated Signals
• Applications
Analysis
Compression
Denoising
Inverse problems
Compressive sensing
Morphological Component Analysis
• The quest for a dictionary
Reconstructed dictionaries
Dictionaries learned from training data
Dictionaries learned from data under test
Learning Methods: MOD, K-SVD, BPFA
Applications in Image Processing
• Compression of Facial Images
• Denoising of Images
• Denoising of Images
Summary