Transcript Compressive Sampling (of Analog Signals)

Compressive Sampling
(of Analog Signals)
Moshe Mishali
Yonina C. Eldar
Technion – Israel Institute of Technology
http://www.technion.ac.il/~moshiko
http://www.ee.technion.ac.il/people/YoninaEldar
[email protected]
[email protected]
Advanced topics in sampling (Course 049029)
Seminar talk – November 2008
Context - Sampling
Analog world
Continuous signal
Digital world
Sampling
A2D
Reconstruction
D2A
2
Compression
“Can we not just directly measure the part that will
not end up being thrown away ?”
Donoho
Original 2500 KB
100%
Compressed 148
392
950 KB
15%
6%
38%
3
Outline
• Mathematical background
• From discrete to analog
• Uncertainty principles for analog signals
• Discussion
4
References
• M. Mishali and Y. C. Eldar, "Reduce and Boost: Recovering
Arbitrary Sets of Jointly Sparse Vectors," IEEE Trans. on Signal
Processing, vol. 56, no. 10, pp. 4692-4702, Oct. 2008.
• M. Mishali and Y. C. Eldar, "Blind Multi-Band Signal
Reconstruction: Compressed Sensing for Analog Signals," CCIT
Report #639, Sep. 2007, EE Dept., Technion.
• Y. C. Eldar, "Compressed Sensing of Analog Signals", submitted to
IEEE Trans. on Signal Processing, June 2008.
• Y. C. Eldar and M. Mishali, "Robust Recovery of Signals From a
Union of Subspaces", arXiv.org 0807.4581, submitted to IEEE Trans.
Inform. Theory, July 2008. #
• Y. C. Eldar, "Uncertainty Relations for Analog Signals", submitted to
IEEE Trans. Inform. Theory, Sept. 2008.
5
Mathematical background
•
•
•
•
Basic ideas of compressed sensing
Single measurement model (SMV)
Multiple- and Infinite- measurement models (MMV, IMV)
The “Continuous to finite” block (CTF)
6
Compressed Sensing
“Can we not just directly measure the part that will
not end up being thrown away ?”
Donoho
“sensing … as a way of extracting information
about an object from a small number of randomly
Candès et. al.
selected observations”
Analog
Audio
Signal
Nyquist rate
Sampling
High-rate
Compressed
Sensing
Compression
(e.g. MP3)
Low-rate
7
Concept
Goal: Identify the bucket with fake coins.
Nyquist:
Weigh a coin
from each bucket
Compressed Sensing:
Bucket #
Compression
numbers
Weigh a linear combination
of coins from all buckets
1 number
Bucket #
1 number
8
Mathematical Tools
y
A
non-zero entries  at least
x
measurements
Recovery: brute-force, convex optimization,
greedy algorithms, and more…
9
CS theory – on 2 slides
Compressed sensing (2003/4 and on) – Main results
is uniquely determined by
Donoho and Elad, 2003
Maximal cardinality of linearly independent column subsets
Hard to compute !
10
CS theory – on 2 slides
Compressed sensing (2003/4 and on) – Main results
is uniquely determined by
Donoho and Elad, 2003
is random
with high probability
Donoho, 2006 and Candès et. al., 2006
Convex and tractable
Donoho, 2006 and Candès et. al., 2006
NP-hard
Greedy algorithms: OMP, FOCUSS, etc.
Tropp, Cotter et. al. Chen et. al. and many other
11
Sparsity models
measurements
unknowns
MMV
SMV
IMV =
How can
Joint sparsity
Infinite Measurement Vectors (countable or uncountable)
with joint sparsity prior
be found ?
Infinite many variables
Infinite many constraints
Exploit prior  Reduce problem dimensions
12
Reduction Framework
Find a frame
for
Solve MMV
Theorem
Mishali and Eldar (2008)
IMV
Deterministic
reduction
Infinite structure allows
CS for analog signals
MMV
13
From discrete to analog
• Naïve extension
• The basic ingredients of sampling theorem
•
•
•
•
Sparse multiband model
Rate requirements
Multicoset sampling and unique representation
Practical recovery with the CTF block
• Sparse union of shift-invariant model
• Design of sampling operator
• Reconstruction algorithm
14
Naïve Extension to Analog Domain
Standard CS
Discrete Framework
Analog Domain
Sparsity prior
what is a sparse analog signal ?
Generalized sampling
Infinite
sequence
Finite dimensional elements
Random
is stable w.h.p
Stability
Operator Continuous
signal
Randomness  Infinitely many
Need structure for efficient implementation
Reconstruction
Finite program, well-studied
Undefined program over a continuous signal
15
Naïve Extension to Analog Domain
Standard CS
Discrete Framework
Analog Domain
Questions:
Sparsity prior
what is a sparse analog signal ?
1. What is the definition of analog sparsity ?
Generalized sampling
2. How to select a sampling operator ?
Infinite
sequence
Finite dimensional elements
Operator Continuous
signal
3. Can we introduce stucture in sampling and still
preserve stability ?
Random
is stable w.h.p
Stability
Randomness  Infinitely many
Need structure for efficient implementation
4. How to solve infinite dimensional recovery
problems ?Reconstruction
Finite program, well-studied
Undefined program over a continuous signal
16
A step backward
Every bandlimited signal (
Hertz)
can be perfectly reconstructed from uniform sampling
if the sampling rate is greater than

“Success has many fathers …”
Whittaker
1915
Nyquist
1928
Kotelnikov Shannon
1933
1949
17
A step backward
Every bandlimited signal (
Hertz)
can be perfectly reconstructed from uniform sampling
if the sampling rate is greater than
Fundamental ingredients of a sampling theorm
• A signal model
• A minimal rate requirement
• Explicit sampling and reconstruction stages
18
Discrete Compressed Sensing
Analog Compressive Sampling
19
Analog Compressed Sensing
What is the definition of analog sparsity ?
A signal with a multiband structure in some basis
1. Each band has an uncountable
number of non-zero elements
2. Band locations lie on an infinite grid
3. Band locations are unknown in advance
no more than N bands, max width B, bandlimited to
(Mishali and Eldar 2007)
20
Multi-Band Sensing: Goals
bands
Analog
Sampling
Infinite
Reconstruction
Analog
Goal: Perfect reconstruction
Constraints:
1. Minimal sampling rate
What is the minimal rate ?
What is the sensing mechanism ?
2. Fully blind system
How to reconstruct from infinite sequences ?
21
Rate Requirement
Theorem (non-blind recovery)
Landau (1967)
Average sampling rate
Subspace scenarios:
Minimal-rate sampling and reconstruction (NB) with known band
locations (Lin and Vaidyanathan 98)
Half blind system (Herley and Wong 99, Venkataramani and Bresler 00)
22
Sampling
Multi-Coset: Periodic Non-uniform on the Nyquist grid
In each block of
samples, only
are kept, as described by
2
3
Analog signal
Point-wise samples
0
0
3
2
0
2
3
Bresler et. al. (96,98,00,01)
23
The Sampler
DTFT
of sampling
sequences
Length .
known
Problems:
in vector form
unknowns
Constant
Observation:
is sparse
matrix
known
1. Undetermined system – non unique solution
is jointly sparse and unique under appropriate
parameter selection (






















)
2. Continuous set of linear systems
24
Paradigm
Solve finite
problem
S = non-zero rows
Reconstruct






















0
1
2
3
4
5
6
25
Continuous to Finite
CTF block
MMV
Solve finite
problem
Reconstruct
span a finite space
Any basis preserves the sparsity
Continuous
Finite
26
Algorithm
CTF
Continuous-to-finite block: Compressed sensing for analog signals
Perfect reconstruction at minimal rate
Blind system: band locations are unkown
Can be applied to CS of general analog signals
Works with other sampling techniques
27
Blind reconstruction flow
Multi-coset with
SBR4
Yes
CTF
Universal
No
No
SBR2
Bi-section
CTF
Yes
Uniform at
Spectrum-blind
Sampling
Ideal low-pass filter
Spectrum-blind
Reconstruction
28
Final reconstruction (non-blind)
Bresler et. al. (96,00)
29
Framework: Analog Compressed Sensing
Sampling signals from a union of shift-invariant spaces (SI)
Subspace
generators
30
Framework: Analog Compressed Sensing
What happen if only K<<N sequences are not zero ?
Not a subspace !
There is no prior knowledge on the exact
indices in the sum
Only k sequences are non-zero
31
Framework: Analog Compressed Sensing
Step 1: Compress the sampling sequences
Step 2: “Push” all operators to analog domain
CTF
System A
High
samplingare
ratenon-zero
= m/T
Only
k sequences
Post-compression
32
Framework: Analog Compressed Sensing
System B
Low sampling rate = p/T
Pre-compression
CTF
Theorem
Eldar (2008)
33
Does it work ?
34
Simulations
Minimal rate
1
0.8
0.6
0.4
SBR4
SBR2
0.2
0
5
Sampling
rate 10
r
Brute-Force
15
Empirical success rate
Empirical success rate
Minimal rate
1
0.8
0.6
0.4
SBR4
SBR2
0.2
0
5
Sampling
rate 10
15
r
M-OMP
35
Simulations (2)
0% Recovery
100% Recovery
0% Recovery
Noise-free
1
20
100% Recovery
20
1
15
0.
10
SNR
0.6
0.4
20
15
0.
0.
10
0.2
SNR
SNR
0.8
15
5 rate10
Sampling
r
0.
15
50
10
0.2
SBR4
5
100
0.4
150
0.6
200
0.8
10
250
5 rate10
Sampling
1
r
15
SBR2
15
r
Empirical recovery
rate
0.2
0.4
0.6
0.8
1
36

Simulations (3)
Signal
4
x 10
1
0.9
5
Reconstruction
filter
Amplitude





4
3
2
0.7
0.6
0.5
0.4
0.3
0.2
1
0.1
0
5
10
15
20
Time (nano secs)
-50
25
0
n
50
Output
Time (nSecs)
4
x 10
5
 






Amplitude
 
 

 
0.8
4
3
2
1
0
5
10
15
20
Time (nano secs)
Time (nSecs)
25
37
Break
(10 min. please)
38
Uncertainty principles
•
•
•
•
Coherence and the discrete uncertainty principle
Analog coherence and principles
Achieving the lower coherence bound
Uncertainty principles and sparse representations
39
The discrete uncertainty principle
Uncertainty principle
40
Discrete coherence
Which bases achieve the lowest coherence ?
41
Discrete coherence
Which signal achieves the uncertainty bound ?
Spikes
Fourier
42
Discrete to analog
• Shift invariant spaces
• Sparse representations
Questions:
• What is the analog uncertainty principle ?
• Which bases has the lowest coherence ?
• Which signal achieves the lower uncertainty bound ?
43
Analog uncertainty principle
Theorem
Eldar (2008)
44
Bases with minimal coherence
In the DFT domain
Fourier
Spikes
What are the analog counterparts ?
• Constant magnitude
• Modulation
• “Single” component
• Shifts
45
Bases with minimal coherence
In the frequency domain
46
Tightness
47
Sparse representations
• In discrete setting
48
Sparse representations
• Analog counterparts
Undefined program !
But, can be transformed into an IMV model
49
Discussion
• IMV model as a fundamental tool for
treating sparse analog signals
• Should quantify the DSP complexity of the
CTF block
• Compare approach with the “analog”
model
• Building blocks of analog CS framework.
50
Thank you
51