Transcript Wavelets, An Introduction

Wavelets
An Introductory Exposition
Features, Analysis Structures, and
Selected Applications
Entry of Wavelets into the
Domain of SP

Most significant event in SP after FT.
 Reference to Wavelet Transform:
“Fourier Transform of 20th Century”

Wavelets fills the missing link in signal
processing: link between time and
frequency studies

“Little wave with big future”
Groups Active in Wavelet Studies
Wavelets
Mathematics: Function
Analysis and Approximation
Engineering, Applied Scientists
In Signal Processing
Applications
Industrial Groups
Key Idea
Alternative Signal Representations
and Transformation

Signal representation in a suitable
domain for information extraction,….
Examples:
 Fourier Transform for spectral analysis
 Hilbert Transform in envelop detection
 KLT(PCA) for optimal function approx.
 Laguerre basis function
 Numerous other transforms( DCT,Radon,…)
Two Domains for Signal
Representation
Projection space,
Basis functions
Signal Domain
Transformed Domain
Transformation
Different projection spaces
tions
different signal representa-
Fourier Transform
Representation in Time
f(t)
Time Domain Studies
Representation in Freq
F(ω)
Frequency Domain Studies
Shortcomings of Fourier Transform
signal f(t) with localized changes
0.04
f(t)
0.02
0
-0.02
0
50
100
150
200
0
200
400
600
800
time t250
sin(t)
1
0
-1
1000
cos(t)
1
0
-1
0
200
400
600
800
Sine and cosine as basis functions of Fourier Transform
1000
- Basis function vary within ± ∞, no localization
- Only one Basis function: Sine and Cosine or its complex form
- Fourier Coefficients: Projection of function f(t) onto sine and
cosine bases functions
- Information about the entire range of a function is contained
in the coefficients, no localization
Fourier Transform
Loss of Local Information
An Illustration
f(x)
x
F(ω)
freq ω


Information about singularities and sharp changes spread across many
frequencies and many basic functions
Cost of computations for transients is high
Starting Point of Wavelets
Real-world Signals




Dominance of transient and nonstationary signals
Information often reside in transients,
changes
The need to develop tools for T-F
analysis
Shortcomings of STFT: Inability to
model most of the nonstationary realworld signals
What Are Wavelets

Wavelets are wavelike oscilatory
signals of finite bandwidth both
in Time and in Frequency

Wavelets are basis functions of
spaces with certain properties
Examples of Wavelets
Db4
Db10
Db 4, Db 10 are from Daubechies family of wavelets.
Db wavelets have no analytical expression, they are
constructed numerically
Examples of Wavelet Functions
Coiflet
WP1,5
2
1
0
-1
0
20
40
60
80
100
120
140
160
180
WP2,5
2
0
-2
-4
0
20
40
60
80
100
120
140
WP3,5
2
0
-2
-4
0
20
40
60
80
100
120
140
160
180
Wavelet Functions Generated from
Bior 3.9
bior3.9, wavelet packet W 30
10
5
0
-5
-10
0
50
100
150
200
250
300
350
600
700
Bior3.9, wavelet packet W 10
4
2
0
-2
-4
0
100
200
300
400
500
Shortcomings of Fourier Transform
Representation in Time
f(t)
Time Domain Studies
Representation in Freq
F(ω)
Frequency Domain Studies
Loss of Time Information in FT
SALAAM with switching the 1st 5000 samples with the tail segment
1
Original
0.5
0
-0.5
-1
0
0.5
1
1.5
2
2.5
3
3.5
x 10
4
1
shifted
0.5
0
-0.5
-1
0
0.5
1
1.5
2
2.5
3
3.5
x 10
4
Fourier Transform
Loss of time information
abs(fft) of SALAAM with shifting the 1st 5000 samples to the tail
4000
3000
2000
1000
0
0
1000
2000
3000
4000
5000
0
1000
2000
3000
4000
5000
4000
3000
2000
1000
0
An Important Property of Wavelets:
Wavelets Filling the Gap
Wavelets
Representation in Time
f(t)
Time Domain Studies
Representation in Freq
F(ω)
Frequency Domain Studies
Transient Nature of Signals
Most

of the signals we deal are
Nonstationary:
Non-stationary
refers to time variancy and
spectral variation of the signals with time

Unpredictable
changes
in statistics of the signals including
changes in pdf function or statistical parameters
The degree of unpredictability varies for
different signals and applications
An Example of a Transient Signal
Speech Signal
SALAAM
1
0
-1
0.5
0
0.5
1
1.5
2
2.5
3
3.5
sAlaam
x 10
4
0
-0.5
saLaam
1
500
1000
1500
2000
2500
3000
0
-1
1
saLaam
0
0
500
1000
1500
0
-1
0
500
1000
1500
2000
Three segments of SALAAM
2500
3000
EEG Signals, Awake and Asleep
EEG (EP)
Examples of Transient Signals
Engine Vibrations
Healthy
2
0
-2
500
1000
1500
2000
2500
3000
500
1000
1500
2000
Time (Samplings)
2500
3000
Faulty
2
0
-2
An Example of Real-world Signals
Engine Vibrations
Wavelet Functions, WP
Bior3.1, wavelet packet W2
40
20
0
-20
-40
0
20
40
60
80
100
80
100
db4, wavelet packet W 15
4
2
0
-2
-4
0
20
40
60
Main Stages in Signal Analysis
Signal domain
Signals
basis functions
Transformed domain, Coeffs
Transformation
Information
Analysis
basis functions
Recon
Signal
Reconstruction
Modified
Coefficients
extraction
Examples of Processing in
Coefficient Domain
 Coding
for communication, transmission
 Compression and Data Storage
 Detection and Pattern Recognition
 Modification for Enhancement purposes
 Noise Reduction
 Watermarking
 Signal Separation
 Modeling
Two Main Tasks of Wavelet
Analysis
 Decomposition:

Information Reside in Signal Constituents /
Components
 Time-Frequency
Representation
Transformation of a signal into timefrequency representation
 Different basis and transformations result in
different constituents and T-f information

Key Concern in SP
Resolution/Localization in Signal
Processing
 Resolution: Information Extraction
at a
Narrow Band of Signal Span.
 Localization both in Time and in Frequency
freq
High freq
Low freq
Time/space
Resolution/Localization
in Time
3
3
3
Resolution/Localization in
Frequency
Power Spectrum, Vibration data at combustion zone, original and denoised
150
2
2
Original
100
50
0
0
2000
4000
6000
8000
10000
12000
14000
0
2000
4000
6000
8000
Freq. Hz
10000
12000
14000
denoised, soft
150
100
50
0
Need for Joint Time-Frequency
Analysis of High Resolution
Why
Joint T-F Analysis:
Transient signal information reside in
different bands of time and frequency
domain
 The need for the study of signal at the
limits of resolution determined by
Heisenberg Uncertainty Limits

Heisenberg Uncertainty Principle





Heisenberg uncertainty principle defined in
quantum physics and is applicable to signal
processing problems as well.
Position and momentum of a particle can not
be determined simultaneously
Sets a limit given as follows:
Δt. Δf ≥ 1/4π
Δx2=⌠{(x-μm )2 |f(x)|2 dx /⌠|f(x)|2 dx
μm
is center of mass of the function f(x):
μm= ⌠{x| f(x)|2 dx /⌠|f(x)|2 dx
Second Concern
Alternative Time–Frequency Tiling

Information about Different signal behavior reside at
different T-F Bands
 Need to have alternative T-F tiling and cell structures
Freq/scale
Time/scale
Third Concern
Adaptive Multi-Resolution Study


Different information reside at different resolutions.
Redundancy in Signal Representation
Freq/scale
High Frequency
Redundancy
in tiling
Low Frequency
Time/scale
Wavelets
Tools for T-F Analysis
Wavelets allow:
 High resolution and focused study of signals in time
and in frequency
 Low resolution, Coarse and more general picture
and trend analysis
Comment:
Wavelet analysis resembles human mental activity i.e
capability for a detail focusing as well as general and
a broadly-based data analysis
Fourier vs Wavelet Transform
Fourier Transform
Wavelet Transform
Stationary signal Analysis
Nonstationary Transient signal Analysis
Frequency Information only,
Time/space information is lost
Joint Time and Frequency Information
Single Basis Function
Many Basis Functions
Computational Cost High
Low computational costs
Analysis Structures:
- FS( periodic functions only)
Numerous Analysis structures:
 CWT,
DWT(2 Band and M Band),
WP
- FT and DFT
DDWT, SWT,
 Adaptive Signal Transform, Numerous Best
basis selection algorithms


Frame structure
Examples of Wavelets
Coiflet 4
coif4 : phi
coif4 : psi
1.2
1.5
1
1
0.8
0.5
0.6
0.4
0
0.2
-0.5
0
-0.2
0
5
10
15
20
25
Scaling function
-1
0
5
10
15
20
Wavelet function
25
Wavelet Functions Generated from
HAAR function
Haar, wavelet packet W 13
2
1
0
-1
-2
0
5
10
15
20
25
30
35
25
30
35
dmey, wavelet packet W 10
2
1
0
-1
-2
0
5
10
15
20
Gabor 2D Wavelet
Gabor Wavelet at different scale
and Orientation
Laguerre Gaussian Wavelet
Laguerre Gaussian Wavelet
Wavelet Transform
 Wavelet Transform
is defined as in other
transforms:


W = <f(t),ψ(t)> = ∫f(t) ψ*(t)dt
wjk = <f(t),ψj,k(t)> = ∫f(t) ψj,k*(t)dt
ψj,k(t) is a shifted and scaled wavelet function, j is
scaling and k translation parameters, J,K can be any
scalars. In DWT they assume integer values.
fj,k(t)w
*
jk
j,k
Physical Interpretation of
Wavelet Transform
 Correlation (extent
of matching) of f(t) with
the wavelet ψjk(t) at scale j and location
k.
 It carries signal information at scale j,
and location k. It gives localized
information of a signal.
 Representation of
a signal in a domain
described by wavelet basis functions
Wavelet Transform
Given a function f(t) єL^2
Ψab(t)= Ψ(at-b)
wab
f(t)
Function Domain
single dimensional
f(t)
Transformed Domain:
two dimensional,
paramters a,b
coeffs
Translation b
Time/space
Scale a /freq
Illustration of Translation
by two different Wavelets
f(t)
t
φ0k (t)
s hifte d wave le ts
t
φ0 k (t)
Haar
s hifte d Haar wave le t
t
Illustrative Examples of WT
Signal f(t)
t
φ0k(t)
shifted wavelets
t
t
φ0k(t)
φjk(t)
Haar
shifted Haar wavelet
t
t
scaled Haar wavelet
t
Wavelet Transform
Window Function Interpretation

Different information are extracted form the same
signal using different wavelets. The need to access
multitude of transforms and wavelet bases for
extraction of different information of a given
signal
 Wavelets as window functions may be considered
as lenses having different resolutions. Different
information are extracted by different lenses at
different scales and different locations of a given
signal
 STFT has only one window function at a given
scale
Wavelet Transform,
Translation of Window Function
Illustration of Wavelet Scaling/Dilation
Common Wavelets

Old Wavelets:







Recent Wavelets












Haar function
Gabor function and wavelet(1D,2D)
Morlet wavelets
Shannon function
1st and 2nd Derivatives of Gaussian function(mexican hat)
Truncated and lapped Sine or Cosine functions
Db Wavelets (orthogonal, biorthogonal),
Coiflets,
Symlets
Biorthgonal wavelets
Mallat Wavelet (1D,2D)
Bathlets
Curvelets
Ridgelets
Meyer wavelets
Banana Wavelets
Your Wavelets
Second Generation Wavelets( wavelets constructed by lifting scheme)
Commonly Used Numerical
Wavelets
 Daubechies
Wavelets
 Biorthogonal Wavelets
 Coiflets
 Symmlets
 Bathlets
 Dmeyer
How Wavelets are Generated
 A few
number of wavelets are constructed
using known analog functions. Some are
expressed in analog function form, e.g.
Morlet wavelets
 Many others are designed and constructed
numerically. Db wavelets are examples of
these wavelets
 New wavelets are introduced by individuals
each year
Categorization of Wavelets











Analytical or Numerical
Real or Complex
Symmetric, antisymmetric, asymmetric (e.g. db)
Compactly supported or not compact
Causal, Non-causal wavelets
Do they have efficient computation algorithms such as FWT
Wavelets with special features e.g. complex wavelet with real part (high
freq) and complex part carry low frequency content
Maximally flat waveletes (Flatness of spectrum, Rate of decay at ω=pi
and origin)
Have Orthogonal, biorthogonal analysis system
Wavelets of high Resolution in time-frequency domain: ∆t ∆ω. Gabor:
H=0.5, Haar: H=0.58 in normalized frequency (0,1).
Balanced Wavelets (Bathlets) are designed under a criteria based on
balancing ∆t and ∆ω.
Two Common Analysis Structures
1- Standard Two Band Discrete Wavelet Transform
High Freq Details
Signal
High Pass
2
High pass
Low Pass
2
Low pass
Low freq Approx
Decimated Wavelet Transform-Analysis Stage
Multiresolution
Pyramidal Signal Analysis Structure
 Multiresolution
as a subband coding of a
signal,
 Signal components at different subbands
are extracted
 Different analyzing wavelets results in
different components at different bands
An Illustration of Multiresolution
Signal Decomposition
Binary Tree of DWT
Common Analysis Structures
Wavelet Packets
High freq details
High pass
High Pass
2
2
V space of the signal
Low pass
2
High pass
Low Pass
2
2
Low pass
Wavelet Packets -Analysis Stages
2
Wavelet Packets Tree
Standard Two Channel DWT and
WP T-F Tiling
Time-Freq. Tiling
Standard DWT
Standard WP
Perfect Reconstruction Filterbank
Filterbank implementation of DWT
Wavelet
Subspace W
High Pass
High pass
Low Pass
Low pass
Perfect Reconstrution Filter bank
Alternative Signal Analysis
Architectures
Wavelets are rich in analysis structures and
algorithms
Main Analysis Structures




ORTHOGONAL
BIORTHOGONAL
REDUNDANT WAVELET TRANSFROM
FRAME-BASED REDUNDANT
TRANSFORM
Beyond Orthogonality: Compression
Requirements
Coeffs 1
Slow rate of decay
Orthogonal Expansion
Coeffs 2
Fast rate of decay
Non-orthogonal Expansion
Primary Features of Wavelets used
in Signal Processing
1. Time- frequency Localization (useful for transient
2.
3.
4.
5.
data analysis)
Sparsity: Coefficients are often grouped into high
and low amplitude
Function Approximation: near Optimal Function
Approximation
Noise Reduction, No wavelet can model white noise
leading to small amplitude coefficients
Decorrelation: Signal components( coefficients) in
wavelet domain have a lower degree of correlation
Categorization of
Wavelet Applications
Applications: Methodology/Algorithm Intensive












Change Detection ( of all kinds)
Compression
Feature Extraction and Pattern Recognition with Numerous Applications
Image Enhancement
Noise reduction with Numerous Applications
Diagnostics (such as NDT, NDE)
Signal Separation, ICA using Wavelets
Solution of Differential and Partial Differential Equations
Analysis of Fractals
System Identification in Control systems
Data and Image Fusion
Watermarking
Applications, Industry
Categorization
Application Areas: Industry Intensive













Power Systems, Transient data Analysis, Component Diagnosis, Signal
detection
NDT, Ultra sonic Flaw Detection, Diagnosis
Communications, Coding, Transmission
Medical Applications: Compression, Noise Reduction, signal separation,
signal classification, image enhancement
Machine Vision, Robotics, geometry and spatial edge detection
Industrial Machinery and Machine Diagnosis,
Remote Sensing, Satellite Image Analysis, Image segmentation, fusion
Biometrics: Fingerprint, Iris and Facial Identification
Nuclear Reactors, Boiler Pipe fault analysis,
Oil Industry, Oil Exploration
Transportation Industry (railroad engine diagnosis, Aero jet engine and
Helicopter drive and gear diagnosis
Seismic Data analysis
Genetics
Applications, Fields of Work
Applications in different fields

Physics, Geophysics,
 Mathematics, function approximation, signal
processing, Harmonic analysis, Differential
Equation, Frames, Function Space Analysis
 Electrical Engineering, numerous aplications
 Biomedical Engineering, EEG, ECG, MRI
 Finance and Financial Data Analysis
 Mechanical, Civil, Ocean Science and Engineering
 Remote Sensing, Forestry
 Nontechnical: Historial document retrieval
Details of Applications


To Be Presented in a Separate
Slide Presentation