Transcript Wavelet Intro

Wavelet – An Introduction
•
•
•
•
•
Multi-resolution decomposition
Wavelet basis
Dilation equations
Wavelet-based image coding
– Embedded zero-tree
Wavelet-based video coding
– Motion compensated temporal filtering
2015/7/17Wavelet, MC 2009
1
Wavelet Transformation
• The wavelet transform describes a multi-resolution decomposition
process in terms of expansion of an image onto a set of wavelet basis
 functions.
– Wavelet transform vs. Short-term Fourier transform
• Wavelet basis function: well-localized in both space and frequency
• Dilation equations  :
 ( x )  k ck (2 x  k ),
  ( x)dx  1
• Two multi-resolution decomposition schemes: pyramid  and
subband.
• Wavelet compression process
– Take wavelet transform of a digitized image.
– Allocate bits among the wavelet coefficients.
2015/7/17Wavelet, MC 2009
2
Wavelet-Based Image Coding
• Embedded Zero-tree Wavelet coding (EZW)
– by J. M. Shapiro (1993)
• Set Partitioning in Hierarchical Trees (SPIHT)
– by A. Said and W. A. Pearlman (1996)
• http://www.cipr.rpi.edu/research/SPIHT/
• Layered Zero Coding (LZC)
– by D. Taubman and A. Zakhor
• Rate-Distortion optimized Embedded coding (RDE)
– by J. Li and S.-M. Lei
• Embedded Block Coding with Optimized Truncation (EBCOT)
– by D. Taubman
– An important part of JPEG2000 Part 1
2015/7/17Wavelet, MC 2009
3
1-D Pyramidal Decomposition
2015/7/17Wavelet, MC 2009
4
Multi-resolution Decomposition
• A multiresolution analysis of the space L2 (R) consists of a sequence of
nested subspaces … V0  V1  …  Vn  … that satisfies certain selfsimilarity relations in spatial and frequency, as well as completeness
and regularity relations.
– Self-similarity in spatial demands that each subspace Vk is invariant under shifts by
integer multiples of 2-k. That is, for each f  Vk, m   there is a g  Vk with
f (x) = g(x+m2-k).
– Self-similarity in frequency demands that all subspaces Vk  Vl , k < l, are spatialscaled versions of each other, with scaling respectively dilation factor 2l-k. That is, for
each f  Vk there is a g  Vl with f (x) = g(x2l-k).
LL1
HL1
LH1
HH1
LL2
HL2
HL1

2015/7/17Wavelet, MC 2009
5
Multi-resolution Analysis and the
Scaling Functions
• Approximated by scaling functions
f (t )  k ak (t  k ).
• Harr (1907) scaling functions
0  t 1
 1
 (t )  
otherwise.
 1
• Dilation version of the “mother” scaling function: In fact, we can obtain
scaling functions at different resolutions in a manner similar to the
procedure used for wavelets,
• In Harr
 j ,k (t )  2 j / 2 (2 j t  k ).
 2
0
1,0 (t )  
0  t  1/ 2
ot herwise.
2015/7/17Wavelet, MC 2009
6
Wavelet Approximations by Harr (1)
 (f 0) (t )   c0,kk (t )
k
c0,k 
2015/7/17Wavelet, MC 2009

k 1
k
f (t )k (t )dt
7
Wavelet Approximations (2)
2015/7/17Wavelet, MC 2009
8
Short-term Fourier Transform (STFT)
• If we have a very non-stationary signal, we would like to know not
only the frequency components but when in time the particular
frequency components occurred.
• Short-term Fourier Transform
– We break the time signal f(t) into pieces of length T and apply Fourier analysis to
each pieces. Thus, we obtain an analysis that is a function of both time and
frequency.
– Boundary effect, (Problem #1)
– To reduce the boundary effect, we window each piece before we take the FT. If the
window shape is given by g(t), the STFT is given by

F ( , T )   f (t ) g * (t   )e jwt dt.

– If the window function g(t) is Gaussian, the STFT is called the Gabor transform.
– The problem with the STFT is the fixed window size. The window size should
contain at least one cycle of the low-frequency component, however, it can not
accurately localize the high frequency spurt.
– Uncertainty principle: If we wish to have finer resolution in time, we end up with
a lower resolution in the frequency domain.
2015/7/17Wavelet, MC 2009
9
2015/7/17Wavelet, MC 2009
10
STFT basis functions
WT basis functions
 Small frequency changes in the FT will produce changes everywhere in the time domain.
 Wavelets are local in both frequency/scale (via dilations) and in spatial/time (via translations).
2015/7/17Wavelet, MC 2009
11
Wavelets
• A wavelet of the mother function (t),
1
t b
ψ a,b 
ψ(
).
a
a
Ondelette in French, by Morlet
– We scale a function f(t) by replacing t with t/a and translate a function to the right or left by
an amount b by replacing t by t-b or t+b.
– If we want the scaled function to have the norm as the original function, we need to
multiply it by 1 / a . Because
2
t
2
f ( )  a f (t ) .
a
• Wavelet coefficients of a function f(t),

wa ,b   ψ a,b f (t )dt.

•
We can recover the function f(t) by
1
f (t ) 
cψ

 

 
where cψ  

0
wa ,bψ a,b
dadb
,
2
a
| (ω) |2
d , (ω)  FT [ (t )].
ω
2015/7/17Wavelet, MC 2009
12
Properties of Wavelet
• C should be finite, so (0) = 0. Since (0) is the average value of (t);
therefore a requirement on the mother wavelet is that it has zero mean.
• We would also like the wavelets to have finite energy. Using Parseval’s
relationship, we can write this requirement as



| ( ) |2  .
For this to happen, | ()|2 have to decay as  goes to infinite. This
requirements mean the energy in () is concentrated in a narrow
frequency band.
• Discrete version: a  2m ,
b  n2 m.
ψ m ,n  2m / 2ψ ( 2m t  n ).
f (t )   wm ,nψ m,n (t ).
m
2015/7/17Wavelet, MC 2009
n
13
Dilation Equations
• The scaling function itself can be represented by its dilations at a high
resolution:
 (t )   hk1,k (t ).
k
• Multi-resolution analysis (MRA) equations:
 (t )   hk 2 (2t  k ).
k
by substituting 1,k (t )  2 (2t  k ).
• Harr scaling function:
1

 h0  h1 

2
hk  0 for k  1.
• Orthonormal transforms
  (t ) (t  m)dt  
2015/7/17Wavelet, MC 2009
m
.
14
 (t )   hk1,k (t ).
k
(x) = ½  (2x) +  (2x-1) + ½  (2x-2)
0
2015/7/17Wavelet, MC 2009
1
2
15
Daubechies Wavelet DAUB4
•
4 coefficients
c02  c12  c22  c32  1

c2 c0  c1c3  0

c3  c2  c1  c0  0


0c3  1c2  2c1  3c0  0
c0  (1 
3) / 4
2 , c2  ( 3 
3) / 4
2
c2  (3 
3) / 4
2 , c3  (1 
3) / 4
2
2015/7/17Wavelet, MC 2009
16
Daubechies Wavelet DAUBx (2)
•
•
•
•
Its derivative exists only almost everywhere (fails on points p/2n)
Left differentiable, but not right differential.
The 130th largest wavelet coefficient has an amplitude less than 10-5 of the
largest coefficient among 1024 coefficients.
Compact wavelets are better for lower accuracy approximation and for
functions with discontinuities (such as edges), while small wavelets are better
for achieving high numerical accuracy.
BAUB6, p=3 moment vanishing
2015/7/17Wavelet, MC 2009
17
Daubechies 9-tap/7-tap Filter
Cohen-Daubechies-Feauveau (CDF)
• Biorthogonal wavelet: is a wavelet where the associated wavelet
transform is invertible but not necessarily orthogonal.
– allows more degrees of freedoms than orthogonal wavelets.
• decomposed into a set of low-pass samples and a set of high-pass
samples.
• Biorthogonal Daubechies 9-tap/7-tap filter (irreversible transform)
it introduces quantization noise that depends
on the precision of the decoder
analysis filter
5+4
4+3
• Le Gall 5-tap/3-tap filter (reversible transform)
2015/7/17Wavelet, MC 2009
18
Subband Decomposition of an NM
Images
2015/7/17Wavelet, MC 2009
19
Embedded Zerotree Wavelet (1)
• Embedded coding
– The bits are ordered in their importance
• The encoding improves as more bits are transmitted.
– The codec can terminate at any point thereby allowing a target rate or distortion
metric to be met exactly.
• Assumption : most energy is compacted into lower bands
– The coefficients closer to the root have higher magnitudes than coefficients further
from the root.
LL1
HL1
LH1
HH1
LL2
HL2
HL1

2015/7/17Wavelet, MC 2009
20
EZW (2)
•
Embedded Image Coding using Zerotrees of wavelet coefficients: DWT,
prediction of the absence of significant information, entropy-coded
successive-approximate quantization and lossless data compression.
•
Zerotree structure
–
If a coefficient has a magnitude less than a given threshold, all its descendants might have
magnitudes less than that threshold.
–
Significant coefficient at level T
•
•
If |x| < T
Symbols
–
Zerotree root (ZR)
•
–
Isolated zero (IZ)
•
–
–
all descenfents (including itself) are insignificant
the coefficient is insignificant but has some
significant descendant
Significant positive (SP)
Significant negative (SN)
2015/7/17Wavelet, MC 2009
21
EZW (3)
• EZW is a multiple-pass algorithm
– Each pass consists of 2 steps
• Significance map encoding (dominant pass)
• Refinement (subordinate pass)
• Two separate lists are maintained
Scanning order
– Dominant list
• coordinates of the coefficients that have not been found to be significant in the relative
order
– Subordinate list
• magnitudes of the coefficients that have been found to be significant
• Successive Approximation
– During a subordinate pass
• the width of the effective quantizer step size (uncertainty interval) is cut in half
1 : the value falls in the upper half of the uncertainty interval
0 : the value falls in the lower half
– The threshold is halved before each dominant pass
2015/7/17Wavelet, MC 2009
22
Flow Chart for Encoding a Coefficient
2015/7/17Wavelet, MC 2009
23
EZW (4)
• Example 1
– Initial threshold
T0  2log2 cmax   2log2 26  24  16
• cmax will be in [T0, 2T0)
26
6 13 10
-7
7
6
4
4
-4
4
-3
2
-2
-2
0
Dominant list
Subband
LL2
HL2
LH2
HH2
2015/7/17Wavelet, MC 2009
Coeff.
26
6
-7
7
Symbol
SP
ZR
ZR
ZR
Subordinate list
“0” : [16, 24)
“1” : [24, 32)
Coef Symbol Recon
26
1
28
24
EZW (5)
• Example 1 (cont.)
Dominant list
– Pass 1
• T1 = T0/2 = 8
X
6 13 10
-7
7
6
4
4
-4
4
-3
2
-2
-2
0
Subband
HL2
LH2
HH2
HL1
HL1
HL1
HL1
Coeff.
6
-7
7
13
10
6
4
Symbol
IZ
ZR
ZR
SP
SP
IZ
IZ
Subband Coeff. Symbol
– Pass 2
• T2 = T1/2 = 4
X
6
X
X
-7
7
6
4
4
-4
4
-3
2
-2
-2
0
2015/7/17Wavelet, MC 2009
HL2
LH2
HH2
HL1
HL1
LH1
LH1
LH1
LH1
HH1
HH1
HH1
HH1
6
-7
7
6
4
4
-4
2
-2
4
-3
-2
0
SP
SN
SP
SP
SP
SP
SN
IZ
IZ
SP
IZ
IZ
IZ
Subordinate list
[8, 16)
[16, 24) [24, 32)
“0” : [8, 12)
[16,20) [24,28)
“1” : [12, 16) [20,24) [28,32)
Coef
26
13
10
Symbol Recon
10
2826
1
14
0
10
Coef
Symbol
Recon
26
10 0
2625
13
10
1413
10
6
-7
7
6
4
4
-4
4
00
0
1
1
0
0
0
0
0
109
5
-7
7
5
5
5
-5
5
25
EZW (6)
• Example 2
Dominant list
– Initial threshold
log2 cmax 
T0  2
63 -34
49
10
log2 63
2
7 13 -12
 2  32
5
7
-31 23
14 -13
3
4
6
-1
15 14
3 -12
5
-7
3
9
9
-7
-14
8
4
-2
3
2
-5
9
-1
47
4
6 -2
2
3
0
-3
2
3
-2
0
4
2
-3
6
-4
3
6
3
6
5 11
5
6
0
3
-4
4
2015/7/17Wavelet, MC 2009
Subband
LL3
HL3
LH3
HH3
HL2
HL2
HL2
HL2
LH2
LH2
LH2
LH2
HL1
HL1
HL1
HL1
LH1
LH1
LH1
LH1
Coeff. Symbol
63
SP
-34
SN
-31
IZ
23
ZR
49
SP
10
ZR
14
ZR
-13
ZR
15
ZR
14
IZ
-9
ZR
-7
ZR
7
ZR
13
ZR
3
ZR
4
ZR
-1
ZR
47
SP
-3
ZR
-2
ZR
Subordinate list
“0” : [32, 48)
“1” : [48, 64)
Coef Symbol Recon
63
34
49
47
1
0
1
0
56
40
56
40
26
EZW (7)
• Example 2 (2nd pass):
Dominant list
– Threshold T1 = 16
63 -34
49
10
7 13 -12
7
-31 23
14 -13
3
4
6
-1
15 14
3 -12
5
-7
3
9
9
-7
-14
8
4
-2
3
2
-5
9
-1
47
4
6 -2
2
3
0
-3
2
3
-2
0
4
2
-3
6
-4
3
6
3
6
5 11
5
6
0
3
-4
4
2015/7/17Wavelet, MC 2009
Subband Coeff. Symbol
LH3
-31
SN
HH3
23
SP
HL2
10
ZR
HL2
14
ZR
HL2
-13
ZR
LH2
15
ZR
LH2
14
ZR
LH2
-9
ZR
LH2
-7
ZR
HH2
3
ZR
HH2
-12
ZR
HH2
-14
ZR
HH2
8
ZR
Subordinate list
[16, 32) [32, 48) [48,64)
“0”: [16, 24) [32,40) [48,56)
“1”: [24, 32) [40,48) [56,64)
Coef Symbol Recon
63
11
5660
34
00
4036
49
10
5652
47
01
4044
-31
1
28
23
0
20
27
SPIHT (1)
• Set Partitioning in Hierarchical Trees
LIS
– Use a partitioning of the spatial orientation trees in a manner that tends to keep
insignificant coefficients together in a larger subsets
– Partitioning decision -> binary decision
• Provide a significance map encoding that is more efficient than EZW
– Each pass consists of 2 steps
• Significance map encoding (set partitioning and ordering step)
• Refinement
• Partitioning types
– O(i, j)
Each node can either have 4 or none
• Set of coordinates of the offsprings at (i, j)
– D(i, j)
• Set of all descendants of the coefficients at (i, j)
– H
• Set of all root nodes
– L(i,j)
• L(i,j) = D(i, j) - O(i, j)
2015/7/17Wavelet, MC 2009
28
SPIHT (2)
• Significant
– A set D(i, j) or L(i,j) is significant if any coefficient in the set has a magnitude
greater than the threshold
• Three lists are maintained
– LIP (list of insignificant pixels)
• Is Initialized with the set H
– LSP (list of significant pixels)
• Is initially empty
– LIS (list of insignificant sets)
• Contain the coordinates of the roots of types D or L
• At each pass
– Significant map encoding step
• Process the members of LIP
– If the coefficient is significant  transmit (1 + sign), move to LSP;
– Otherwise  transmit 0
• Then process the members of LIS
– Insignificant set  0
– Refinement step
• Process the elements of LSP
2015/7/17Wavelet, MC 2009
29
SPIHT (3)
 An example
– Initialize
n  log2 cmax   log2 26  4
• T = 16
– 1st pass (T = 16)
• LIP {(0,0)  26, (0,1)  6, (1,0)  -7, (1,1)  7}
• LIS {(0,1)D, (1,0)D, (1,1)D}
• LSP { }
• LIP
–
–
–
–
• LIS
–
–
–
1 + sign
26
6 13 10
-7
7
6
4
4
-4
4
-3
2
-2
-2
0
26  1 + 0  move to LSP
6 0
-7  0
7 0
10 0 0 0 0 0 0
(0,1)D  0
(1,0)D  0
(1,1)D  0
2015/7/17Wavelet, MC 2009
30
SPIHT(4)
 An example (cont.)
–
2nd pass (T = 8)
• n--  n = 3, T = 8
• LIP { (0,1)  6, (1,0)  -7, (1,1)  7}
• LIS {(0,1)D, (1,0)D, (1,1)D}
• LSP { (0,0)  26 }
•
26
6 13 10
-7
7
6
4
4
-4
4
-3
2
-2
-2
0
LIP
– 6 0
– -7  0
– 7 0
•
LIS
– (0,1)D  1
» 13  1 + 0  move to LSP
» 10  1 + 0  move to LSP
» 6  0  move to LIP
» 4  0  move to LIP
– (1,0)D  0
– (1,1)D  0
•
LSP
–
0 0 0 10 10 0 0 0 0 1
26 = (11010)2  1
2015/7/17Wavelet, MC 2009
31
SPIHT (5)
 An example (cont.)
–
3rd pass (T = 4)
• n--  n = 2, T = 4
• LIP {(0,1)  6, (1,0)  -7, (1,1)  7, (1,2)  6, (1,3)  4}
• LIS {(1,0)D, (1,1)D}
• LSP {(0,0)  26, (0,2)  13, (0,3)  10}
•
LIP
–
 1 + 0  move to LSP
-7  1 + 1  move to LSP
7  1 + 0  move to LSP
6  1 + 0  move to LSP
4  1 + 0  move to LSP
–
(1,0)D  1
–
–
–
–
•
6 13 10
-7
7
6
4
4
-4
4
-3
2
-2
-2
0
6
10 11 10 10 10 10 11 0 0 10 0 0 0 0 1 0
LIS
»
4  1 + 0  move to LSP
»
-4  1 + 1  move to LSP
0
» -2  0
(1,1)D  1
»
–
•
26
LSP
–
2
 move to LIP
 move to LIP
»
4  1 + 0  move to LSP
»
-3  0
»
»
LIP {(3,0)2, (3,1)-2, (2,3)-3, (3,2)-2, (3,3)0}
 move to LIP
LIS { }
-2  0
 move to LIP
00
 move to LIP
LSP { (0,0)  26, (0,2)  13, (0,3)  10, (0,1)  6,
26  0, 13  1, 10  0
2015/7/17Wavelet, MC 2009
(1,0)  -7, (1,1)  7, (1,2)  6, (1,3)  4,
(2,0)  4, (2,1)  -4, (2,2)  4}
32
Interframe Wavelet Coding
Motion-Compensated 3-D Subband
Coding of Video
IEEE TRANSACTIONS ON IMAGE PROCESSING
VOL. 8, NO. 2, FEB 1999
Seung-Jong Choi and John W. Woods
Adaptive Motion-Compensated Wavelet
Filtering for Image Sequence Coding
IEEE TRANSACTIONS ON IMAGE PROCESSING
VOL. 6, NO. 6, JUNE 1997
Jean-Pierre Leduc, Jean-Marc Odobez, and Claude Labit
Three-dimensional subband coding
with motion compensation
IEEE TRANSACTIONS ON IMAGE PROCESSING
VOL. 3, SEPT 1993
J.-R. Ohm
Outline
• Wavelet Transform-based Coding
• Motion Compensated Temporal Filtering
(MCTF)
• Block Diagram of Interframe Wavelet
Coding
• Experimental Results
• Conclusions
2015/7/17Wavelet, MC 2009
34
Wavelet Transform-based Coding
• Discrete wavelet transform (DWT)
• decomposes a non-stationary signal into multi-scaled
subbands
(1) Wavelet transform
(2) Quantization
(Embedded)
(3) Entropy coding
(Zero-tree)
2015/7/17Wavelet, MC 2009
35
Motion Compensated Temporal
Filtering (MCTF)
• The spatio-temporal scenes will be segmented
according to motion
build the trajectories by a spatio-temporal segmentation algorithm and a
related trajectory construction
• Temporal filtering will be applied along the assumed
motion trajectories
2015/7/17Wavelet, MC 2009
36
Motion Compensated Temporal
Filtering (MCTF)
(Haar filter [1, 1] [1, -1])
2015/7/17Wavelet, MC (Haar
2009
filter [1, 1] [1, -1]) with motion compensation
37
Motion Compensated 3D Wavelet
Coding by Lifting Filter Implementation
• The “highpass” pictures are very much the same
as difference frame in a frame prediction
• can be achieved in combination with motion
compensation
• A “lifting filter” structure is an efficient
implementation of wavelet decomposition
• Consisting of a “prediction step” (highpass filter)
and an “update step” (lowpass filter)
2015/7/17Wavelet, MC 2009
38
An MCTF Four-level Decomposition
(GOP=16 frames)
• Divides consecutive frames into
groups (GOP=16)
• Perform MC filtering on pairs of
frames to produce temporal low and
high frames.
• Temporal low frames decomposed
again with the same MC filtering
2015/7/17Wavelet, MC 2009
39
A Three-level 3D motion Compensated
Decomposition
• Pictures at leaves of tree are subject to spatial
wavelet decomposition
video sequence
1st temporal level
H
2nd temporal level
LH
3rd temporal level
LLL
LLH
2015/7/17Wavelet, MC 2009
40
Block Diagram of Interframe Wavelet
Motion Compensated Temporal Filtering + Zero-Tree Coding
John Woods, Jens Ohm
2015/7/17Wavelet, MC 2009
41
Experimental Results
Examples of comparison between MCTF and AVC (GOP=16 frames)
2015/7/17Wavelet, MC 2009
42
Conclusions
• Scalability: spatial, temporal, SNR and
complexity
(Wavelet coding which inherently possess scalability)
• Providing flexible scalability of a single
bitstream
(encoding once, decoding multiple times)
• Error resilience
(a non-recursive coder)
2015/7/17Wavelet, MC 2009
43