Transcript Distributed Video Coding VLBV, Sardinia, September 16, 2005 Bernd Girod Information Systems Laboratory Stanford University.

Distributed Video Coding
VLBV, Sardinia, September 16, 2005
Bernd Girod
Information Systems Laboratory
Stanford University
Outline

Foundations of distributed coding
– Slepian-Wolf Theorem and practical Slepian-Wolf coding
– Wyner-Ziv results and practical Wyner-Ziv coding

Low-complexity video encoding
– Pixel-domain and transform-domain coding
– Hash-based receiver motion estimation
– Wyner-Ziv residual coding

Error-resilient video transmission
– Systematic lossy joint source-channel coding
– Improving the error-resiliency of MPEG (or anything else)
by Wyner-Ziv coding
B. Girod: Distributed Video Coding
2
Outline

Foundations of distributed coding
– Slepian-Wolf Theorem and practical Slepian-Wolf coding
– Wyner-Ziv results and practical Wyner-Ziv coding

Low-complexity video encoding
– Pixel-domain and transform-domain coding
– Hash-based receiver motion estimation

Error-resilient video transmission
– Systematic lossy joint source-channel coding
– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding
3
Compression of Dependent Sources
Source
X
p XY  x, y  
p X  x  pY  y 
Joint
Encoder
R  H  X ,Y 
Joint
Decoder
X
Y
Source
Y
B. Girod: Distributed Video Coding
4
Distributed Compression of Dependent Sources
Source
X
Encoder
X
RX  ??
p XY  x, y  
Joint
Decoder
p X  x  pY  y 
Source
Y
Encoder
Y
B. Girod: Distributed Video Coding
X
Y
RY  ??
5
Slepian Wolf Theorem
Achievable rate region for i.i.d sequences
RY
[bits]
Independent decoding
H Y 
HX
B. Girod: Distributed Video Coding
RX [bits]
6
Slepian Wolf Theorem
Achievable rate region for i.i.d sequences
RY
[bits]
Independent decoding:
No errors
H Y 
Joint decoding:
Vanishing error probability
for long sequences
H Y | X 
RX  RY  H  X , Y 
H X |Y H  X 
RX [bits]
[Slepian, Wolf, 1973]
B. Girod: Distributed Video Coding
7
Lossless Compression
with Receiver Side Information
Source X
Encoder
Decoder
X
Y
Y
R  H X |Y 
RY
[bits]
H Y 
H Y | X 
RX  RY  H  X , Y 
H X |Y H  X 
B. Girod: Distributed Video Coding
RX [bits]
8
Distributed Compression and Channel Coding
Source
X|Y
Decoder
Encoder
X Y D
X
P
Y
Y
Idea
X 
Y 
D
01001100010101
01001101010101
00000001000000
B. Girod: Distributed Video Coding

Interpret Y as a “noisy” version of X with
“channel errors” D

Encoder generates “parity bits” P to
protect against errors D

Decoder concatenates Y and P and
performs error-correcting decoding
9
Towards Practical Slepian-Wolf Coding








Convolution coding for data compression [Blizard, 1969, unpublished]
Convolutional source coding [Hellman, 1975]
Coset codes [Pradhan and Ramchandran, 1999]
Trellis codes [Wang and Orchard, 2001]
Turbo codes
[Garcia-Frias and Zhao, 2001]
[Bajcsy and Mitran, 2001]
[Aaron and Girod, 2002]
LDPC codes [Liveris, Xiong, and Georghiades, 2002]
...
...
B. Girod: Distributed Video Coding
10
Slepian-Wolf Coding Using Turbo Codes
Systematic
Convolutional
Code
X
Parity bits
Systematic
bits X . . . . . . . Y
SISO
Decoder
Decision
Interleaver
X
Systematic
Convolutional
Code
SISO
Decoder
Parity bits
“Correlation channel”
X
Y
[Aaron and Girod, 2002]
B. Girod: Distributed Video Coding
11
Lossy Compression with Side Information
Source
X Encoder
Decoder
X'
Y
Y
[Wyner, Ziv, 1976] For mse distortion and Gaussian statistics,
rate-distortion functions of the two systems are the same.
Source
Y
X Encoder
Y
B. Girod: Distributed Video Coding
Decoder
X'
Y
12
Practical Wyner-Ziv Encoder and Decoder
Wyner-Ziv Decoder
Wyner-Ziv Encoder
X
Quantizer
Q
SlepianWolf
Encoder
SlepianWolf
Decoder
Y
B. Girod: Distributed Video Coding
Q
Minimum
Distortion
Reconstruction
X'
Y
13
Non-Connected Quantization Regions

Example: Non-connected intervals for scalar quantization
q 1
q2
q3
x

Decoder: Minimum mean-squared error reconstruction with side information
conditional centroid
f X |Y  x | y 
2
xˆ  arg min E  X  xˆ  | y, q 


xˆ
x
B. Girod: Distributed Video Coding
14
Outline

Foundations of distributed coding
– Slepian-Wolf Theorem and practical Slepian-Wolf coding
– Wyner-Ziv results and practical Wyner-Ziv coding

Low-complexity video encoding
– Pixel-domain and transform-domain coding
– Hash-based receiver motion estimation

Error-resilient video transmission
– Systematic lossy joint source-channel coding
– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding
15
Interframe Video Coding
Predictive
Interframe
Encoder
X
Y
Predictive
Interframe
Decoder
Y
X’
Side
Information
B. Girod: Distributed Video Coding
16
Video Coding with Low Complexity
Wyner-Ziv
Intraframe
Encoder
X
[Witsenhausen, Wyner, 1978]
[Puri, Ramchandran, Allerton 2002]
[Aaron, Zhang, Girod, Asilomar 2002]
B. Girod: Distributed Video Coding
Wyner-Ziv
Interframe
Decoder
Y
X’
Side
Information
17
Low Complexity Encoding and Decoding
B. Girod: Distributed Video Coding
19
Pixel Domain Wyner-Ziv Coder
Interframe Decoder
Intraframe Encoder
Video
frame
Scalar
Quantizer
Turbo
Encoder
Buffer
Turbo
Decoder
Request
bits
X
Slepian-Wolf Codec
previous
B. Girod: Distributed Video Coding
X’
Y
Key
frames
[Aaron, Zhang, Girod, Asilomar 2002]
[Aaron, Rane, Zhang, Girod, DCC 2003]
Reconstruction
Interpolation
next
20
Pixel Domain Wyner-Ziv Coder
Decoder side information
generated by motioncompensated interpolation
PSNR 30.3 dB
B. Girod: Distributed Video Coding
After Wyner-Ziv Decoding
16-level quantization – 1.375 bpp
11 pixels in error
PSNR 36.7 dB
21
Pixel Domain Wyner-Ziv Coder
Decoder side information
generated by motioncompensated interpolation
PSNR 24.8 dB
B. Girod: Distributed Video Coding
After Wyner-Ziv Decoding
16-level quantization – 2.0 bpp
0 pixels in error
PSNR 36.5 dB
22
Stanford Camera Array
B. Girod: Distributed Video Coding
Courtesy Marc Levoy,
Stanford Computer Graphics Lab
23
Distributed Compression
Wyner-Ziv Cameras
Conventional Cameras
…
…
WZ-ENC
WZ-ENC
WZ-DEC
WZ-DEC
Distributed
Encoding
Geometry
Reconstruction
Joint
Decoding
Rendering
[Zhu, Aaron, Girod, 2003]
B. Girod: Distributed Video Coding
24
Light Field Compression
Wyner-Ziv, Pixel-Domain
JPEG-2000
Rate: 0.11 bpp
PSNR 39.9 dB
Rate: 0.11 bpp
PSNR 37.4 dB
B. Girod: Distributed Video Coding
25
DCT-Domain Wyner-Ziv Video Encoder
Low freq
coeffs
2 level
bit-plane 1
Extract bit- bit-plane 2
Quantizer
planes
…
Input
video
frame
DCT
Mk
Turbo
Encoder
Wyner-Ziv
parity bits
bit-plane Mk
For each low frequency coefficient band k
High freq
coeffs
Previous-frame
quantized high freq coefficients
Comparison
Quantizer
B. Girod: Distributed Video Coding
Entropy
Coder
High
frequency
bits
26
Wyner-Ziv Video Decoder
with Motion Compensation
Decoded
frame
For each low frequency band
Wyner-Ziv
parity bits
Turbo
Decoder
Reconstruction
Refined
side information
Side information
High
frequency
bits
IDCT
DCT
DCT
Motioncompensated
Extrapolation
Extrapolation
Refinement
Previous frame
Entropy Decoder
and
Inverse Quantizer
B. Girod: Distributed Video Coding
Reconstructed high frequency coefficients
27
Rate-Distortion Performance - Salesman
3 dB
6 dB
Every 8th frame is a key
frame
Salesman QCIF sequence
at 10fps 100 frames
B. Girod: Distributed Video Coding
28
Rate-Distortion Performance – Hall Monitor
3 dB
8 dB
Every 8th frame is a key
frame
Hall Monitor QCIF
sequence at 10fps 100
frames
B. Girod: Distributed Video Coding
29
Rate-Distortion Performance – Foreman
2 dB
1.5 dB
Every 8th frame is a key
frame
Foreman QCIF sequence
at 10fps 100 frames
B. Girod: Distributed Video Coding
30
Salesman at 10 fps
DCT-based Intracoding
149 kbps
PSNRY=30.0 dB
Wyner-Ziv DCT codec
152 kbps
PSNRY=35.6 dB
GOP=8
Hall Monitor at 10 fps
DCT-based Intracoding
156 kbps
PSNRY=30.2 dB
Wyner-Ziv DCT codec
155 kbps
PSNRY=37.1 dB
GOP=8
Foreman at 10 fps
DCT-based Intracoding
290 kbps
PSNRY=34.4 dB
Wyner-Ziv DCT codec
293 kbps
PSNRY=35.5 dB
GOP=8
Wyner-Ziv Residual Coding
X’n-1
X’n-1
Xn
Xn
Side
Information
Frame difference
Wyner-Ziv
Encoder
Wyner-Ziv
Decoder
Y
X’n
Side
Information
B. Girod: Distributed Video Coding
34
Rate-Distortion Performance – Foreman
Every 8th frame is a key
frame
Foreman QCIF sequence
at 30fps 100 frames
B. Girod: Distributed Video Coding
35
Rate-Distortion Performance – Foreman
Every 8th frame is a key
frame
Foreman QCIF sequence
at 30fps 100 frames
B. Girod: Distributed Video Coding
36
Outline

Foundations of distributed coding
– Slepian-Wolf Theorem and practical Slepian-Wolf coding
– Wyner-Ziv results and practical Wyner-Ziv coding

Low-complexity video encoding
– Pixel-domain and transform-domain coding
– Hash-based receiver motion estimation

Error-resilient video transmission
– Systematic lossy joint source-channel coding
– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding
37
Systematic Lossy Error Protection (SLEP)
of Compressed Video
Analog channel (uncoded)
Any Old
Video Encoder
Wyner-Ziv
Encoder A
Wyner-Ziv
Encoder B
Video Decoder with
Error Concealment
Error-Prone channel
S
S’
Wyner-Ziv
Decoder A
S*
Wyner-Ziv
Decoder B
S**
[Aaron, Rane, Girod, ICIP 2003]
Graceful degradation without a layered signal representation
B. Girod: Distributed Video Coding
38
MPEG with Systematic Lossy Error Protection
main
MPEG
Encoder
ED
Q
-1
-1
T
S*
+
MC
Channel
S
Reconstructed
Frame at
Encoder
ED
q-1
-1
T
S’
+
MC
MPEG
Encoder
R-S
Encoder
Parity
only
R-S
Decoder
coarse
Slepian-Wolf
Encoder
Wyner-Ziv Encoder
B. Girod: Distributed Video Coding
Side
information
MPEG
Encoder
coarse
[Rane, Aaron, Girod, VCIP 2004]
39
Reed-Solomon Coding Across Slices
Transmit parity slices only
RS code across slices
1 byte in slice
filler byte
parity byte
B. Girod: Distributed Video Coding
40
Results: CIF Foreman
FEC
SLEP
Main Stream @ 1.092 Mbps
FEC (n,k) = (40,36)
FEC bitrate = 120 Kbps
Total = 1.2 Mbps
SLEP
FEC
WZ Stream @ 270 Kbps
SLEP (n,k) = (52,36)
WZ bitrate = 120 Kbps
Total = 1.2 Mbps
B. Girod: Distributed Video Coding
41
MPEG with Systematic Lossy Error Protection
main
MPEG
Encoder
ED
Q
-1
-1
T
S*
+
MC
Channel
S
Reconstructed
Frame at
Encoder
ED
q-1
-1
T
S’
+
MC
MPEG
Encoder
R-S
Encoder
Parity
only
R-S
Decoder
coarse
Slepian-Wolf
Encoder
Wyner-Ziv Encoder
B. Girod: Distributed Video Coding
Side
information
MPEG
Encoder
coarse
[Rane, Aaron, Girod, VCIP 2004]
42
SLEP MPEG Codec with Simple Decoder
LEGACY BROADCASTING SYSTEM
Input
Video
MPEG2
Encoder
Conventionally encoded stream
Quantization parameter (Q)
Entropy
Decoding
Q-1
T-1
MPEG2
Decoder
+
Q1
Entropy
Coding
RS
Encoder
Coarse
Quantizer
Parity
only
Q1
Entropy
Coding
Fallback to coarser version
Side Info
Side Information
(motion vectors,
mode decisions)
(motion vecs, mode decisions)
Quantized
transformed
Prediction error
Error-prone Channel
MC
RS
Decoder
Entropy
Decoding
Q1-1
Decoded motion vecs
WYNER-ZIV ENCODER
B. Girod: Distributed Video Coding
WYNER-ZIV DECODER
43
Performance at Symbol Error Rate 10-4
MPEG-2 video: 2 Mbps
+ FEC 222 Kbps
(PSNR 29.78 dB)
B. Girod: Distributed Video Coding
MPEG-2 video: 2 Mbps
+ Wyner-Ziv 222 Kbps
(PSNR 35.45 dB)
44
Distributed Coding of Video:
Why Should We Care?

Chance to reinvent compression from scratch
–
–
–
–
–
–

Entropy coding
Quantization
Signal transforms
Adaptive coding
Rate control
...
Enables new compression applications
–
–
–
–
–
–
Very low complexity encoders
Compression for networks of cameras
Error-resilient transmission of signal waveforms
Digitally enhanced analog transmission
Unequal error protection without layered coding
...
B. Girod: Distributed Video Coding
45
The End
Further interest:
B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding,"
Proceedings of the IEEE, Special Issue on Video Coding and Delivery.
January 2005.
http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf