Transcript Slide 1

Error Control and Concealment
for Video Communication
CMPT820
Summer 2008
Michael Jia
Reference
“Error Control and Concealment
for Video Communication : A
Review”
YAO WANG, Member, IEEE
QIN-FAN ZHU, Member, IEEE
Outline
 Introduction
 Error Detection
 Error Concealment at Coder
 Error Concealment at Decoder
 Interactive Error Concealment
 Conclusion
Introduction
 2 Types of Transmission Errors

Random Bit Errors


Bit inversion, bit insertion, bit deletion
Erasure Errors

Packet loss, burst errors, system failures

More destructive
 Common usage of VLC makes them no differences in video
streaming
Introduction
 Lossless recovery

FEC (Forward Error Correction)

ECC (Error Control Coding)

ARQ (Automatic Retransmission Request)
 Not necessary for video transmission, human eyes can tolerate
a certain degree of distortion
 Focus on signal-reconstruction and error-concealment
techniques
Introduction
 Typical structure of a video communication system
Introduction
 3 groups of error-concealment techniques
 Forward error concealment


Error concealment by post-processing


Encoder plays the primary role
Decoder fulfills the task
Interactive error concealment
 Main concerns
 Effectiveness
 Required delay
 Bit-rate overhead
 Processing complexity
Outline
 Introduction
 Error Detection
 Error Concealment at Coder
 Error Concealment at Decoder
 Interactive Error Concealment
 Conclusion
Error Detection
 At transport layer

Adding header information (sequence number)


H.223
FEC (Forward Error Correction)

H.223, H.261

More reliable

Need more bandwidth
Error Detection
 At decoder
 Based on characteristics of natural video signals






Pixel value differences of neighboring lines (compare to a threshold)
Differences between boundary pixels in a block and its four neighbor
blocks
Obvious false value of quantization step size or DCT coefficients
Insert synchronization code word at the end of line of blocks
No additional bits or very few
Rely on smoothness of signal
Outline
 Introduction
 Error Detection
 Error Concealment at Coder
 Error Concealment at Decoder
 Interactive Error Concealment
 Conclusion
Error Concealment at Coder (1)
 Layered Coding with Transport Prioritization
 Most popular and effective (MPEG-2)
Error Concealment at Coder (1)
 Layered Coding
 Base layer – most important layer, with acceptable quality
 Transport prioritization
 Deliver base layer with higher degree of error protection
 High priority channel
 Re-transmission and/or FEC
 No explicit bit-rate overhead
 Complicate structure and coding overhead
 H.264 AVC/SVC
 Redundant pictures
 Data Partitioning
Error Concealment at Coder (2)
 Multiple-Description Coding (MDC)
 Several parallel channels
 Independent error events
 Small probability of “all channels down”
 Multiple “Descriptions”
 Several coded bit streams
 Transmitted over separate channels
 Any one will work
Error Concealment at Coder (2)
 Multiple-Description Coding (MDC)
Error Concealment at Coder (3)
 Joint Source and Channel Coding

Source-channel interaction at a lower level

Given channel error characteristics, design quantizer and entropy
coder for to minimize the effect of errors

For general sources, noisy channel  coarse quantizer is better

For image signals, noisy channel 

fewer bits to high-frequency coefficients

more bits to low-frequency coefficients
Error Concealment at Coder (4)
 Robust Waveform Coding

Intentionally keep some redundancy in source-coding


Adding auxiliary information in waveform coder


Layered coding and MDC both belong to this category
MPEG-2: sending motion vectors for microblocks in I-frames
Restricting prediction domain

H.263/H.264: prediction is confined within each slice
Error Concealment at Coder (5)
 Robust Entropy Coding

Add redundancy in entropy-coding

To help detect bit errors and prevent error propagation

Self-Synchronizing entropy coding


Add a synchronization code word

H.261, H.263, MPEG-4
Error-Resilient entropy coding (EREC)

MPEG-4 uses RVLC (reversible VLC)
Error Concealment at Coder (7)
 Transport-Level Control

Add redundancies at transport level

Prioritized transport for layered coding

Robust packetization

Spatial block interleaving

Dual transmission of important information

H.264 AVC/SVC

NAL unit syntax structure

Parameter Sets
Error Concealment at Coder (Summary)
Outline
 Introduction
 Error Detection
 Error Concealment at Coder
 Error Concealment at Decoder
 Interactive Error Concealment
 Conclusion
Error Concealment at Decoder
 Preview

Perform error concealment at the decoder

Can be used in conjunction with the auxiliary information provided
by the source coder

Low frequency components dominate images of natural scenes


Color values of adjacent pixels vary smoothly except sharp edges
Human eyes tolerate more distortion to high-frequency
components
Error Concealment at Decoder (1)
 Motion-Compensated Temporal Prediction

Replace damaged MB with the motion compensated block

Very effective when all the motion information in the base layer

Widely used (MPEG-2)

What if motion information or coding mode is lost?

Will discuss in next slide
Error Concealment at Decoder (2)
 Recovery of Motion Vectors and Coding Modes
 What if we lost motion vectors or coding modes?


Interpolated from spatially and temporally adjacent blocks
Estimation of coding modes


Simple: treat as ‘intracoded’
More sophisticate: MPEG-2 (See tables)
Error Concealment at Decoder (2)
 Estimation of motion vectors

Set to zeros (works well for low motion video)

Use MV of the corresponding block in the previous frame

Use the average of MVs from spatially adjacent blocks

Use the median of MVs from spatially adjacent blocks

Select one of the above methods depending on least boundary
matching error
Error Concealment at Decoder (3)
 Maximally Smooth Recovery

A constrained energy minimization approach

Minimize a measure of spatial and temporal variation between
adjacent pixels in this block and its spatially and temporally
neighboring blocks

Measure differences in 3 domains

Spatial – adjacent blocks

Temporal – prediction block in previous frame

Frequency – received coefficients for this block
Error Concealment at Decoder (3)
 Maximally Smooth Recovery
 2 sample spatial smoothness measures
Error Concealment at Decoder (4)
 Spatial- and Frequency-Domain Interpolation

A coefficient in a damaged block is likely to be close to the
corresponding coefficients in spatially adjacent blocks

Interpolate from four neighbor blocks


Maybe not accurate (8-pixel is too far)
Interpolate from four 1-pixel-wide boundaries

2 pixels in 2 nearest boundaries

4 pixels in all 4 boundaries

See picture in next slide
Error Concealment at Decoder (4)
 Spatial- and Frequency-Domain Interpolation
 Interpolate from four 1-pixel-wide boundaries
Error Concealment at Decoder (Summary)
Outline
 Introduction
 Error Detection
 Error Concealment at Coder
 Error Concealment at Decoder
 Interactive Error Concealment
 Conclusion
Interactive Error Concealment
 Preview

If a backward channel is available, can achieve better
performance by cooperation

Based on the feedback

At source coder – coding parameters can be adapted

At transport level – adjust the portion of bandwidth used for error
control

Decoding delay issue
Interactive Error Concealment (1)
 Selective Encoding for Error Concealment


Simple – code next frame in intramode

Error stopped in about one round-trip time

Will cause bit-rate increase
Send identity info back, perform error concealment at the same
time

Continue encode without using the affected area

Perform same error concealment procedure (need a prediction frame
buffer)

See picture in next slide
Interactive Error Concealment (1)
 Selective Encoding for Error Concealment
Interactive Error Concealment (1)
 Selective Encoding for Error Concealment
Interactive Error Concealment (1)

H.263 uses more prediction frame buffers (reference
picture selection mode)
Interactive Error Concealment (2)
 Adaptive Transport for Error Concealment


Retransmission is unacceptable for real-time video applications?

Not always

For one-way video broadcast, we may tolerate a few seconds delay

For multipoint video conferencing, use MCU (multipoint control unit)
If retransmission is controlled properly, end-to-end quality can be
improved

Both H.323 and H.324 defined such mechanisms
Interactive Error Concealment (3)
 Retransmission Without Waiting


Wait for the retransmission data

Not good, may freeze the display

Cause transit delay and accumulation delay
Without waiting

Request the retransmission

Conceal the error

Track the affected pixels

Correct them upon the arrival of the retransmission data
Interactive Error Concealment (4)
 Prioritized, Multi-copy Retransmission

Effective in very lossy channels

Video streaming via PSTN

Are you kidding me?

Send multiple copies of a lost packet

Use layered coding

# of retransmission trials and # of copies are proportional to the
importance of the layer
Interactive Error Concealment (Summary)
Outline
 Introduction
 Error Detection
 Error Concealment at Coder
 Error Concealment at Decoder
 Interactive Error Concealment
 Conclusion
Conclusion
 Real-time video communication doesn’t require lossless
delivery; signal-reconstruction and error-concealment
techniques are more effective.
 Add redundancy when encoding or delivering
 Estimate missing information when decoding
 Inform sender what is lost
 Burstiness has a significant impact on the choice of algorithms
Questions?
Thank You