Transcript Introduction to Multimedia Systems

Multimedia Over IP Networks -- III
Hao Jiang
Computer Science Department
Boston College
Nov. 13, 2007
CS335 Principles of Multimedia Systems
Outline
 We will discuss different issues related to robustness
of multimedia data over IP network.
 Topics include:
Error Correction
Error Concealment
Congestion Control
CS335 Principles of Multimedia Systems
Error Correction for Bit Errors
 Schemes to make encoded media stream resistant to
bit errors:
–
–
–
–
–
Adding error correction codes.
Insertion of re-synchronization bits.
Bi-directional entropy code.
Self-synchronization entropy code.
Error resilient entropy coding (EREC).
CS335 Principles of Multimedia Systems
Error Resilient Entropy Coding
 Basic Idea: Convert variable length code to some
kind of fixed length code.
1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
CS335 Principles of Multimedia Systems
EREC Encoding
Phase1
Phase2
1 2 3 4 5 6 7 8 9 10
Phase3
1 2 3 4 5 6 7 8 9 10
Final
1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
CS335 Principles of Multimedia Systems
EREC Encoding Scheme
1. The original code to be trans-coded is {Ci}, i = 0,…,M-1
2. Select slot width {Si} =0,…,M-1. Sum( Si ) = Sum ( length(Ci) ). Select
search sequence {D(n)}, n =0,.., M-1. Set Phase = 0;
3. For i = 0 to M-1
If Si >= length(Ci)
Put the code in slot Si; Update left space;
else
Search slot mod( i+D(phase), M );
If there is space left, put the code in;
4. If all the codes fit into the slot
Terminate;
else phase++; Goto 3
CS335 Principles of Multimedia Systems
EREC Decoding Scheme
1. Get the header information about slotlength and slotwidth and M.
2. Set Phase = 0; Set {codelen(i) = 0, i = 0, …, M-1}, Set {cflag(i) =0, I=0,1,
…, M-1}
3. For i = 0 to M-1
If codeword in slot i is fully decodable. Store codelen(i) = code length;
else search slot mod( i+D(phase), M ); Get bits from the position of
codelen(mod( i+D(phase), M )); If fully decodable, update codelen(i)
and cflag(i). Else get all the bits left in the searching slot.
4. If no cflag(i) == 0 or some no change since last phase, stop
else phase++; Goto 3
CS335 Principles of Multimedia Systems
EREC for JPEG Coding
Header Information
1011110110011010
10111011010
1101001001001010
10110011011111000111111001001010
101000001000011110011010
10101001010
1010110010011101011010
0110101011010
001010
01011010
011101010
01011010
011101010
01011010
01011010
01011010
01011010
001010
001010
1000011001001010
0010110100011111101101010
101110111101011010
1011101110001010
10111001010
10111001010
110111011101001110100110110010101110110011101110001010
1101000111010111111011010100100111000101110000101111011011101011010
10001001010011110001010
011001010
011011010
01001010
10100101001111101010011111110110010111011100100100001110000110011000111101010011010
10111101010
10111110000001110011010
110100010011110001010
100111101111001111111101010111010
11001111010011101011010
10101001010
101010101001010
011000001010
001010
Original code
42 475
101111011001101001010010110110101011000010
101110110100111011010011010010111001011000
110100100100101011100011000000101001000101
101100110111110001111110010010101100010010
101000001000011110011010111101111110100000
101010010101010110101011001011111010010010
101011001001110101101001111100111000101000
011010101101010000101101111101110110010001
001010011101101101010101111011101101010111
010110100100101010100110010001010100111101
011101010001100111100011000101001101100010
010110101011111100101010101010011110000001
011101010010111011101100010011010111111101
010110101010101011000010101111101101110110
010110101010111111010100010101110110001111
010110100111101001011101110111011100000111
010110101101001010100111111110110110000001
001010010010001110011010010111011101010010
001010001101110110101001110101001101001000
100001100100101000110110010000011111111010
001011010001111110110101011000110111000101
101110111101011010010100101000111011100110
101110111000101010111011110101011111111111
101110010100101100011011111001110110001111
101110010100100101010000100000111000111010
110111011101001110100110110010101110110011
110100011101011111101101010010011100010111
100010010100111100010100000101111011011101
011001010011010101110001010100100101101110
011011010011010110010101110111011100011110
010010101011010011010001111111111010101111
101001010011111010100111111101100101110111
101111010100010010000111000011001100011110
101111100000011100110101010011010011111111
110100010011110001010110011110001010010110
100111101111001111111101010111010101000101
110011110100111010110100100101100011010110
101010010100010100000011111111101111001111
101010101001010011011101101010100110111110
011000001010001111111110111100101101011100
001010110000011101100100011111111100111010
001010011010101011111101110100110101011101
1011110110011010
10111011010
1101001001001010
10110011011111000111111001001010
101000001000011110011010
10101001010
1010110010011101011010
0110101011010
001010
01011010
011101010
01011010
011101010
01011010
01011010
01011010
01011010
001010
001010
1000011001001010
0010110100011111101101010
101110111101011010
1011101110001010
10111001010
10111001010
110111011101001110100110110010101110110011101110001010
1101000111010111111011010100100111000101110000101111011011101011010
10001001010011110001010
011001010
011011010
01001010
10100101001111101010011111110110010111011100100100001110000110011000111101010
011010
10111101010
10111110000001110011010
110100010011110001010
100111101111001111111101010111010
11001111010011101011010
10101001010
101010101001010
011000001010
001010
Recovered code
Decode
EREC Code
CS335 Principles of Multimedia Systems
JPEG Coding in Noisy Channels
Original image
JPEG coding of Lena. Q=50.
CS335 Principles of Multimedia Systems
JPEG Coding in Noisy Channels
JPEG coding of Lena. Q=50. BER=0.01
JPEG coding of Lena. Q=50. BER=0.001
CS335 Principles of Multimedia Systems
JPEG Coding in Noisy Channels
JPEG coding of Lena. Q=50. BER=0.0001
JPEG coding of Lena. Q=50. BER=0.00001
CS335 Principles of Multimedia Systems
EREC-JPEG in Noisy Channels
EREC-JPEG coding of Lena.
Q=50. BER=0.01
EREC-JPEG coding of Lena.
Q=50. BER=0.001
CS335 Principles of Multimedia Systems
EREC-JPEG in Noisy Channels
EREC-JPEG coding of Lena
Q=50. BER=0.001
EREC-JPEG coding of Lena
Q=50. BER=0.0001
CS335 Principles of Multimedia Systems
Another Example on Color Image Coding
Original image
JPEG coding picture, Q=50
CS335 Principles of Multimedia Systems
JPEG Coding in Noisy Channel
JPEG coding in with BER = 0.01
JPEG coding in with BER = 0.001
CS335 Principles of Multimedia Systems
EREC Coding Results
Error resilient image coding in
BER = 0.01.
Error resilient image coding in
BER = 0.001.
CS335 Principles of Multimedia Systems
Comparison Results
PSNR(dB)
35
30
JPEG
25
EREC
20
15
10
5
0
1.00E-02
1.00E-03
1.00E-04
Performance comparison JPEG and EREC.
Lena. Q = 50. BPP = 0.6478. Random error.
CS335 Principles of Multimedia Systems
1.00E-05 BER
Comparison Results
PSNR(dB)
35
30
25
20
15
10
5
0
JPEG
EREC
1.00E-02
1.00E-03
1.00E-04
1.00E-05
Figure 5. Performance comparison JPEG and EREC.
Gold Hill. Q = 50. BPP = 0.7567. Random Error.
CS335 Principles of Multimedia Systems
BER
Error Correction for Packet Loss
 An error correction method
P1
P2
P3
C=P1 xor P2 xor P3
Reconstructed
packet
P1
P1 xor P3 xor C
P3
CS335 Principles of Multimedia Systems
C
Another Error Correction Scheme
D(n) R(n-1) D(n+1) R(n)
D(n+2) R(n+1)
D(n+3) R(n+2)
R(n) is usually a “rougher” coding of D(n)
P(n)
Recovered packet
n+1
P(n+2)
CS335 Principles of Multimedia Systems
P(n+3)
Dealing with Burst Packet Loss
 Transform burst loss into random loss -- interlacing.
Original
packets
1
2
Scrambled
packets
3
5
Received
packets
2
3
4
1
4
Burst packet lost
3
Received packet loss pattern is random.
CS335 Principles of Multimedia Systems
5
2
5
Error Concealment
 Error correction usually needs additional error
correction bits.
 In fact, it is possible to recover or at least cover the
lost data using only the correctly received data –
error concealment.
 Error concealment can be used for images, videos
and audios.
CS335 Principles of Multimedia Systems
Images with Partial Data Missing
25% blocks lost
50% blocks lost
CS335 Principles of Multimedia Systems
Reconstructed Images
Reconstructed images for cases of 25% and 50% block loss respectively.
CS335 Principles of Multimedia Systems
Directional Adaptive Error Concealment
Color continuous Orientation
continuous
Continuous
in the desired
orientation
Missing image
block
CS335 Principles of Multimedia Systems
Error Concealment for Videos
 For MPEG video, we can simply recover the lost
blocks in P frames by replacing the lost blocks with
the corresponding blocks in the previous I or P frame.
v
Previous I or P
P frame
CS335 Principles of Multimedia Systems
Congestion Control
 The need for congestion control
– Internet is a network that does not regulate the traffic of each
source strictly.
– Aggressive sources may obtain more bandwidth than other
information sources.
– When throughput in a network exceeds some threshold,
congestion collapse will happen.
Packet
delivering
rate
Normal
operation
Congestion
collapse
Packet sending rate
CS335 Principles of Multimedia Systems
TCP Congestion Control
 TCP uses a slow linear increase and exponential
decrease scheme for congestion control.
Packet
Rate
Slow start
Fast decrease
CS335 Principles of Multimedia Systems
Time
TCP Congestion Control (cont)
Packet 1
ACK
packet2
ACK
Send-and-wait protocol. Maximum packet rate is 1/round_trip_time.
CS335 Principles of Multimedia Systems
Sliding Window Scheme
Packet 1
Packet 2
Packet 3
Packet 4
Packet 5
Packet 6
ACK
If there is no bandwidth limitation, we can send data as fast
as we can.
CS335 Principles of Multimedia Systems
TCP Congestion Control
 TCP protocol controls the bandwidth usage by
adjusting the sliding window size (the number of
packets sent out in a burst).
 TCP detects congestion by timeout or the number of
duplicate ACK.
Packet 1
Packet 2
Packet 3
x
Packet 1
Packet 2
Packet 3
Ack1
Ack1
Ack1
Ack1
Ack3
CS335 Principles of Multimedia Systems
Multimedia and Congestion Control
 Should multimedia traffic follow the same rules like
TCP?
 It depends. But some kind of scheme to avoid the
congestion collapse of the whole network is worth to
study.
 Such congestion control schemes generally follow
the same pattern like TCP congestion control
schemes:
• Slowly increase the bandwidth usage to fully use the network
resources.
• Reduce the usage of bandwidth if congestion is likely to
happen.
CS335 Principles of Multimedia Systems
TCP Friendly Protocols
 For applications such as video conferencing, the
video source can adapt its bit rate by changing the
quantization factor.
 Video source detects congestion condition by the
parameters such as round trip time, packet loss ratio
etc. using RTCP.
 The bit rate of the video source can increase linearly
in each round trip time if no congestion happens.
 If congestion occurs, the video source should
decrease the rate (as much as possible like a TCP
source).
CS335 Principles of Multimedia Systems