Comparisons of FEC and Codec Robustness on VoIP Quality and Bandwidth Efficiency
Download
Report
Transcript Comparisons of FEC and Codec Robustness on VoIP Quality and Bandwidth Efficiency
Comparisons of FEC and Codec
Robustness on VoIP Quality and
Bandwidth Efficiency
Wenyu Jiang
Henning Schulzrinne
Columbia University
ICN 2002, Atlanta, GA
Aug 29, 2002
Introduction to VoIP
The Internet is still best-effort
Subject to packet loss and delay jitter
Options for repairing packet loss
Forward error correction (FEC)
Packet loss concealment (PLC)
Low complexity; bit-exact recovery
Receiver-only; no extra BW overhead
More robust (error resilient) codec
better PLC quality, and higher bit-rate
Question: use FEC or a more robust codec?
Metric of VoIP Quality
Mean Opinion Score (MOS) [ITU P.830]
Obtained via human-based listening tests
Listening (MOS) vs. conversational (MOSc)
4
Grade
Quality
5
Excellent
4
Good
3
Fair
2
Poor
1
Bad
iLBC 14kb/s
G.729 8kb/s
G.723.1 6.3kb/s
3.5
MOS
3
2.5
2
1.5
0
0.03 0.06 0.09 0.12 0.15
average loss probability
FEC and IP Header Overhead
An (n,k) FEC code has (n-k)/k overhead
Typical IP/UDP/RTP header is 40 bytes
codec
iLBC
(4,2) FEC
media pkt size
(T=30ms)
rmedia
rIP
54 bytes 14.4 kb/s
25.1 kb/s
108 bytes 28.8 kb/s
39.5 kb/s
G.729
(4,2) FEC
30 bytes
8 kb/s
18.7 kb/s
60 bytes
16 kb/s
26.7 kb/s
G.723.1
(4,2) FEC
24 bytes
6.4 kb/s
17.1 kb/s
48 bytes 12.8 kb/s
23.5 kb/s
Predicting MOS in VoIP
The E-model: an alternative to humanbased MOS estimation
Do need a first-time calibration from an
existing human MOS-loss curve
In VoIP, the E-model simplifies to two
main factors: loss (Ie) and delay (Id)
A gross score R is computed and
translated to MOS.
Loss-to-Ie mapping is codec-dependent
and calibrated
Predicting MOS in VoIP, contd
35
From loss and
delay to their
impairment scores
and to MOS
E-model Id
45
Ie (loss impairment)
Example mappings
40
35
30
25
20
15
10
0
4
25
3.5
MOS
20
0.18
R to MOS mapping
3
2.5
15
2
10
1.5
5
1
0
G.729 T=20ms random loss
0.03 0.06 0.09 0.12 0.15
average loss probability
4.5
30
Id (delay impairment)
50
0
50 100 150 200 250 300 350 400
delay (ms)
0.5
20
40
60
R value
80
100
Predicting MOS under FEC
Compute final loss probability pf after
FEC [Frossard 2001]
Bursty loss reduces FEC performance
Increasing the packet interval T makes FEC
more efficient under bursty loss [Jiang
2002]
Plug pf into the calibrated loss-to-Ie
mapping
FEC delay is n*T for an (n,k) code
Compute R value and translate to MOS
Quality Evaluation of FEC vs.
Codec Robustness
Codecs under evaluation
iLBC: a recent loss-robust codec proposed at
IETF; frame-independent coding
G.729: a near toll quality ITU codec
G.723.1: an ITU codec with even lower bit-rate,
but also slightly lower quality.
Utilize MOS curves from IETF presentations
for FEC MOS estimation
Assume some loss burstiness (conditional
loss probability of 30%)
Default packet interval T = 30ms
G.729+(5,3) FEC vs. iLBC
Ignoring delay effect, a larger T improves FEC
efficiency and its quality
When considering delay, however, using a
60ms interval is overkill, due to higher FEC
delay (5*60 = 300ms)
4
3.8
3.8
3.6
3.6
3.4
3.4
MOS
MOS_c
4
3.2
3.2
3
2.8
G.729+(5,3)
2.6 G.729+(5,3),T=60ms
iLBC,no FEC
2.4
0
0.03
0.06
0.09
0.12
average loss probability
G.729+(5,3)
G.729+(5,3),T=60ms
iLBC, no FEC
3
2.8
2.6
0.15
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
G.729+(5,2) vs. iLBC+(3,2)
When iLBC also uses FEC, and still keeping
similar gross bit-rate
G.729 still prevails, except for low loss conditions
when considering delay
4
3.8
3.8
3.6
3.6
3.4
3.4
MOS
MOS_c
4
3.2
3.2
3
2.8
2.4
2.8
G.729+(5,2)
G.729+(5,2),T=60ms
iLBC+(3,2)
2.6
0
0.03
0.06
0.09
0.12
average loss probability
3
G.729+(5,2)
G.729+(5,2),T=60ms
iLBC+(3,2) FEC
2.6
0.15
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
G.729+(7,2) vs. iLBC+(4,2)
Too much FEC redundancy (e.g., for G.729)
very long FEC block and delay
not always a good idea
iLBC wins in this case, when considering delay
4
3.8
3.8
3.6
3.6
3.4
3.4
MOS
MOS_c
4
3.2
3.2
3
2.8
2.4
2.8
G.729+(7,2)
iLBC+(4,2)
2.6
0
0.03
0.06
0.09
0.12
average loss probability
3
G.729+(7,2)
iLBC+(4,2)
2.6
0.15
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
G.729+(3,1) vs. iLBC+(4,2)
Using less FEC redundancy may actually
help, if the FEC block is shorter
Now G.729 performs similar to iLBC
4
3.8
3.8
3.6
3.6
3.4
3.4
MOS
MOS_c
4
3.2
3.2
3
2.8
2.4
2.8
G.729+(3,1)
iLBC+(4,2)
2.6
0
0.03
0.06
0.09
0.12
average loss probability
3
G.729+(3,1)
iLBC+(4,2)
2.6
0.15
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
Comparison with G.723.1
MOS(G.723.1) < MOS(iLBC) at zero loss
iLBC dominates more low loss areas compared
with G.729, whether delay is considered or not
4
4
G.723.1+(2,1)
G.723.1+(2,1),T=60ms
iLBC, no FEC
3.8
3.6
3.8
3.6
3.4
MOS
MOS_c
3.4
3.2
3.2
3
2.8
2.8
2.6
2.4
3
0
0.03
0.06
0.09
0.12
average loss probability
0.15
G.723.1+(2,1)
2.6 G.723.1+(2,1),T=60ms
iLBC,no FEC
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
G.723.1+(3,1) vs. iLBC+(3,2)
iLBC is still better for low loss
G.723.1 wins for higher loss
4
3.8
3.8
3.6
3.6
3.4
3.4
MOS
MOS_c
4
3.2
3.2
3
2.8
G.723.1+(3,1)
2.6 G.723.1+(3,1),T=60ms
iLBC+(3,2)
2.4
0
0.03
0.06
0.09
0.12
average loss probability
3
2.8
0.15
G.723.1+(3,1)
2.6 G.723.1+(3,1),T=60ms
iLBC+(3,2)
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
G.723.1+(4,1) vs. iLBC+(4,2)
iLBC dominates in this case whether delay is
considered or not,
(4,2) code already suffices for iLBC
(4,1) code’s performance essentially “saturates”
4
3.8
3.8
3.6
3.6
3.4
3.4
MOS
MOS_c
4
3.2
3.2
3
2.8
G.723.1+(4,1)
2.6 G.723.1+(4,1),T=60ms
iLBC+(4,2)
2.4
0
0.03
0.06
0.09
0.12
average loss probability
3
2.8
0.15
G.723.1+(4,1)
2.6 G.723.1+(4,1),T=60ms
iLBC+(4,2)
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
The Best of Both Worlds
Observations, when considering delay:
Example: max bandwidth 14 kb/s
4
iLBC is usually preferred in low loss conditions
G.729 or G.723.1 + FEC better for high loss
Consider delay impairment (use MOSc)
iLBC,no FEC
G.729+(5,3)
G.723.1+(2,1),T=60ms
3.8
3.6
MOS_c
MOS_c
3.4
3.2
3
2.8
2.6
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
4
iLBC Max BW: 14 kb/s
3.8
G.723.1+(2,1),T=60ms
3.6
3.4
3.2 G.729+(5,3)
3
2.8
2.6
2.4
0 0.03 0.06 0.09 0.12 0.15
average loss probability
Max Bandwidth: 21-28 kb/s
4
iLBC, no FEC
G.729+(5,2)
3.8
3.6
MOS_c
MOS_c
3.4
3.2
3
2.8
2.6
2.4
0
4
0.03
0.06
0.09
0.12
average loss probability
0.15
iLBC, no FEC
G.729+(3,1)
G.729+(5,2)
3.8
3.6
MOS_c
MOS_c
3.4
3.2
3
2.8
2.6
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
4
iLBC Max BW: 21 kb/s
3.8
3.6
3.4
G.729+(5,2)
3.2
3
2.8
2.6
2.4
0 0.03 0.06 0.09 0.12 0.15
average loss probability
4
iLBC Max BW: 28 kb/s
3.8
3.6
3.4 G.729+(3,1)
G.729+(5,2)
3.2
3
2.8
2.6
2.4
0 0.03 0.06 0.09 0.12 0.15
average loss probability
Effect of Max Bandwidth on
Achievable Quality
14 to 21 kb/s: significant improvement in MOSc
From 21 to 28 kb/s: marginal change due to
increasing delay impairment by FEC
4
3.8
3.6
3.4
MOS_c
3.2
3
2.8
Max BW: 14 kb/s
Max BW: 21 kb/s
Max BW: 28 kb/s
2.6
2.4
0
0.03
0.06
0.09
0.12
average loss probability
0.15
Conclusions
Compared listening and conversational MOS
achieved by conventional vs. robust codecs,
with same BW constraint
iLBC is better under low loss conditions
Conventional codec + FEC is better under
high loss, but
Usefulness of FEC redundancy saturates
beyond a certain point considering delay
At roughly a max BW of 21 kb/s
Reveals max achievable quality with current FEC
mechanism
Future Work
Implement the MOS prediction and
optimization procedure in software
Investigate the effect of jitter on
conventional vs. robust codecs
FEC cannot reduce jitter unless there are
many out-of-order packets
PLC in a robust codec like iLBC incurs a
much lower delay, thus may be preferable
to FEC
References
W. Jiang and H. Schulzrinne, Comparison
and optimization of packet loss repair
methods on VoIP perceived quality under
bursty loss, NOSSDAV 2002
P. Frossard, FEC performance in multimedia
streaming, IEEE Comm Letter 3/2001
ITU-T, Subjective performance assessment
of telephone-band and wideband digital
codecs, Recommendation P.830 2/1996