Loss Recovery and Concealment for Packet Voice
Download
Report
Transcript Loss Recovery and Concealment for Packet Voice
New Methods for Voice Quality
Evaluation for IP Networks
Lingfen Sun
Emmanuel Ifeachor
Department of Communication and Electronic Engineering
University of Plymouth, U.K.
Outline
Introduction
Subjective MOS test from traditional to
Internet based
Intrusive voice quality measurement from
listening-only to conversational
Non-intrusive measurement from subjective
based to objective based
Conclusions
4 September, 2003
ITC-18, Berlin, Germany
2
Introduction
Aims: to investigate new subjective and
objective measurement methods for VoIP
applications
Subjective tests
MOS (Mean Opinion Score), user perceived quality
Benchmarking objective methods
Slow, time consuming and expensive
Objective measurements
Intrusive methods (e.g. PESQ, only listening quality)
Non-intrusive methods (e.g. E-model, only for limited
codec/applications)
4 September, 2003
ITC-18, Berlin, Germany
3
Traditional MOS test
Strict test requirement (e.g. sound proof
room)
Guarantee consistent testing environment
Slow, time-consuming and expensive
Test environment far to reality
Originated from codec quality assessment
(the quality difference is subtle).
4 September, 2003
ITC-18, Berlin, Germany
4
Internet-based MOS test
Based on Internet, carried out at office,
project room or classroom, …, close to
reality
Easier access to large number of
subjects (e.g. 39 tests at the same time)
Save time and money
Lack of a controlled testing environment
(e.g. background noise).
4 September, 2003
ITC-18, Berlin, Germany
5
Online MOS test
4 September, 2003
ITC-18, Berlin, Germany
6
Online MOS test
4 September, 2003
ITC-18, Berlin, Germany
7
Online MOS test results
4 September, 2003
ITC-18, Berlin, Germany
8
Voice Quality Evaluation
MOS
(PESQ)
PESQ
encoder
Trace data
Jitter
buffer
Reference
speech
Subjective test
E-model
4 September, 2003
Degraded
speech
decoder
ITC-18, Berlin, Germany
MOS
(Web or
Room)
MOS
(E-model)
9
Performance Comparison
MOS Comparison
Room_MOS
PESQ_MOS
Web_MOS
E-model_MOS
5
MOS
4
3
2
1
0
0
5
10
15
20
25
30
Packet Loss (%)
4 September, 2003
ITC-18, Berlin, Germany
10
Performance Comparison
Correlation coefficients for MOS comparison
Name
PESQ
vs.
RMOS
Corr.
Coeff.
0.933
PESQ Emodel Emodel WMOS Emodel
vs.
vs.
vs.
vs.
vs.
WMOS RMOS WMOS RMOS PESQ
0.984
0.935
0.964
0.952
0.975
Results show that Internet-based MOS test compares
well with traditional MOS test.
Two objective test methods (PESQ and E-model) can
both predict subjective MOS score well.
4 September, 2003
ITC-18, Berlin, Germany
11
Intrusive conversational
quality measurement
PESQ is the latest ITU standard for
intrusive voice quality measurement.
It can only predict one-way listening
voice quality.
How to extend it to predict
conversational quality?
One solution: to combine with E-model
4 September, 2003
ITC-18, Berlin, Germany
12
Intrusive conversational
measurement
MOS (PESQ)
Reference
speech
Ie
PESQ
Degraded
speech
E-model
End-to-end
delay
MOS R Ie
Delay model
MOSc
Id
Procedures:
Obtain MOS from PESQ based on a comparison of
reference and degraded speech
Convert it back to R-factor, then to Ie
Obtain Id from end-to-end delay
Calculate MOSc from Id and Ie from E-model
4 September, 2003
ITC-18, Berlin, Germany
13
R-factor with MOS
R-value vs. MOS
100
R-value
80
60
40
20
from G.107
3rd order polynomial fitting
0
0.5
1
1.5
2
2.5
3
MOS
3.5
4
4.5
5
( From R to MOS, from G.107)
MOS 1
for
R0
MOS 1 0.035 R R( R 60)(100 R)7 106
for
0 R 100
MOS 4.5
for
R 100
(1)
( From MOS to R, by 3rd order polynomial)
R 3.026 MOS 3 25.314 MOS 2 87.060 MOS 57.336
4 September, 2003
ITC-18, Berlin, Germany
(2)
14
Combined with E-model
Ie 93.2 R
(3)
I d 0.024Ta 0.11(Ta 177.3) H (Ta 177.3)
H ( x) 0
where
H ( x) 1
if x 0
(4)
if x 0
R 93.2 I d Ie
(5)
Convert R back to MOS to obtain conversational voice
quality
4 September, 2003
ITC-18, Berlin, Germany
15
Non-intrusive
objective measurement
E-model is the latest ITU standard for nonintrusive voice quality prediction.
Its parameters/equations (e.g. Ie) are still
based on subjective tests.
Difficult to apply to new codecs/applications
How to improve it ?
Based on objective methods (e.g. PESQ) to derive
parameters/equations for new codecs and
applications.
4 September, 2003
ITC-18, Berlin, Germany
16
Procedure 1:
obtain MOS (PESQ)
MOS vs. Packet loss rate for AMR (12.2Kb/s)
4.5
4
MOS(PESQ)
3.5
3
2.5
2
1.5
0
5
10
15
20
Packet Loss (random loss, %)
25
30
Obtain MOS (PESQ) vs. packet loss rate (random)
for the codec (e.g. AMR 12.2 Kb/s)
MOS (PESQ) value was obtained by averaging over
different speech samples (male and female) in ITU
data base and different loss locations/patterns.
4 September, 2003
ITC-18, Berlin, Germany
17
Procedure 2:
Obtain Ie equation
Ie vs. Packet loss rate for AMR (12.2Kb/s)
55
50
from PESQ
from fitting
45
40
Ie
35
30
25
20
15
10
0
5
10
15
20
Packet Loss (random loss, %)
25
30
Convert MOS vs. packet loss to Ie vs. packet
loss using Equations (2) and (3).
Curve fitting to obtain equation of Ie vs. loss.
Ie 13.2 15.84 ln(1 0.38 * loss)
4 September, 2003
ITC-18, Berlin, Germany
(6)
18
Procedure 3:
Obtain MOS (loss, delay)
MOS vs. packet loss and delay (for AMR,12.2Kb/s)
4.5
4
3.5
MOS
3
2.5
2
1.5
1
0.5
0
0
5
10
15
20
200
25
30
400
Delay (ms)
Packet loss (%)
From Ie and Id, R can be derived, then MOS can be obtained
directly from packet loss and end-to-end delay.
This can be used for non-intrusive quality monitoring and
perceptual buffer optimization and perceived QoS control.
4 September, 2003
ITC-18, Berlin, Germany
19
Conclusions/Future work
Investigated novel subjective and
objective voice quality evaluation
methods.
Future work
More extensive Internet-based MOS test
and comparison with P.800 MOS test
New applications for predicted perceived
voice quality.
4 September, 2003
ITC-18, Berlin, Germany
20
Contact
MOS Test Website:
http://www.tech.plymouth.ac.uk/spmc/people/lfsun/mos/
Contact
Lingfen Sun: [email protected]
Emmanuel Ifeachor: [email protected]
Thank you!
4 September, 2003
ITC-18, Berlin, Germany
21