Perceptual Evaluation of Speech Quality (PESQ)

802.16 IP Telephony Lab

Speaker: Wen-Jen Lin Date: Dec. 3 2009 1 TAC2000/2000.7

802.16 IP Telephony Lab

Outline

 Introduction  ITU-T P.862

 P.862.3

 P.862.2

 P.862.1

 P.862 - Overview of the basic philosophy used in PESQ  Conclusion  Experience  Demo PESQ program  Reference 2 TAC2000/2000.7

802.16 IP Telephony Lab

Introduction

 Classic Quality Measurement  Noise Ratio  Frequency Response Functions etc.

 Speech coding ( ETSI GSM EFR/AMR, ITU-T G.728/729/723.1 etc)  New types of distortions  Voice over IP (packet loss and variable delay)  Voice over ATM (cell loss)  Voice over mobile (GSM, UMTS, frame repeat, front end clipping, comfort noise generation) 3 TAC2000/2000.7

802.16 IP Telephony Lab

Introduction (cont.)

 PSQM (Perceptual Speech Qualify Measure)  P.861 - The first international standard for the perceptual quality measurement of telephone band (300-3400 Hz) speech signals.

 The scope of recommendation P.861 was limited to the assessment of telephone-band speech codecs only.

4 TAC2000/2000.7

802.16 IP Telephony Lab

ITU-T P.862

 P.862

 Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs  Amendment 2  P.862.1

 Mapping function MOS-LQO for transforming P.862 raw result scores to  P.862.2

 Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs  P.862.3

 Application guide for objective quality measurement based on Recommendations P.862, P.862.1 and P.862.2

5 TAC2000/2000.7

802.16 IP Telephony Lab

Application

 P.862.3 – Example of measurement set-up and terminology 6 TAC2000/2000.7

802.16 IP Telephony Lab

P.862.1

 MOS-LQO (P.800.1)  ITU-T Rec. P.862 provides raw scores in the range –0.5 to 4.5.

 The mapping function: 7 TAC2000/2000.7

802.16 IP Telephony Lab

Figure of Mapping Function

8 TAC2000/2000.7

802.16 IP Telephony Lab

P.862 - Overview of the basic philosophy used in PESQ

9 TAC2000/2000.7

802.16 IP Telephony Lab

P.862 - Overview of the basic philosophy used in PESQ (cont.)

10 TAC2000/2000.7

802.16 IP Telephony Lab

Addition of background noise

11 TAC2000/2000.7

802.16 IP Telephony Lab

Factors for which PESQ had demonstrated acceptable accuracy

 Speech input levels to a codec  Errors in the transmission channel between an encoder and a decoder  Bit rates if a codec has more than one bit-rate mode  Transcodings  Environmental noise in the sending side 12 TAC2000/2000.7

802.16 IP Telephony Lab

PESQ is not intended to be used to assess

 Effect of listening level  Conversational delay  Talker echo, where a subjects hears his own voice delayed  Talker sidetone, where a subjects may hear its own voice distorted  Non-intrusive measurements, where only output signals are available from the system music 13 TAC2000/2000.7

802.16 IP Telephony Lab

which PESQ has not currently been validated

14 TAC2000/2000.7

802.16 IP Telephony Lab

Conclusion

 PESQ has been evaluated on a very wide range of speech codecs and telephone network tests.

 It has been found to produce accurate predictions of quality in the presence of diverse end-to-end network behaviors.

 PESQ represents a significant step forward in the accuracy and range of applicability of objective speech quality assessment methods.

15 TAC2000/2000.7

802.16 IP Telephony Lab

Head and Torso Simulator(HATS)

16 TAC2000/2000.7

Spirent Abacus

802.16 IP Telephony Lab

17 TAC2000/2000.7

802.16 IP Telephony Lab

Abacus Control GUI

18 TAC2000/2000.7

DSLAII Handset Interface

Testbed

802.16 IP Telephony Lab

Controller Malden 19 TAC2000/2000.7

RTP Flow

802.16 IP Telephony Lab

20 TAC2000/2000.7

802.16 IP Telephony Lab

Demo PESQ program

21 TAC2000/2000.7

802.16 IP Telephony Lab

Reference

 HATS http://www.bksv.com/doc/bp2240.pdf

 ITU-T Rec. P.862, “Perceptual Evaluation of Speech Quality (PESQ), an Objective Method for End-to-end Speech Quality Assessment of Narrowband Telephone Networks and Speech Codecs”, International Telecommunication Union, Geneva, Switzerland (2001 Feb.)  Malden - http://www.malden.co.uk/dsla.htm

 Perceptual Evaluation of Speech Quality (PESQ), the new ITU standard for end-to-end speech quality assessment. Part II – Psychoacoustic model http://www.mp3 tech.org/programmer/docs/2001-P03b.pdf

 Spirent - http://www.spirent.com/ 22 TAC2000/2000.7