Workshop on “Monitoring Quality of Service and Quality of Experience of Multimedia Services in Broadband/Internet Networks” (Maputo, Mozambique, 14-16 April 2014) Overview of ITU-T.
Download ReportTranscript Workshop on “Monitoring Quality of Service and Quality of Experience of Multimedia Services in Broadband/Internet Networks” (Maputo, Mozambique, 14-16 April 2014) Overview of ITU-T.
Workshop on “Monitoring Quality of Service and Quality of Experience of Multimedia Services in Broadband/Internet Networks” (Maputo, Mozambique, 14-16 April 2014) Overview of ITU-T Study Group 12 Recommendations on QoS and QoE Joachim Pomy [email protected] OPTICOM, Germany SG 12 Recommendations E.420-E.479, E.800-E.859 G.100-series, except G.160-, G.180- and G.190-series G.1000-series I.350-series (including Y.1501/G.820/I.351), I.371, I.378, I.381 P-series, except P.900-series Y.1220-, Y.1530-, Y.1540-, Y.1560-series http://www.itu.int/ITU-T/recommendations/index_sg.aspx?sg=12 Maputo – Mozambique – 14 – 16 April 2014 2 Work Programme (1/8) Coordination of QoS/performance studies Communications Quality of Service: A framework and definitions Vocabulary for performance and quality of service Mean Opinion Score (MOS) terminology Transmission characteristics for wideband digital handset and headset telephones - Inclusion of extended wideband terminals Transmission characteristics for cordless and moblie digital terminals Transmission characteristics for wideband digital loudspeaking and hands-free telephony terminals - Inclusion of extended wideband terminals Technical requirements and test methods for the universal wired headset or headphone interface of digital mobile terminals Narrowband hands-free communication in motor vehicles Wideband hands-free communication in motor vehicles Maputo – Mozambique – 14 – 16 April 2014 3 P.800.1 Relation between some MOS qualifiers Maputo – Mozambique – 14 – 16 April 2014 4 Work Programme (2/8) User interface requirements for automotive applications Subsystem requirements for automotive speech services Artificial ears Calculation of loudness ratings for telephone sets Use of head and torso simulator (HATS) for hands-free and handset terminal testing Bone conduction testing setup Multiple test positions setup Setups and testing techniques for terminal performance measurements with background noise Artificial voices Speech processing devices for acoustic enhancement Transmission characteristics and speech quality parameters of hands-free terminals Maputo – Mozambique – 14 – 16 April 2014 5 Typical development cycle for a car speakerphone hands-free system and applicability of clauses of Rec. P.1100 Typically HFT development for a new car Research end Pre-development end Dev. B sample end A sample Dev. C sample end B sample Dev. D sample end C sample PPAP SOP D sample Application of the different clauses of this Recommendation Part 8 digital interface Part 8 digital interface Parts 11 and 12 HFT and SRWTEP parameters Part 13 subjective car-to-car test Part 10 microphone parameters What is covered by the various parts of the test Recommendation? Part 7: Covers test arrangements used within this Recommendation. Part 8: Covers non-mandatory objective measurements using a digital interface. Part 9: Covers test signals used within this Recommendation. Part 10: Covers non-mandatory objective measurements of the microphone or/and microphone system. Part 11: Covers mandatory objective measurements of the hands-free terminal (HFT). Part 12: Covers non-mandatory objective measurements of short-range wireless transmission enabled phone (SRWTEP). Part 13: Covers non-mandatory subjective evaluation of the hands-free terminal (HFT). P.1110(09)_F6-1 Maputo – Mozambique – 14 – 16 April 2014 6 Work Programme (3/8) Test signals for use in telephonometry Objective test methods for speech communication systems using complex test signals Methods for determining the ‘Objective Quality Number’ as overall quality value for terminals Applicability of psycho-acoustic models to HATS based setups for the benefit of terminal testing Setup signals for terminal with background noise Subjective quality evaluation of telephone services based on spoken dialogue systems Subjective performance of active signal processing devices Cultural/language/nationality dependence of subjective quality Subjective evaluation of generic sound activity detectors Multidimensional scaling quality assessment Subjective assessment methods for music quality of narrowband and wideband telephony Maputo – Mozambique – 14 – 16 April 2014 7 Rating scales used in ITU-T P.806 subjective testing methodology P.MULTI Perceptual Quality Scales PQ Scales S-FLT S-RUF S-LFC S-HFC B-LVL B-VAR Description Scale Descriptors Slow-varying Degradation in the fluttering, babbling, Speech Signal discontinuous Fast-varying Degradation in the rough, raspy, harsh Speech Signal Degradation of Low-Frequency dull, muffled, smothered Coloration in the Speech Signal Degradation of High-frequency small, distant, thin Coloration in the Speech Signal Degradation due to the Level of hissing, rushing, roaring Background Noise Degradation due to the Variability bubbling, intermittent, of the Background Noise variable Overall Scales Description LOUD Overall Loudness in the Speech Signal + Background Noise OVRL Overall Quality in the Speech Signal + Background Noise Maputo – Mozambique – 14 – 16 April 2014 8 Work Programme (4/8) Test methodology for web browsing and browser based applications Subjective usability evaluation of speech-based or multimodal interactive services Q.7/12 defines test plans for tests of speech coders, and reports the test results and analysis E-model extension for non-handset user interfaces E-model extension for speech processing devices E-model update for conversational quality features Mixed-band and beyond-wideband E-model E-model for monitoring Operational quality estimator Perceptual approaches for multi-dimensional analysis Perceptual objective noise reduction Maputo – Mozambique – 14 – 16 April 2014 9 Test setup for Rec. P.1501 Quality of web browsing Maputo – Mozambique – 14 – 16 April 2014 10 Work Programme (5/8) Perceptual Objective Prediction of Speech Intelligibility No-reference models for quality prediction Subjective method for simulated conversation tests addressing audio and audiovisual call quality Effect of delays on the telemeeting quality Use of auditory and visual cues for high-quality telemeetings Overall telemeeting quality value (long term goal) Quality implications and requirements for telemeeting and conferencing services Spatial audio meetings quality evaluation Frequent updates of Appendices Quality of Experience aspects of Multi Connection End-to-end QoE aspects of tandemmed speech processing devices Maputo – Mozambique – 14 – 16 April 2014 11 Rec. P.1301 Decision tree for Subjective quality evaluation of audio and audiovisual multiparty telemeetings Start Communication mode including text / graphics (e. g., web conferencing) ? Yes ITU-T P.1301 Annex B No Audio Communication mode? Video ITU-T P.1301 Annex C Audiovisual Test mode = audio Test mode = video Audio Video Test mode? Audio-visual Type of quality? Conversational Continue with Flowchart B Type of quality? Noninteractive Continue with Flowchart C Conversational Continue with Flowchart D Type of quality? Noninteractive Continue with Flowchart E Conversational Special test method required Noninteractive Continue with Flowchart F P.1301(12)_FA Maputo – Mozambique – 14 – 16 April 2014 12 Work Programme (6/8) End-to-end performance for managed voice over LTE networks This Recommendation describes the key aspects impacting endto-end performance of managed voice applications over LTE networks. Definitions and associated measurement methods for user-centric parameters for call handling in cellular mobile voice service Mobile QoS Supplement XX to ITU-T E.800-series Recommendations (Guidelines on Regulatory Aspects of QoS) End-user multimedia QoS categories Network model for evaluating multimedia transmission performance over Internet Protocol Opinion model for video and audio streaming applications QoE of Web-browsing QoE requirements for multicast video streaming services QoE requirements for telepresence services Guidance for the use of P.1201 and P.1202 in operational contexts Maputo – Mozambique – 14 – 16 April 2014 13 Rec. G.1010 – Model for user-centric QoS categories Error tolerant Conversational voice and video Voice/video messaging Streaming audio and video Fax Error intolerant Command/control (e.g. Telnet, interactive games) Transactions (e.g. E-commerce, WWW browsing, Email access) Messaging, Downloads (e.g. FTP, still image) Background (e.g. Usenet) Interactive (delay <<1 s) Responsive (delay ~2 s) Timely (delay ~10 s) Non-critical (delay >>10 s) T1213060-02 Maputo – Mozambique – 14 – 16 April 2014 14 Work Programme (7/8) QoE-diagnostic information from P.120X.Y type models on technical causes of QoE impairments Integral index of quality for general service monitoring (per user-session); KQI definitions Media-session QoE model (temporal pooling, long sequences; UDP- and TCP-based streaming) Parametric non-intrusive assessment of audiovisual progressive download, considering stalling/re-buffering & quality integration Parametric non-intrusive assessment of TCP-based multimedia streaming quality, considering adaptive streaming In-service non-intrusive measurement device – Voice service measurements Analysis and interpretation of INMD voice-service measurements Conformance testing for voice over IP transmission quality assessment models Conversational model Framework for invoking diagnostic functions Voice Service Diagnosis Framework Maputo – Mozambique – 14 – 16 April 2014 15 Rec. G.1029 - Voice Service Diagnosis Framework Terminals Framework for invoking diagnostic functions (Work Item 1) QoE/QoS Decision parameters Decision process Invocation network QoE/QoS Diagnostic parameters Diagnostic parameter to root-cause mapping QoE/QoS Root-cause analysis parameters Framework for mapping diagnostic parameters to root-cause parameters (Work Item 2) Remedial action QoE/QoS Root-cause analaysis application Maputo – Mozambique – 14 – 16 April 2014 16 Work Programme (8/8) Technical cause analysis Buffer models for development of client performance metrics Hand-over performance among multiple access networks Internet protocol data communication service – IP packet transfer and availability performance parameters Network performance objectives for IP-based services Measurements in IP networks for inter-domain performance assessment Ethernet frame transfer and availability performance Ethernet service activation test methodology Home network performance parameters Proposed Appendices for Y.1566 : QoS mapping and interworking between Ethernet, IP and MPLS Maputo – Mozambique – 14 – 16 April 2014 17 Cannot find your Topic ? Work in ITU-T is contribution driven ! Next Meeting of Study Group 12 2 – 11 September 2014 in Cartagena de Indias, Colombia (tbc) Deadline for Contributions 20 August 2014 http://www.itu.int/net/ITU-T/ddp/Default.aspx?groupid=T13-SG12 Maputo – Mozambique – 14 – 16 April 2014 18 Any questions ? Joachim Pomy [email protected] Maputo – Mozambique – 14 – 16 April 2014 19