Transcript Contents Terms of reference Highlights of achievements / Projects / FGs Future work Conclusions Supplemental slides ITU-T Study Group 12 Performance and Quality of Service International Telecommunication Union.

Contents
Terms of reference
Highlights of achievements / Projects /
FGs
Future work
Conclusions
Supplemental slides
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
2
Terms of Reference
Responsible for Recommendations on the endto-end transmission performance of
terminals and networks, in relation to the
perceived quality and acceptance by users of
text, data, speech, and multi-media applications.
Although this work includes the related
transmission implications of all networks and all
telecommunication terminals, a special focus is
given to IP QoS, interoperability and
implications for NGN, and also includes work on
performance and resource management.
Lead SG for Performance and Quality of Service
(QoS)
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
3
Highlights of achievements (I)
The test methodologies for subjective
assessment of speech and audio are always
evolving to take into account the new services
and the new environments
SG12
is assessing the quality of codecs (26
test plans and 15 test results)- Collab. with
SG16
New Recommendation P.805 for conversation
test.
The creation of 2 Focus Groups on car
communication FITCAR (closed in Jan.2008)
and CARCOM(opened in June 2008)
NewRecommendation
P.505 " One-view
visualisation of
quality" "Quality
Pie"
Round Robin
Test on
Human/artificial
ears impedance
 Planned
update of P.57
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
4
Highlights of achievements (II)
One of the major area on which SG12 has the global
leadership is the development of quality models and tools
for speech, but now also for data and multimedia services
Recommendation G.107 ("E-model") is updated on a
regular basis. One of the major improvements has been to
cover wideband speech.
Perceptual models for narrow and wideband speech
–
–
–
P.862.x-series Recommendations give a complete set of
models and information on how to use them. These types of
models are working with reference signal.
P.862.2 Wideband extension to Recommendation P.862
P. 862.3 Application guide for objective Quality
Measurement based on Recommendation P.862
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
5
Highlights of achievements (III)
"In-service non-intrusive" quality models,
without reference signal.
Recommendation P.564, which has also been
implemented for wideband speech, specifies the
minimum criteria for objective speech quality
assessment models that predict the impact of observed
IP network impairments on the one-way listening quality
experienced by the end-user in IP/UDP/RTP-based
telephony applications.
A call for proposal has been published for P.NAMS
(Non Intrusive assessment of performance
for video streaming)  2009
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
6
Highlights of achievements (IV)
Recommendations for other media than speech,
e.g. for videophone (G.1070) and for
multimedia (G.1050).
–
–
G.1070 provides  algorithm estimating videophone
quality to be used by QoS/QoE planners to ensure end
user satisfaction. Recommendation
G.1050  a network model for evaluating multimedia
transmission performance over IP; based on
statistical models on a broad range of known network
configurations.
Recommendation G.1030 which provides a
perceptual model for web browsing and defines
a framework of tools to estimate end-to-end IP
network performance for user applications.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
7
Highlights of achievements (V)
One of the most popular Recommendations
(Y.1541), initiated in SG13, has been updated
several times by SG12 during the study period, to
cover new services and media,
e.g. Appendix VIII on effects of IP network performance
on digital television transmission QoS.
This fundamental Recommendation should be the core for
a global end-to-end QoS for interoperability.
–
The set of existing Recommendations has been
completed by Y.1542, "a Framework for
achieving End-to-End Performance
objectives", linked with Y.1541.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
8
Highlights of achievements (VI)
SG12 experts and Officials has been active in
Focus Groups and several Recommendations
have been developed :FG-IPTV-GSI (G.1080 and
1081 have been approved and two other Rec. for
monitoring are under progress), FG-NGN-GSI
(Y.1542 has been approved, TR-NGN.QoS and TRNGN.NHNperf are under progress).
A new recommendation addressing Ethernet
performance, Recommendation Y.1563
"Ethernet frame transfer and availability
performance" has been approved in May 2008.
For SLA (service level agreements) matters,
G.1040 provides performance metric to be used
by network service providers.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
9
Future Work (I)
For speech and audio quality : The hot topics are
"bandwidth extension" (superwideband and
fullband) and "binaural perception"



consequences on loudness, binaural modelling,
acoustical interfaces (for terminals and test equipments
such as artificial ear an mouth) and test methodologies
(subjective and objective test methods).
Result from FG CARCOM : a specification for
wideband car communication
To identify end-user performance expectations
and associated metrics for audio, video, text,
graphics quality, and control functionality.
 to improve the accessibility of services.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
10
Future Work (II)
Industry needs to assess Quality of all types of media.
SG 12to work on audiovisual quality, and visual and
multimedia quality measurements in telecommunication
services.
This work to be done in close partnership with VQEG.
SG12, the major actor for the development of quality
modelling for speech. This leadership should be increased.
 development of perceptual and parametric models for
voice, audio, audiovisual and multimedia,
 application of measurement tools and development
and application of all planning and measurement tools.
 framework for diagnostic functions and their
interaction with external objective models predicting media
quality.
 operational aspects of QoS, for the benefits of in
particular developing countries.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
11
Future Work (III)
Quality models – Next future
- New objective models for quality predictions are expected
for speech quality evaluation (P.OLQA) at the end of 2009
(call already sent) and for objective prediction of P.835
scores (P.ONRA).
- The plans to define a model for spoken conversations
have been agreed.
- Regarding multimedia, the call for proponents for
P.NAMS has been sent out  a new Recommendation
planned for 2009, Next step following P.NAMS : P.NBAMS
a non-intrusive bit stream model for the assessment of
performance of multimedia streaming.
QoE, QoS and Perf for multimedia (IPTV, HN)
 G.OMVAS “Opinion Model for Video and Audio
Streaming applications” and
 Recs. on IPTV Quality monitoring (in addition to G. 1081)
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
12
Future Work (IV)
NGN performance interworking issues requiring
consideration include end-to-end multimedia performance
interworking and interoperability issues as well as impacts
of interworking functions.
To define static and dynamic apportionment of performance
objectives across networks and technologies.
To examine the suitability of numerical objectives of IP QoS
provisional Classes 6 and 7 given in Recommendation
Y.1541.
–
e.g. for MPEG-4 video and various Audio schemes.
Urgent topics to be solved : IP Service Availability
applicable in the network operator’s environment
For QoS, QoE and Performance, SG12 plans to contribute
significantly to improve the interoperability and
convergence of services and networks.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
13
Future Work (V)
• A special focus is given to interoperability
to ensure end-to-end users' satisfaction.
• SG12 also provides cross-ITU coordination
for the many aspects of Quality-of-Service
(QoS) and Quality of Experience (QoE)
and to foster consistency among ITU-T
Study Groups, as well as with other
related bodies such as the IETF, 3GPP so
as to advance the interoperability of any
QoS solutions.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
14
Conclusion
In order to cover the future plans, it is proposed
to rename Study Group 12 to “Performance,
QoS and QoE” and define its mandate as
follows:
Study Group 12 is responsible for Recommendations on
performance, QoS and QoE for the full spectrum of
terminals, networks and services ranging from speech
over fixed circuit based networks to multimedia
applications over networks that are mobile and packetbased.
A special focus is given to interoperability to ensure endto-end users' satisfaction.
SG12 is Lead Study Group on QoS and QoE.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
15
Supplemental Slides
Management team
Structure and Implementations
Working Parties
Statistics
Workshops
QoS Flyer
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
16
Management Team
Klemens Adler (G)
VP & WP2/12 Chair
Paul Coverdale (Ca)
VP & WP3/12 Chair
Catherine
Quinquis (F)
Chuck Dvorak (US)
VP & Lead SG Resp.
Judit Katona
Kiss, TSB
WP1/12
Co-Chair
Gérald Lecucq (F)
WP1/12 co-Chair
Missing:
JY Monfort ,SG Chair, is behind the camera
ITU-T Study Group 12
Performance and Quality of Service
JeanJacques
MassimaLandji
(Gabon) VP
International
Telecommunication
Union
17
Study Group Structure
WP1 Telephonometry, Terminals and
Subjective assessment
Develops Recommendations on artificial heads/ears/Mouths/signals
and associated test methods
Develops subjective methods and test plans
WP2 Performance, Planning and Tools
Develops quality models for speech and multimedia (E-Model,
Perceptual models, INMD,…)
WP3 IP-Related QoS and resource
management
Defines parameters and models for QoS classes and IP
performance.
Develop frameworks for IP-related QoS
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
18
Study Group Implementations
Focus groups FITCAR and CARCOM
After 5 meetings, the specification on characteristics and test
methods for narrowband speech terminals in cars has been finalised
by FG FITCAR. The specification has been submitted to SG12. The
draft new Recommendation P.1100, based on the specification
produced by the Focus Group FITCAR had consent at the last
meeting of SG12.
The new Focus Group CARCOM is focused on Wideband Speech
APSC-Telemov
The Advisory Panel for Standards Cooperation on
Telecommunications related to Motor Vehicles - APSC TELEMOV was
created at the end of the previous study period as a result of the
Workshop on Standardization in Telecommunication for Motor
Vehicles and intended as a cooperation group on all aspects of
standardization related to telecommunications within and for motor
vehicles.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
19
Study Group Implementations
SG12 Regional Group for Africa
To encourage to take an active participation in
work of ITU-T SG12 by African Regulators and
Operators, and in the implementation of the ITU-T
Recommendations on Performance/QoS and QoE…
JRG on Multimedia Quality assessment has
been formed from SG12 and SG9 in early 2004
with the motivation to represent the interest of
ITU-T within VQEG. The work of JRG is successfully
focussing on Multimedia quality aspects within
VQEG Multimedia (MMQA) working group
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
20
WP 1 /12 – Telephonometry, terminals
and Subjective assessment
Speech transmission characteristics of speech
Q.3/12
terminals …
Mr Shi Denian
Q.4/12
Hands-Free communications in vehicles
Mr H.W. Gierlich
Q.5/12
Telephonometrics methodologies for handsets
and headset terminals
Mr L. Madec
Q.6/12
Analysis methods using complex measurement
signals….
Mr H.W. Gierlich
Q.7/12
Methods, tools and test plans for the subjective
assessment… Mr P. Usai and Mrs C. Quinquis
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
21
WP 2/12 – Performance, Planning and Tools
Q.8/12
Extension of the E-Model
Mrs U. Jekosch and Mr S. Möller
Q.9/12
Perceptual-based objective methods for voice,
audio and visual quality…
Mr J. Berger
Q.10/12
Transmission planning and performance
considerations…
Mr V. Siply
Q.12/12
Performance evaluation of services based on
speech technology Mr S.Möller&MrA. Raake
Q.14/12
In-Service Non Intrusive Assessment of voice
transmission performance
Mr V. Barriac
Q.20/12
Objective assessment of conversational speech
Quality in networks
Mr J. Pomy
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
22
WP 3/12 – IP-Related QoS andRessource
management
Q.2/12
Speech transmission Charact. And Meas.methods
for IP terminals and gateways Mr U. Brunner
Q.11/12
IP based multimedia services end to end transm.
planning ….
M. J. Pomy
Q.13/12
IPTV and multimedia QoE/QoS performance
requirements and assessment methods A. Takahashi
Q.16/12
Broadband and IP-related resource
management
Mr D. Mustill
Q.17/12
Q.19/12
Performance of IP-based networks
Mr A. Morton -Mr H. Kim(associate)
Call processing performance
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
23
Statistics
During the study period, Study Group 12
examined a total of 364 contributions
On the basis of these documents and of a
thousand of temporary documents, Study Group
12:
– drew up 21 new Recommendations;
– amended/revised 23 Recommendations;
– developed 1 Supplement,
– developed 27 Test plans and 15 Test
reports in the scope of collaboration with ITU-T
SG16 for speech and audio coders.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
24
Workshops organized by SG12 (I)
End-to-End QoS-QoE / 14-16 June 2006 / Geneva
http://www.itu.int/ITU-T/worksem/qos/200606/index.html
Organised by SG12 with the collaboration with SG2 and SG16
The sessions were dedicated to Inter-domain QoS,
Interoperability testing and terminal issues, Home
Networking QoS/QoE, Quality of Experience (QoE) of new
services, NGN-related QoS and the accumulation of network
impairements, and Quality prediction
This workshop brought together more than 100 experts
from all over the world who are involved in the definition
and the delivery of end-to-end quality of experience (QoE)
and quality of service (QoS).
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
25
Workshops organized by SG12 (II)
From Speech to Audio/10-12 Sept 2008/Lannion, France
http://www.itu.int/ITU-T/worksem/speechaudio/index.html
The workshop was organized by WP1/12 and involved participants from
ITU-T SG12, SG16, ITU-R 6C. Representatives of ETSI STQ, 3GPP SA4 also
joint the workshop that was supported by France Telecom, Alcatel Lucent
and DEGA (the German acoustical society)
The workshop was very productive
and agreed on some actions
- to enhance the constructive dialogue
between universities, standardization
bodies and companies.
Ideas to improve the future collab
have been raised need to be
implemented
- to provide inputs to develop new
methodologies covering the complete
audio bandwidth from Narrow Band to
Fullband  New Recs. There is a need
for improved collab between ITU-T and
ITU-R for audio quality.
Keynote Speakers
Pr. J. Blauert
Bochum Univ.
Pr. S. Meunier
Univ. Aix-Marseille
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
26
Workshops supported by SG12
The Fully Networked Car, Geneva (Palexpo)
–
–
http://www.itu.int/ITU-T/worksem/ict-auto/index.html
2-4 March 2005
7 - 9 March 2007
5 - 7 March 2008
The next one : 4 - 5 March 2009
Opportunities and Challenges in Home Networking Geneva,
13 - 14 October 2005
2nd ETSI STQ Workshop on Wideband Speech Quality in
Terminals and Networks: Assessment and Prediction, 22-23
June 2005, Mainz, Germany
ETSI Workshop on 'Effects of transmission performance on
Multimedia QoS'
Prague, Czech Republic, 17-19 June 2008
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
27
A QoS Flyer (page 2)
True Quality of Service captures the User Experience (QoE)
The G.1000 Framework helps capture the many QoS dimensions, and
acknowledges that QoS can be viewed from four different perspectives
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
28
Acknowledgements
Thanks to the Management Team (including
Rodolfo Ceruti) for the strong support, to Hans
Gierlich (as Chair of the Focus Group and a Fully
networked car workshop), to the efficient
Rapporteurs and contributors and to Daniel
Waturu for his involvement in SG12 Regional
Group for Africa.
Many thanks to TSB support and specially Judit
and Sarah for SG12, Sami and his team for the
specific websites (E-model,..), Paolo and his team
for the organization of workshops, Sylvie, Kristin
and Richard for the meetings facilities.
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
29
Thank you!
Mr Jean-Yves Monfort
France Telecom/Orange-Labs
Standardization Senior Manager & Senior expert in QoS
During three years, I was deputy Director for standards
in France Telecom R&D.
I was previously involved as deputy Head of a Lab.
Dealing with speech, Audio, video and usability Quality
of services, terminals and equipments
One of my hobbies
ITU-T Study Group 12
Performance and Quality of Service
International
Telecommunication
Union
30