Transcript ITU Workshop on “Quality of Service and Quality of Experience of Multimedia Services in Emerging Networks” (Istanbul, Turkey, 9-11 February 2015) Monitoring End-User.

ITU Workshop on “Quality of Service and
Quality of Experience of Multimedia Services in Emerging Networks”
(Istanbul, Turkey, 9-11 February 2015)
Monitoring End-User QoS for
Data Services
Alberto Pérez
Business Development Consultant, Omnitele
[email protected]
agenda
©Omnitele Ltd. 2015
2
1.
Introduction
− About Omnitele
2.
Customer Experience Challenge
KPIs that capture the End-User experience
3.
Methodologies for Monitoring End-User QoS
Different approaches – one goal
4.
Take-outs
Summary
Maximised Experience, Minimised Cost
International And Independent
We provide consulting and expert services for telecom
operators and regulators in network strategy, design
and quality assurance. Our mission is to maximise
mobile subscriber quality of experience and minimise
operator network expenditures.
Our headquarters is located in Helsinki, Finland. We
have local presence in the Netherlands and the
Hashemite Kingdom of Jordan. Our company is owned
by Finnish telecom investors and we are independent
of operator groups and network vendors.
Delivering Omnitele Experience
The Omnitele Way
The company was founded in 1988 to set up world’s
first GSM network. Since then we have completed over
1000 projects in over 80 countries around the globe.
Always delivering Omnitele Experience – a fact proven
by our long lasting client relationships.
Our unique way of working sets us apart from the
competition and gives us a strong identity in the world
of telecommunications. We call this the Omnitele Way,
which means being Straightforward, Trusted and
Intelligent.
www.omnitele.com
©Omnitele Ltd. 2015
3
our services
Technology
Strategy
Design and
Optimisation
Audit and
Benchmark
Performance
Management
maximised customer experience
minimised network cost
©Omnitele Ltd. 2015
4
agenda
©Omnitele Ltd. 2015
5
1.
Introduction
− About Omnitele
2.
Customer Experience Challenge
KPIs that capture End-User experience
3.
Methodologies for Monitoring End-User QoS
Different approaches – one goal
4.
Take-outs
Summary
Monitoring End-User QoS in…
…pre-HSPA networks
General user experience
depends on two basic services
Voice Services
Data Services
MOS, Success Rates, Drop
Rates, Setup Time
Throughput
©Omnitele Ltd. 2015
6
Generally available bitrates are not sufficient for data
services. As throughput is the bottleneck, it becomes the
measure to define the QoS
Customer Experience Challenge
Bitrate is no longer the ultimate mobile
network quality indicator
QoS: WWW page waiting time [s]
1000KB WWW Page Waiting Time vs. Bitrate
HSPA
10
LTE
8
6
4
2
0
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
NW Performance: Bitrate [Mbit/s]
©Omnitele Ltd. 2015
7
Customer Experience Challenge
big difference in NW performance…
…thin margins in Quality of Service
bitrate [Mbit/s]
buffering time [s]
40
2
30
1.5
20
1
10
0.5
0
0
T-Mobile
Vodafone
KPN
T-Mobile
Vodafone higher bitrate,
but KPN faster YouTube
©Omnitele Ltd. 2015
8
Vodafone
KPN
Customer Experience Challenge
QUALITY OF EXPERIENCE
How well service quality meets expectations?
overall Satisfaction, e.g. 1-5 ”stars”
Highly market dependent and user segment specific
QUALITY OF SERVICE
How well does a service perform
Measured application/service KPIs
call completion rate sms send time sms
completion rate file transfer time www page waiting time www
page success rate video buffering time
video setup success rate
call setup time
product of one or more network performance factors
NETWORK PERFORMANCE
Technical measurements of the network
e.g., delay, jitter, packet loss, throughput...
©Omnitele Ltd. 2015
9
RSCP
Ec/N0
latency
jitter
packet loss
RSRP
RLC throughput
RSRQ
RAB setup success rate
modulation
coding
TX power
G-factor
Channel C/I
Timing Advance
Mobile TX Power
Voice Codec Usage
Handover Success Rate
MCS Usage Distribution
Time Slot Utilization
Block Error Ratio
Active Set
Size
SHO Success Rate
ISHO Success Rate
CQI
E-DPDCH Throughput
PDSCH
modulation
MAC DL BLER
MAC UL BLER
MAC UL Retransmission Rate
Customer Experience Challenge
all-IP PS network
OFDMA access scheme
spectral efficiency 15 bit/s/Hz
SC-FDMA uplink
adaptative modulation
low latency
300 Mbps
20 MHz bandwidth
Carrier Aggregation
improved cell-edge performance
8x8 MIMO
©Omnitele Ltd. 2015
10
Customer Experience Challenge
all-IP PS network
news refreshing time?
OFDMA access scheme
app download
spectral efficiency 15 bit/s/Hz
time?
SC-FDMA uplink
it takes too long to
adaptative
upload
a pic in modulation
Instagram…
low latency
300 Mbps
©Omnitele Ltd. 2015
11
20 MHz bandwidth
facebook
Carrier Aggregation
refreshing
improved cell-edge performance
time?
8x8 MIMO
agenda
©Omnitele Ltd. 2015
12
1.
Introduction
− About Omnitele
2.
Customer Experience Challenge
KPIs that capture End-User experience
3.
Methodologies for Monitoring End-User QoS
Different approaches – one goal
4.
Take-outs
Summary
Monitoring Methodologies
in mobile networks
pasive
monitoring
PMS/Network Counters
CDR
Crowdsourcing
©Omnitele Ltd. 2015
13
active
monitoring
Probes (static/mobile)
Drive Testing
Monitoring Methodologies
in mobile networks
pasive
monitoring
PMS/Network Counters
CDR
Crowdsourcing
©Omnitele Ltd. 2015
14
Vast amount of data
24/7 live monitoring
Big picture of network utilisation trends
Not available KPIs on End-User level
Challenging implementation (Crowdsourcing)
Average results do not tell much
Monitoring Methodologies
in mobile networks
active
monitoring
Probes (static/mobile)
Drive Testing
©Omnitele Ltd. 2015
15
Controlled environment
Reproduces End-User cases
Collects essential data for troubleshooting
Amount of samples is limited
Limited coverage information
Expensive exercise (Drive Testing)
Monitoring Methodologies
best methodology?
PMS/Network Counters
Drive Testing
Probes
End-User
Monitoring
Crowdsourcing
a combination of all of them
©Omnitele Ltd. 2015
16
CDR
Monitoring Methodologies
set up a process
©Omnitele Ltd. 2015
17
4. Correlate
data from
different
sources
1. What do I
want to
Monitor?
3. What
tools do I
have
available?
2. How my
subscribers
use the
service?
”Be-the-Customer” Approach
Methodology for capturing the true end-user experience
when using mobile networks
WHERE | Dense population areas independently
selected by Omnitele
WHEN | Test focus on peak hours, no empty
network testing
HOW | Terminals and test cases as per real
subscriber behaviour
©Omnitele Ltd. 2015
18
Latvia Benchmark – SoW
Country-wise Benchmark
3 Operators
VS
©Omnitele Ltd. 2015
19
Latvia Benchmark – SoW
Hotspot Testing
Web Browsing
YouTube
Drive Testing
Facebook
File Transfers
Instagram
Twitter
Dropbox
App Testing
Network Performance Testing
Tests to be carried out with latest
Samsung Galaxy S terminal
(4G/3G/2G floating mode)
©Omnitele Ltd. 2015
20
Voice Calls
Latvia Benchmark – SoW
Hotspot Testing
Drive Testing
212 locations
+2 000 km
#
City
1
2
3
4
5
6
7
8
Population Hotspots
643
87
71
57
49
36
29
24
368
403
926
332
750
677
948
446
115
16
13
11
9
7
6
5
23
23
17
16
15
657
269
606
751
828
5
5
4
3
3
14 Olaine
15 Kuldīga
11 543
3
11 392
3
16 Sigulda
17 Saldus
10 997
2
9
10
11
12
13
Rīga
Daugavpils
Liepāja
Jelgava
Jūrmala
Ventspils
Rēzekne
Ogre
Valmiera
Jēkabpils
Tukums
Salaspils
Cēsis
TOTAL LATVIA
10 895
2
212
all towns +10k inhabitants
©Omnitele Ltd. 2015
21
Hotspot to Hotspot Routes
Main Connecting Roads
agenda
©Omnitele Ltd. 2015
22
1.
Introduction
− About Omnitele
2.
Customer Experience Challenge
KPIs that capture End-User experience
3.
Methodologies for Monitoring End-User QoS
Different approaches – one goal
4.
Take-outs
Summary
Summary
1. Data throughput is not longer the unique KPI to take into
consideration when monitoring Mobile Data QoS
2. Different Monitoring Methodologies give you insights to
different type of data – everything is valuable!
3. Set a process that helps achieving your goals, taking into
consideration the actual users of the service
©Omnitele Ltd. 2015
23
thank you for your attention!
questions
©Omnitele Ltd. 2015
24
We answer
Contact details
Omnitele Ltd
Name
Alberto Pérez
Phone
+358 9 695 991
Title
Business Development Consultant
Email
[email protected]
Phone
+358 40 537 80 52
Website
www.omnitele.com
Email
[email protected]
Address
Omnitele Ltd. Mäkitorpantie 3B P.O. Box 969 00101 Helsinki, Finland
©Omnitele Ltd. 2015
25
max imis ed c u s t omer ex p erien ce
min imis ed n et w ork c os t
©Omnitele Ltd. 2015
26
www.omnitele.com