Transcript Yu-Chun Chang, Po-Han Tseng, Kuan-Ta Chen, Chin

Understanding the Performance of
Thin-Client Gaming
Yu-Chun Chang1, Po-Han Tseng2, Kuan-Ta Chen2, and Chin-Laung Lei1
1Department
of Electrical Engineering, National Taiwan University
2Institute of Information Science, Academia Sinica
2011/5/11
CQR 2011 / Yu-Chun Chang
1
Outline
• Introduction
• Experiment methodology
– Experiment setup
– Performance metric extraction
• Performance evaluation
• Conclusion & future work
2011/5/11
CQR 2011 / Yu-Chun Chang
2
Introduction (1/2)
• Thin-client system
User’s inputs
Display updates
Client
2011/5/11
Server
CQR 2011 / Yu-Chun Chang
3
Introduction (2/2)
• Motivation
– To understand which performance metric is more
sufficient for thin-client gaming
• Frame rate, frame delay, frame loss, and etc
• Challenges
– Most thin-client products are proprietary
• Image compression, data-transmission protocol and display update
mechanism
2011/5/11
CQR 2011 / Yu-Chun Chang
4
Our focus
Thin-client program
Network
Condition
Perf.
Metric
QoE
Server
User
Client
2011/5/11
CQR 2011 / Yu-Chun Chang
5
Outlines
• Introduction
• Experiment methodology
– Experiment setup
– Performance metric extraction
• Performance evaluation
• Conclusion & future work
2011/5/11
CQR 2011 / Yu-Chun Chang
6
Experiment Methodology
2011/5/11
7
Why Use Ms. Pac-Man?
• Move Pac-Man to eat pills and get the score
• Control through thin-client applications and move
Pac-Man in the game of server
– Good network condition: score↑
– Bad network condition: score↓
• Score  Quality of Experience
2011/5/11
CQR 2011 / Yu-Chun Chang
8
Ms. Pac-Man & Bot
• Ms. Pac-Man
– Save score after the pacman ran out of 3 lives
Number
Score
220
10
Power pill
4
50
Ghost
4
200 (after eating power pills)
Pill
• Bot: ICE Pambush3 (published in IEEE CIG 2009)
– Java-based controller to move the pacman
– Capture the screen of the game and determine the position of the
pacman, ghosts, and pills
2011/5/11
CQR 2011 / Yu-Chun Chang
9
• Three thin-client systems
– LogMeIn
– UltraVNC
– TeamViewer
• Network conditions
2011/5/11
Network condition
Settings
Network delay
0 ms, 100 ms, 200 ms
Network loss rate
0%, 2.5%, 5%
Bandwidth
Unlimited, 600 kbps, 300 kbps
CQR 2011 / Yu-Chun Chang
10
• Performance metric
– Display frame rate
– Frame distortion (MSE: Mean Square Error)
• Record game play as video files in 200 FPS
2011/5/11
CQR 2011 / Yu-Chun Chang
11
Outlines
• Introduction
• Experiment methodology
– Experiment setup
– Performance metric extraction
• Performance evaluation
• Conclusion & future work
2011/5/11
CQR 2011 / Yu-Chun Chang
12
Thin Clients are Different!
2011/5/11
CQR 2011 / Yu-Chun Chang
13
Visual Difference Really Matters!
2011/5/11
CQR 2011 / Yu-Chun Chang
14
Statistical Regression
Independent factors
Display frame rate
Frame distortion
2011/5/11
Regression
Model
CQR 2011 / Yu-Chun Chang
QoE
(score)
15
Frame-Based QoE Model
• Linear model
• QoE =
Adjusted R-squared: 0.72
2011/5/11
CQR 2011 / Yu-Chun Chang
16
Frame-Based QoE Model
17
Which Performance Metric is More Sufficient?
• QoE degradation
– Optimal user’s QoE – user’s QoE predicted by model
• Frame rate is
more sufficient!
2011/5/11
18
Frame Rate and Network Conditions
Thin-client program
Network
Condition
Perf.
Metric
QoE
Server
User
Client
2011/5/11
CQR 2011 / Yu-Chun Chang
19
The Frame Rate Prediction Model
• Frame rate =
• app1, app2: dummy variables
– LogMeIn :
app1 = 1, app2 = 0
– TeamViewer : app1 = 0, app2 = 1
– UltraVNC :
app1 = 0, app2 = 0
• d: delay,
l: loss rate,
b: bandwidth
• dl, dt, du : delay of LogMeIn, delay of TeamViewer, delay of UltraVNC
2011/5/11
CQR 2011 / Yu-Chun Chang
20
The Frame Rate Prediction Model
Delay of LogMeIn
Delay of UltraVNC
Bandwidth of LogMeIn
Bandwidth of UltraVNC
Adjusted R-squared: 0.85
2011/5/11
CQR 2011 / Yu-Chun Chang
21
Predicted Frame Rate
Network delay
2011/5/11
Bandwidth
CQR 2011 / Yu-Chun Chang
22
Which Thin-Client is Better?
Thin-client program
Network
Conditions
Perf.
Metric
QoE
Server
User
Client
2011/5/11
CQR 2011 / Yu-Chun Chang
23
Network-Based QoE Model
• QoE =
2011/5/11
Adjusted R-squared: 0.81
The Thin-Client with Best Performance
• o symbol: empirical network condition
– 300 records collected by PingER project
2011/5/11
CQR 2011 / Yu-Chun Chang
25
Conclusions & Future Work
• Display frame rate and frame distortion are both
critical to gaming performance on thin-clients
• LogMeIn performs the best among the three
implementations we studied
• Future work
– Add more thin-clients to see comparisons of performance
– Design a generalizable experiment methodology for thinclient gaming with different game genres
2011/5/11
CQR 2011 / Yu-Chun Chang
26
Thank you for your attention!
2011/5/11
CQR 2011 / Yu-Chun Chang
27