Transcript Slides
A Provider-side View of
Web Search Response Time
YINGYING CHEN, RATUL MAHAJAN,
BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA)
MICROSOFT
Web services are the dominant way
to find and access information
Web service latency is critical to
service providers as well
Google
Latency
+0.5 sec
revenue
-20%
Bing
Latency
+2 sec
revenue
-4.3%
Understanding SRT behavior is challenging
SRT (ms)
300+t
t
W
T
Th
F
200+t
SRT (ms)
M
t
peak
off-peak
S
Su
Our work
Explaining
Identify
Root
systemic SRT variation
SRT anomalies
cause localization
Client- and server-side instrumentation
𝑇𝑓𝑠
𝑇ℎ𝑒𝑎𝑑
𝑇𝑓𝑐
𝑇𝑠𝑐
𝑇𝑏𝑟𝑎𝑛𝑑
𝑇𝑖𝑛𝑡𝑐ℎ𝑘1
𝑇𝑟𝑒𝑠𝐻𝑇𝑀𝐿
𝑇𝑡𝑐
𝑇𝐵𝑂𝑃
𝑇𝑖𝑛𝑡𝑐ℎ𝑘2
𝑇𝑒𝑚𝑏𝑒𝑑
𝑇𝑟𝑒𝑓
𝑇𝑠𝑐𝑟𝑖𝑝𝑡
Referenced
content
Impact Factors of SRT
server
𝑇𝑓𝑠
network
browser
query
𝑇ℎ𝑒𝑎𝑑 𝑇𝑏𝑟𝑎𝑛𝑑 𝑇𝑖𝑛𝑡𝑐ℎ𝑘1 𝑇𝑟𝑒𝑠𝐻𝑇𝑀𝐿 𝑇𝐵𝑂𝑃 𝑇𝑖𝑛𝑡𝑐ℎ𝑘2 𝑇𝑒𝑚𝑏𝑒𝑑 𝑇𝑠𝑐𝑟𝑖𝑝𝑡 𝑇𝑟𝑒𝑓 𝑇𝑓𝑐 𝑇𝑠𝑐 𝑇𝑡𝑐 𝑇𝑛𝑒𝑡
Primary factors of SRT variation
Apply
Analysis of Variance (ANOVA) on the time intervals
𝑉𝑎𝑟 𝑆𝑅𝑇 =
SRT
variance
𝑘 𝑉𝑎𝑟
𝑇𝑘 , 𝑆𝑅𝑇
+ ƞ
Variance explained Unexplained
variance
by time interval k
Explained
variance (%)
60
40
20
0
𝑇𝑛𝑒𝑡
server
𝑇𝐵𝑂𝑃
𝑇𝑟𝑒𝑓
𝑇𝑠𝑐𝑟𝑖𝑝𝑡
network
𝑇𝑟𝑒𝑠𝐻𝑇𝑀𝐿
𝑇𝑓𝑐
𝑇ℎ𝑒𝑎𝑑
browser
𝑇𝑠𝑐
𝑇𝑡𝑐
query
Primary factors: network characteristics, browser speed, query type
Server-side processing time has a relatively small impact
RTT
Variation in network characteristics
Explaining network variations
Residential
networks send a higher fraction of
queries during off-peak hours than peak hours
Residential
networks are slower
residential
enterprise
unknown
RTT (ms)
1.25t
25%
t
residential enterprise
Residential networks are slower
Residential networks send a higher fraction of
queries during off-peak hours than peak hours
Variation in query type
Impact of query on SRT
Server
processing time
Richness
of response page
Measure: number of image
Explaining query type variation
Peak hours
Off-peak hours
Browser variations
Two most popular browsers: X(35%), Y(40%)
Browser-Y sends a higher fraction of queries during off-peak hours
Browser-Y has better performance
Javascript
exec time
1.82t
82%
t
Browser-X Browser-Y
Summarizing systemic SRT variation
Server:
Little impact
Network:
Query:
Poorer during off-peak hours
Richer during off-peak hours
Browser:
Faster during off-peak hours
Detecting anomalous SRT variations
Challenge:
interference from systemic variations
Week-over-Week (WoW) approach
𝑆𝑅𝑇 = Long term trend + Seasonality + Noise
Comparison with approaches that do not
account for systemic variations
WoW
False negative
10%
False positive
7%
One Gaussian Change point
model of SRT
detection
35%
40%
17%
19%
Conclusions
Understanding
SRT is challenging
Changes
in user demographics lead to systemic
variations in SRT
Debugging
Must
SRT is challenging
factor out systemic variations
Implications
Performance
Should
understand performance-equivalent classes
Performance
Should
management
consider the impact of network, browser, and query
Performance
monitoring
debugging
End-to-end measures are tainted by user behavior changes
Questions?