Transcript Overview of Hyper-Proxy Project

Overview of Hyper-Proxy Project:
High-Quality Streaming Media Delivery
(an academia-industry collaboration)
Xiaodong Zhang
(Major Team Members: Songqing Chen, Bo Shen, Susie Wee )
Founded by NSF and HP Labs
1
Proxy Caching
Reduce
network traffic
Reduce
response
time to client
Reduce
server’s load
Server
Intermediary
Client
2
Existing Internet Status
Very large sizes
A large number
of proxies with:
disk, memory,
and CPU cycles
Media Objects
Very rigorous
real-time delivery
constrains:
small startup
latency,
continuous
delivery
Servers
Servers
Intermediaries
Intermediaries
Clients
Clients
Diverse client
access devices:
computers,
PDAs,
cell-phones
3
A Missing Component in Proxy Caching
• Current streaming media delivery status
(WWW’05)
– Client-Server Model (poor scalability, heavy traffic)
Downloading (about 90%).
Many media servers limits downloading to protect
copyrights.
– CDNs (very expensive, idle resources)
Dedicated services.
• Why not leverage existing proxies in Internet?
4
Typical Engineering Solutions
• Examining existing proxy structure: Squid is an open
source software.
• Redesign Squid: add the streaming function.
• Integrating segmentation methods: restructure the
proxy to make it segmentation enable.
• Testing and evaluating the streaming proxy.
5
Three Aimed Impacts
• Academic Impact: influencing the research
community via publications, and producing
strong Ph.Ds.
• HP Product Impact: the research prototype is
identified by product divisions for technology
transfer.
• Industry Impact: a successful technology
transfer or a product becomes demanding in
industry or in the market.
6
Aimed Impacts from NSF
• Intellectual merits and challenges:
–
–
–
–
Knowledge Advancement and Discovery.
Originality and creativeness of the research.
Problem solving ability and qualifications.
Activity organization and resource availability.
• Broader impact:
– Wider applications and deeper impact to society.
– Education impact and diverse workforce training (e.g.,
gender, ethnicity, disability, geographic, etc. )
– Dissemination of the tools and software in public.
– Enhancement of infrastructure for research and education
7
Current Solutions for Proxy Streaming
• Dealing with large size
–
Object segmentation – partial caching
• Along the quality domain
• Along the viewing time domain
• Reducing startup latency, increasing Byte Hit Ratio
–
Priority caching – caching beginnings popular objects.
• Guaranteeing continuous streaming
– Little research done
8
The State-of-the-art of
Segmentation & Priority Caching
• Prefix Caching (INFOCOM’99-’02)
prefix
Priority: lowsuffix
startup latency, bursty smoothing
• Uniform Segmentation (JSAC’02)
…………………….
Priority:
load balance, low startup latency
• Exponential Segmentation (WWW’01)
……………………..
Priority: low
startup latency, quick replacement
• Adaptive-Lazy Segmentation (NOSSDAV’03)
Priority: high byte hit ratio
Existing strategies emphasize on byte hit ratio or startup latency.
Automatically provide
continuous streaming?
Conflicting
9
A Case of Conflicting Performance
Interests
Evaluated on a typical workload: WEB
10
Problems in Streaming Proxy Design
• Independently pursuing highest byte hit ratio or
minimizing startup latency
– But they conflict with each other!
• Lack of serious considerations to guarantee the
continuous streaming
– But this is the most important QoS concern to viewers!
– Providing continuous streaming also conflicts with
improving byte hit ratio!
11
Our Design Model
• An efficient design model should guarantee
continuous delivery subject to low startup
latency and high byte hit ratio.
• Challenges
– How to reconcile guaranteeing continuous streaming
delivery with improving byte hit ratio?
– How to balance reducing startup latency with
improving byte hit ratio?
12
Algorithms Design in Hyper Proxy
(intellectual challenges)
• Active Prefetching Technique
– Prefetching uncached segments to ensure continuous delivery.
• Lazy Segmentation Method
– Segmenting objects adapting the access patterns.
• Priority-based Admission Policy
– Continuous streaming > small startup latency > byte hit ratio
• Differentiated Replacement Policy
– Keeping appropriate (not necessarily the most popular) segments
in the proxy
13
Our Contributions
• Addressed two pairs of conflicting interests
• Designed Hyper-Proxy following our efficient
design model
• Implemented Hyper-Proxy and deployed it at
Hewlett-Packard Company
• It is the first deployed segment-based streaming
proxy
14
Outline
• Addressing two pairs of conflicting interests
– Provide Continuous Streaming via Active Prefetching
– Minimize Startup Latency via Performance Trade-off
• Hyper-Proxy Design
• Conclusion
15
Proxy Jitter: Minimization Objective
Proxy Jitter: the delay of
fetching uncached segments
?
?
?
N
Client
Streaming
Proxy
Bs: encoding rate
!
Y
Internet
Bt: average bandwidth
Prefetching is done in a unicast channel
Streaming
Server
16
Window-based Prefetching
• Principle: to fetch the next segment after a
client starts to access the current segment
Streaming speed = encoding rate, Bs
Prefetching speed = average network bandwidth, Bt
Prefetching
L
Bs ≤ 2Bt
Streaming…
17
Active Prefetching
• Principle:prefetching as early as when a client starts to
access the object.
• Additional notation
– Ns: the number of cached segments of an object
• Uniformly Segmented Object
– Ns < Bs/Bt – 1 (converted to number of segments)
 Some segments can not be prefetched in time
– Ns ≥ Bs/Bt – 1
Proxy Jitter!
 All segments can be prefetched in time
18
Active Prefetching?
• Is it possible to guarantee an always-in-time
prefetching?
• Yes! To increase Ns!
– i.e., to cache more segments of those objects who
have difficulty to afford in-time prefetching
• How many segments of an object should be
cached so that active prefetching always works?
19
Active Prefetching – A Step Further
• Minimum cached length of an object:
– To ensure the prefetching of its uncached segments
to be in time. (Lobj: length of segment, L1: Length of 1st
cached segment).
Uniformly Segmented Object Exponentially Segmented Object
Bt


(
1

)

L
obj


Bs


L1




Bt


(1  )  Lobj 

Bs
log
2
(
)  1

L1




20
Active Prefetching – In Practice
Conflicting
Byte Hit Ratio slightly decreases; Proxy Jitter is totally eliminated!
Cashed length by popularity
High
Medium
Low
Low popularity document
Same required minimum length
needs additional
for continuous streaming. But
caching space,
Insufficient for popular files.
which will be provided by others.
High
Medium
Low
Low
Not all popular segments need to be cached, but appropriate ones!
21
Outline
• Addressing two pairs of conflicting interests
– Guarantee Continuous Streaming via Active Prefetching
- Tradeoffs between Startup Latency and byte hit ratios
• Hyper-Proxy Design
• Conclusion and Future work
22
Modeling Assumptions
• Assumption 1: there is no premature termination.
• Assumption 2: objects are sequentially
accessed.
• Assumption 3: Zipf-like distribution of object
popularities (USITS’01, NOSSDAV’02)
• Assumption 4: Poisson distribution of request
arrival intervals (USITS’01, NOSSDAV’02)
23
Modeling Results
• We get the relative changing rate of byte hit ratio
to delayed start request ratio mathematically.
• We found byte hit ratio decreases much slower
than delayed startup request ratio.
• We can use a small reduction of byte hit ratio to
trade for a large decrease of delayed startup
request ratio.
24
Outline
• Addressing two pairs of conflicting interests
– Guarantee Continuous Streaming via Active Prefetching
– Minimize Startup Latency via Performance Trade-off
• Hyper-Proxy Design
• Conclusion and Future work
25
Basic Hyper-Proxy Operations
(Lazy Segmentation based)
CDL = Size
its Prefetching Length<= CDL < Size
CDL < its Prefetching Length
CDL: cached data length
Size: Full Size of a Media Object
Replacement:
Admission:
Lazy
Admission:
Segmentation:
fully
Replacement:
kept
based
admit
partial
on
the
triggered
prefetching
segments,
initially
find
victim.
byaccessed
asegment
shift
calculation.
replacement.
to objects.
next
list.
More
Prefetching:
replacements:
for
shortage
adding
of
partial
segments,
list
shift.
sets.
27
Workload Summary
#req
#obj
Size
λ

WEB
15188
400
51
4
PART
15188
400
51
REAL
9000
403
20
Workload
Length
Encoding
(Bs)
Transfer
(Bt)
Duration
0.47
2-120
28~256
[0.5, 2]
1
4
0.47
2-120
28~256
[0.5, 2]
1
-
-
-
-
-
10
λ : of Poisson, in second
 : of Zipf-like, the skew factor
Length : of objects, in minute
Duration : of simulation time, in days
Size: of total objects, in GB
Encoding, Transfer: in Kbps
32
Hyper-Proxy Performance
(comparing with Lazy-Segmentation, and LazySegmentation with prefetching)
33
Hyper-Proxy Performance
(comparing with Lazy-Segmentation, and LazySegmentation with prefetching)
34
Hyper-Proxy Performance
(comparing with Lazy-Segmentation, and LazySegmentation with prefetching)
35
Hyper-Proxy – Protocol
HTTP; No Changes
RTP/RTSP
Hyper-Proxy
Servers
Intermediaries
Clients
Existing Web servers can provide “real” streaming service!
39
Hyper-Proxy – Architecture
Perform streaming
ghost.mp4,
Meta
please
data
RTP/RTSP
Client
Interfacing two
engines
Hyper-Proxy
Streaming Engine
Local Manager&Scheduler
Disk
st segment,
Next
segment,
please
1
Metaplease
data
HTTP
Internet
Segmentation
Enabled
Cache Engine
Fast data
Media
Synchronization
Mediadata
data path
Object segmentation,
Segment admission, replacement
40
Hyper-Proxy Evaluation
in Global Intent Environment
• US West coast -- HP Labs, Palo Alto, CA
– Server (apache 2.0.45): Pentium III 1 GHz, SuSE Linux 8.0
– Hyper-Proxy: HP x4000 workstation 2 GHz
– Many Clients: Pentium III 1GHz, SuSE Linux 8.0
• Asia – NTT, Takaido, Japan
– Server (apache 2.0.45): Pentium III 1 GHz, SuSE Linux 8.0
– Hyper-Proxy: HP x4000 workstation 2 GHz
– Many Clients: Pentium III 1 GHz, SuSE Linux 8.0
• US East Cost – William and Mary, Williamsburg, VA
– Hyper-Proxy: HP x4000 workstation 2 GHz
– Many clients: diverse types.
42
Conclusions
• Hyper-Proxy design and implementation
– Provide an efficient design model
– Enable streaming delivery service on Web servers
– Cause little playback jitter
– Produce a small startup latency
48
Current Status and Impacts
• Impact of Hyper-Proxy at HP (product impact)
– Deployed at Printing Division for world-wide training
– Deployed at Telecommunication Division
– Used by Imaging and Signal Processing Team in HP
Labs
• Potential Industry Impact
– Enterprise solution after the trial stage
– Education solution for remote education (currently
several universities are using the prototypes).
– Two patents pending.
49
Project 1: Mobility
Device
Hand-Off
Cooperation
Servers
Intermediaries
Clients
50
Project 2: P2P Streaming
Sharing contents
and streaming
resources
Servers
Intermediaries
Clients
51
Project 3: Diversity of Devices
Different screen
sizes, color
depths, et. al.
transcoding
Servers
Intermediaries
Clients
52
Project 4: Live Streaming
The Oscar
Annual Academy
Servers
Awards
cannot cache
Intermediaries
Clients
53
Acknowledgement
• NSF and HP Labs for research grants
• Collaborators: Bo Shen, Susie Wee, Sumit
Roy, Yong Yan, Sujoy Basu, Dan (Waitian) Tan, Ankcorn John, Mitch Trott,
Zhichen Xu ; Zhen Xiao
• Members and alumni of HPCS Lab@WM
54
Related Publications (academic impact)
1. S. Chen, B. Shen, Y. Yan, S. Basu, and X. Zhang. SRB: Shared Running
Buffers in Proxy to Exploit Memory Locality of Multiple Streaming Sessions.
IEEE ICDCS 2004.
2. S. Chen, B. Shen, S. Wee, and X. Zhang. Designs of High Quality
Streaming Proxy Systems. IEEE INFOCOM 2004.
3. S. Chen, B. Shen, S. Wee, and X. Zhang. Investigating Performance Insights
of Segment-based Proxy Caching of Streaming Media Strategies.
ACM/SPIE MMCN 2004.
4. S. Chen, B. Shen, S. Wee, and X. Zhang. Streaming Flow Analyses for
Prefetching in Segment-based Proxy Caching Strategies to Improve Media
Delivery Quality. WCW 2003.
5. S. Chen, B. Shen, S. Wee, and X. Zhang. Adaptive and Lazy Segmentation
Based Proxy Caching for Streaming Media Delivery. ACM NOSSDVA 2003.
Songqing Chen received his Ph.D., has been placed
in a research academic institution (George Mason U.)
55