Transcript Πολυμεσικό Υλικό στο Internet

Πολυμεσικό Υλικό στο Internet:
Συγχρονισμός, Επεξεργασία και
Διακίνηση
Διακίνηση Video με χρήση του HTTP
8/1/2015
Β. Μάγκλαρης <[email protected]>
Μ. Γραμματικού <[email protected]>
Δ. Καλογεράς <[email protected]>
www.netmode.ntua.gr
Outline
•
•
•
•
Streaming over Internet
1st Generation Progressive
2nd Generation
UDP based Real-Time Protocols (RTP, RTCP,
RTSP)
• 3rd Generation TCP based
– Proprietary – Adobe
– Adaptive HTTP based
Challenges
 TCP/UDP/IP , best-effort, with no guarantees on
expectation or variance of packet delay
 Streaming applications delay of 5 to 10 seconds
is typical and has been acceptable, but
performance deteriorate if links are congested
 Real-Time Interactive requirements on delay
and its jitter have been satisfied by overprovisioning (providing plenty of bandwidth),
what will happen when the load increases?...
On-Demand Streaming
 Important and growing application due to reduction
of storage costs, increase in high speed net access
from homes, enhancements to caching
 Interactive control by user
(but often with long response time)
 Ubiquitous on the web:
–
–
–
–
YouTube, Netflix, Vimeo
Television networks, Hollywood, etc.
Most local radio & TV stations
Virtually everywhere on websites
4
First Generation: HTTP Download
 A simple architecture is to have the Browser
request the object(s) and after their reception pass
them to the player
for display
– No pipelining
Helper Application – i.e. media player
 Displays content, which is typically requested via a
Web browser; typical functions:
– Decompression
– Jitter removal
– Error correction: use redundant packets to be used for
reconstruction of original stream
– GUI for user control
 Examples:
–
–
–
–
–
RealPlayer
Adobe Flash Player
Windows Media Player
QuickTime
DivX Web Player
First Gen: HTTP Progressive Download
(2)
 Alternative: set up connection between server and
player; player takes over
 Web browser requests and receives a Meta File
(a file describing the object) instead of receiving the
file itself;
 Browser launches the appropriate Player and passes it
the Meta File;
 Player sets up a TCP connection with Web Server and
downloads or streams the file
Meta file requests
8
HTTP Progressive Download
 With helper application doing the download, playback can start
immediately...
 Or after sufficient bytes are buffered
 Sender sends at maximum possible rate under TCP; retransmit
when error is encountered; Player uses a much larger buffer to
smooth delivery rate of TCP
HTTP Progressive Download
Buffering Continuous Media
 Jitter = variation from ideal timing
 Media delivery must have very low jitter
– Video frames every 30ms or so
– Audio: ultimately samples need < 128ms jitter
 But network packets have much more jitter
that that!
 Solution: buffers
– Fill them with best effort
– Drain them via low-latency, local access
Max Buffer Duration
"Bad": Buffer
overrflows
Max Buffer Size
File Position
Streaming, Buffers and Timing
Buffer
Duration
"Good" Region:
smooth playback
Buffer
Size
= allowable jitter
"Bad": Buffer
underflows and
playback stops
Buffer almost empty
Time
The myth of SBR
• Any single bitrate you choose for a video is by
definition the wrong video for 99% of users.
Average Throughput for your Site Visitors
100
90
These users will rebuffer
80
70
These have lower quality than they could
sustain
60
50
40
30
20
10
0
100 300 500 700 900 1100 1300 1500 1700 1900 2100 2300 2500 2700 2900 3100 3300 3500 3700 3900 4100 4300 4500 4700 4900 5100
Drawbacks of HTTP Progressive
Download (2)
 HTTP connection keeps data flowing as fast as possible
to user's local buffer
– May download lots of extra data if you do not watch the
video
– TCP file transfer can use more bandwidth than necessary
 Mismatch between whole file transfer and
stop/start/seek playback controls.
– However: use file range requests to seek to video position
 Cannot change video quality (bit rate) to adapt to
network congestion
2nd Generation:
Real-Time Streaming
 This gets us around HTTP, allows a choice of UDP vs. TCP and the
application layer protocol can be better tailored to Streaming; many
enhancements options are possible
Legacy Video Streaming Approaches
• RTSP and RTP
Example: Real Time Streaming
Protocol (RTSP)
• For user to control display: rewind, fast forward,
pause, resume, etc…
• Out-of-band protocol (uses two connections, one
for control messages (Port 554) and one for media
stream)
• RFC 2326 permits use of either TCP or UDP for the
control messages connection, sometimes called
the RTSP Channel
• As before, meta file is communicated to web
browser which then launches the Player; Player
sets up an RTSP connection for control messages in
addition to the connection for the streaming media
RTSP Operation
18
RTSP Media Stream
 Stateful Server keeps track of client's state
 Client issues Play, Pause, ..., Close
 Steady stream of packets
– UDP - lower latency
– TCP - may get through more firewalls, reliable
Credit: some content adapted from Alex Zambelli
19
RTSP Exchange Example
C: SETUP rtsp://audio.example.com/xena/audio RTSP/1.0
Transport: rtp/udp; compression; port=3056; mode=PLAY
S: RTSP/1.0 200 1 OK
Session 4231
C: PLAY rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0
Session: 4231
Range: npt=0 (npt = normal play time)
C: PAUSE rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0
Session: 4231
Range: npt=37
C: TEARDOWN rtsp://audio.example.com/xena/audio.en/lofi RTSP/1.0
Session: 4231
S: 200 3 OK
20
Example 2: RTMP - Real-Time
Messaging Protocol




Proprietary Adobe protocol
Runs over TCP
Manages audio, video, and other
Multiplex multiple streams over TCP
connection
21
Flash Streaming Server
 Flash Streaming Server communicates with
its clients using the Adobe patented RTMP
over TCP, which manages a two-way
connection, allowing the server to send
and receive,video, audio and data between
client and server
 Typically used with FLV encoded files
Variations of RTMP
 RTMP



Standard,unencrypted RTMP.
The default port is 1935.
RTMPT- RTMP “tunneled”
over HTTP
The default port is 80
RTMPS- RTMP sent over an SSL. SSL enables secure
TCP/IP connections. The default port is 443
RTMPE- Enhanced and encrypted version of RTMP.
RTMPTE- Encrypts the communication channel,
tunneling over HTTP
Drawbacks of RTSP, RTMP
 Web downloads are typically cheaper than
streaming services offered by CDNs and
hosting providers
 Streaming often blocked by routers
 UDP itself often blocked by firewalls
 HTTP delivery can use ordinary proxies and
caches
 Conclusion: rather than adapt Internet to
streaming, adapt media delivery to the
Internet
24
Bandwidth is never constant


Even preselecting bitrates will lead to rebuffering problems as bandwidth fluctuates
over the duration of the vide, especially for long form content
Ideally we would like to switch bitrates constantly to always give the user the highest
quality they can sustain at any point in time.
Buffering


Bandwidth is never constant
Even preselecting bitrates will lead to rebuffering problems as bandwidth fluctuates
over the duration of the vide, especially for long form content
Ideally we would like to switch bitrates constantly to always give the user the highest
quality they can sustain at any point in time.
Adaptive delivery
1500kbps
1000kbps
Buffering
500kbps
3rd Generation: HTTP Streaming
 Other terms for similar concepts: Adaptive Streaming, Smooth
Streaming, HTTP Chunking
 Client-centric architecture with stateful client and stateless
server
– Standard server: Web servers
– Standard Protocol: HTTP
– Session state and logic maintained at server




Video is broken into multiple chunks
Chunks begin with keyframe so independent of other chunks
A series of HTTP progressive downloads of chunks
Playing chunks in sequence gives seamless video
Adaptive Bit Rate with HTTP
Streaming
 Encode video at different levels of
quality/bandwidth
 Client can adapt by requesting different sized
chunks
 Chunks of different bit rates must be
synchronized
– All encodings have the same chunk boundaries and all
chunks start with keyframes, so you can make smooth
splices to chunks of higher or lower bit rates
How does segmentation work?
3
1 Incoming video
500 kbps
1000 kbps
2
is split by an encoder
2000 kbps
into multiple short
blocks. Each block holds
the same section of
video, encoded at a
different size and bitrate.
How does adaptive delivery work?
The segmented video is stored on a
server, along with a text file which
describes the names of each
segment. This text file is called a
manifest.
A player downloads the manifest
and then begins requesting
individual segments of video.
It makes its choice based on
bandwidth conditions, grabbing the
best quality it can at the time.
SERVER
CLOUD
PLAYER
Media Source Extension – the future
of browser video
 A W3C standard https://dvcs.w3.org/hg/html-media/rawfile/tip/media-source/media-source.html
 Supported today by Chrome, IE11 and Opera
 MSE allows JavaScript to
- fetch and parse a manifest
- retrieve video, audio and text segments using XHR
- feed them to the <video> tag to be decoded and
rendered.
MSE gives developers incredible control over playback,
buffering and adaption logic of in-page video.
Pareto principle in video popularity
• Also known as the “80-20” (or 90-10) rule
[http://en.wikipedia.org/wiki/Pareto_principle]
• Associated with highly skewed distributions
• Caused by positive-feedback effects (“the rich get richer”)
• Does non-UGC content (say movies from NetFlix) follow a similar
pattern?
• A practical implication of this skewness: caching
applies!
Caching works…
• Let’s build CDNs
• Take advantage of locality and relativity
• A practical implication of this skewness: caching
works!
• Akamai
• You tube
• Netflix based on Amazon infrastracture
Adaptive HTTP Streaming System
(Protocol)
 Server
– Can be standard web
server
– Media segment can be
prepared in-line or offline
 Client
– Sends series of HTTP GET
segment requests and
receives segments
– Performs rate adaptation
before sending a new GET
segment request
Advantages of HTTP Streaming
• Easy to deploy: it's just HTTP, work with existing
caches/proxies/CDN/Firewall
• Fast startup by downloading lowest
quality/smallest chunk
• Bitrate switching is seamless
• Many small files
– Small with respect to the movie size
– Large with respect to TCP
• 5-10 seconds of 1Mbps – 3Mbps  0.5MB – 4MB
per chunk
35
Which adaptive streaming formats
exist today?
 HTTP Live Streaming (HLS)
- Controlled by Apple
- Known as HLS, it is supported well by iOS, Safari and half-heartedly by
Android.
 HTTP Dynamic Streaming (HDS)
- Controlled by Adobe
- Known as HDS, it is played back only by Flash clients with custom apps.
 Smooth Streaming (SMOOTH)
- Controlled by Microsoft
- Played back via Silverlight clients, Xbox,
 MPEG-DASH (DASH)
- An ISO standard
- Playback via MSE in browsers, native apps on iOS, Android and WInOS.
Simple HLS example
<video id="player" controls poster="sintel.jpg"
preload="none" style="width:480px;height:204px">
<source src="http://multiplatformf.akamaihd.net/i/multi/april11/sintel/sintelhd_,512x288_450_b,640x360_700_b,768x432_1000_b,.mp4.
csmil/master.m3u8" type='application/x-mpegURL' />
<source src="http://akamaicorppmd.edgesuite.net/flashvod/video/mp4/sintel.mp4"
type='video/mp4'/>
<source src="http://akamaicorppmd.edgesuite.net/flashvod/video/mp4/sintel.webm"
type='video/webm' />
</video>
Example of HLS Meta Data
Other Video Streaming
Approaches
• RTMP
Adaptive video streaming
over HTTP
 Several players support it
 Several commercial content providers use it




Microsoft Smooth Streaming Player
Netflix Player
Adobe OSMF/Zeri Player
Apple Player
HTTP Adaptive Video Streaming
HTTP Adaptive Video Streaming:
Manifest File and Fragments
Smooth Streaming Player
Smooth Streaming Player
• Sample HTTP Request:
– GET
/mediadl/iisnet/smoothmedia/Experience/BigBuckBunny7
20p.ism/QualityLevels(2040000)/Fragments(video=400000
000) HTTP/1.1
Adobe OSMF/Zeri Player
Adobe OSMF Player
• Sample HTTP Request:
– GET /content/inouteditmbr/inoutedit\_h264\_3000Seg1-Frag5 HTTP/1.1
What Makes Netflix Interesting?
• Commercial, feature-length movies and TV shows
 and not free; subscription-based
• Nonetheless, Netflix is huge!


25 million subscribers and ~20,000 titles (and growing)
consumes 30% of peak-time downstream bandwidth in North America
• A prime example of cloud-sourced architecture
 Maintains only a small “in-house” facility for key functions


Majority of functions are sourced to Amazon cloud (EC2/S3)



e.g., subscriber management (account creation, payment, …)
user authentication, video search, video storage, …
DNS service is sourced to UltraDNS
Leverage multiple CDNs (content-distribution networks) for video
delivery

Akamai, Level 3 and Limelight
Netflix Architecture
• Netflix has its own “data center” for certain crucial operations (e.g., user
registration, billing, …)
• Most web-based user-video interaction, computation/storage operations
are cloud-sourced to Amazon AWS
• Video delivery is out/cloud-sourced to 3 CDNs
• Users need to use MS Silverlight player for video streaming
Netflix Videos and Video Chunks
• Netflix uses a numeric ID to identify each movie
– IDs are variable length (6-8 digits): 213530, 1001192, 70221086
– video IDs do not seem to be evenly distributed in the ID space
– these video IDs are not used in playback operations
• Each movie is encoded in multiple quality levels, each is identified
by a numeric ID (9 digits)
– various numeric IDs associated with the same movie
appear to have no obvious relations
49
Netflix Player
Netflix Player
 Sample HTTP Request:
 GET /sa2/946/1876632946.wmv/range/22120592252058?token=1283923056\_d6f6112068075f1fb6
0cc48eab59ea55\&random=1799513140 HTTP/1.1
Netflix Videos and Video Chunks
• Videos are divided in “chunks” (of roughly 4 secs), specified using (byte)
“range/xxx-xxx?” in the URL path:
Limelight:
http://netflix-094.vo.llnwd.net/s/stor3/384/534975384.ismv/range/057689?p=58&e=1311456547&h=2caca6fb4cc2c522e657006cf69d4ace
Akamai:
http://netflix094.as.nflximg.com.edgesuite.net/sa53/384/534975384.ismv/
range/057689?token=1311456547_411862e41a33dc93ee71e2e3b3fd8534
Level3:
http://nflx.i.ad483241.x.lcdn.nflximg.com/384/534975384.ismv/range/057689?etime=20110723212907&movieHash=094&encoded=06847414df
0656e697cbd
• Netflix uses a version of (MPEG-)DASH for video streaming
52
Terms and Definitions of Adaptive
HTTP Streaming
 Need
– Media Presentation Description (MDP) which
provides metadata
• For requesting (GET request) media segments
• For rate adaptation purpose
– Segment which may include media data or
metadata to decode
 Use DASH an example in the few slides
Example: DASH Client
Thomas Stockhammer, Qualcomm, “DASH – Design Principles and Standards ,
Presentation at MMSys 2011
Meta Data
 DASH uses MPD (Media Presentation Descriptor) and
Index Information as metadata for DASH Access Client
 Initialization and Media Segments for Media Engine
– Reuse of existing container format
Source: Stockhammer, Qualcomm, “DASH – Design Principles and
Standards , Presentation at MMSys 2011
Media Presentation Data Model
MDP - description of accessible segments and corresponding timing
Source: Stockhammer, Qualcomm, “DASH – Design Principles and
Standards , Presentation at MMSys 2011