Transcript MASS 2007
School of Computing Science Simon Fraser University
CMPT 820: Multimedia Systems Streaming Media Over Packet Networks
Hamed S. Neshat Mohamed Hefeeda 1
Outline
Introduction
Source coding, channel coding, sending, transmission and arrival rates
Source coding rate control
-
Control theoretical model
-
Control objective
-
Target schedule design
-
Controller design
-
Effect of frame rate
-
Controller interpretation Mohamed Hefeeda 2
Introduction (1)
Packet networks are
-
Neither isochronous nor lossless
-
Are shared by multiple communication
Networks have time-varying behaviour
Regardless of network conditions, with an average bandwidth, users expect
-
Low start-up delay
-
Continues play back
-
High quality, as much as possible
using buffer Mohamed Hefeeda 3
Introduction (2)
Buffering serves several distinct but simultaneous purposes:
-
It allows the media to be coded with a variable instantaneous bit rate
-
It allows the client to compensate for short term variation in packet transmission delay(i.e. jitter)
-
It allows the client to continue playing back the media during lapses in network bandwidth
-
It gives t he client time to perform pocket loss recovery if needed
This section is about how to keep the client buffer sufficiently full, on average, to serve all purposes. 4 Mohamed Hefeeda
Source coding, channel coding, sending, transmission and arrival rates
-
R C
: source coding rate
R S
: sending rate - rate at which data injected into transport layer
•
Measured in bits/s of client time
-
R X
: transmission rate - rate which data injected into network layer (TCP or UDP)
-
R X
- R
S
= error control overhead
R S
/ R
X
•
= channel coding rate is determined by the retransmission mechanism in TCP or by any other error control layer
-
R a
: arrival rate
•
the sending rate R
rate R a a S is over the long run equal to the arrival
Mohamed Hefeeda 5
Source coding, channel coding, sending, transmission and arrival rates
Many streaming media practitioners assume:
-
except during the initial buffering period, during which v = 0 and R
a
> 0
Unlocking R
c
and R
a
cam lead to important advantages
-
makes possible continuous control of the number of seconds of content in the client buffer (client buffer duration)
-
makes efficient streaming possible with arbitrary transports, such as TCP, DCCP, or other proprietary transport protocols.
6 Mohamed Hefeeda
Source coding, channel coding, sending, transmission and arrival rates
client buffer duration
-
That is, if R
a > R c
ν then the buffer duration increases; otherwise it decreases
-
in the long run, ν cannot remain very much different from the user’s preferred playback speed
-
arrival rate R
a
is essentially limited by the network capacity
R c
is the only appropriate way to maintain the buffer duration
source coding rate control
-
Adjusting the source coding rate in the face of time-varying network conditions 7 Mohamed Hefeeda
Source coding, channel coding, sending, transmission and arrival rates
if compatibility with TCP is not required, then it is possible for the source coding rate control and the error control modules illustrated in Figure a to be combined into a single module, as illustrated in Figure b.
-
This is a joint source-channel coding problem sometimes known as the scheduling problem and is treated in detail in Chapter 10, using the RaDiO framework.
8 Mohamed Hefeeda
Source coding rate control
The goal is to control the client buffer duration to a target by choosing R
c as a function of
-
R a
-
the client buffer duration and its history
We assume there are multiple media streams with various bit rate available at the server Mohamed Hefeeda 9
Control Theoretic Model
-
frame n arrives at the client b(n)/Ra seconds after frame n − 1
-
t a
(n) :arrival time of frame n at the client.
t b
(n) : upper bound, the time that frame n is guaranteed to arrive.
t d
(n): the time at which frame n is scheduled to be instantaneously decoded and played Mohamed Hefeeda 10
Figure 14.5
Mohamed Hefeeda 11
Control Theoretic Model
In reality the arrival rate is not constant
-
frame would take longer to be transmitted
-
frame arrival times would become increasingly delayed
R a > R c ν
Adjusting buffer duration by selecting different source coding rate R
c . How?!
let t
a
(n − 1) and t
a
(n) be the arrival times of frames n and n−1, respectively. instantaneous arrival rate at frame n is : 12 Mohamed Hefeeda
Control Theoretic Model
-
the average arrival rate at frame n will be estimated by a moving average ˜R
a
(n) of previous values of R
a
(n), as detailed in the Appendix.
v(n) :an error term that captures the effect of using ˜R
a R a (n).
(n) instead of
-
How does the source coding rate R
c
fit into all this?
Mohamed Hefeeda 13
Control Theoretic Model
-
From Figure 14.5, it is clear that the decoder buffer fullness
F d = B −F e
(n) can also be expressed as : Mohamed Hefeeda 14
Control Theoretic Model
Mohamed Hefeeda 15
Control Theoretic Model
-
Finally, we can find this equation:
-
With (14.34), the client can compute t
b (n−1) .
Then using (14.36), the client can control the coding rate R
c
(n) so that t
b
(n) reaches a desired value, assuming the frame rate and arrival rate remain roughly constant.
Mohamed Hefeeda 16
Control Theoretic Model
Mohamed Hefeeda 17
Control Objective
-
uninterrupted playback can be achieved by regulating the coding rate so that the client buffer does not underflow.
-
target schedule: a margin of safety that increases over time
-
By regulating the coding rate, we attempt to control the buffer tube upper bound so that it tracks the target schedule.
-
If the buffer tube upper bound is close to the target schedule, then the arrival times of all frames will certainly be earlier than their playback deadlines and thus uninterrupted playback will be ensured 18 Mohamed Hefeeda
Control Objective
-
We try to
•
control the upper bound to the target schedule
•
minimize quality variations due to large or frequent changes to the coding rate
the first term penalizes the deviation of the buffer tube upper bound from the target schedule the second term penalizes the relative coding rate difference between successive frames.
N
is the control window size and σ is a Lagrange multiplier or weighting parameter to balance the two terms.
Mohamed Hefeeda 19
Target Schedule Design
-
The gap between the playback deadline and the target schedule is the desired minimum client buffer duration.
-
Small gap at the beginning : allows a small start-up delay Larger gap over time: increases the client’s ability to counter jitter, delays, and throughput changes.
Mohamed Hefeeda 20
Controller Design
-
We wish to control the upper bound by adjusting the source coding rate:
-
As each frame arrives at the client, a feedback loop can send a message to the server to adjust the source coding rate.
-
by the time frame n arrives completely at the client, frame n+1 has already started streaming from the server
at time t
a
(n), frame n + 2 is the earliest frame for which the controller can determine the coding rate 21 Mohamed Hefeeda
Controller Design
-
By defining error rate:
-
We obtain:
-
So,
-
We next define the control input and disturbance: Mohamed Hefeeda 22
Controller Design
-
Then (14.50) can be rewritten
-
Therefore, defining the state vector:
-
Thus, Mohamed Hefeeda 23
Controller Design
-
Assuming,
•
the disturbance d(n) is a pure white noise
•
we can measure all components of e(n) without using an estimator(perfect state measurement )
Then, the disturbance d(n) does not affect the controller design.
-
Thus we can use a linear controller represented by: where G is a vector feedback gain Mohamed Hefeeda 24
Controller Design
-
By the time frame n is completely received, all elements of e(n) are available at the client and u(n) can thus be computed. The ideal coding rate for frame n+2 can then be computed as Mohamed Hefeeda 25
Effect of Frame Rate
-
Until now, we assumed that the frame rate is constant, but this assumption is not always true:
•
usually video and audio are streamed together, and their merged coding schedule may have no fixed frame rate
•
Even if there is a fixed frame rate f we may wish to operate the controller at a rate lower than f to reduce the feedback rate
-
To address these issues, in practice we use the notion of a virtual
frame rate (f)
-
We partition media time into intervals of size 1/f ;
-
and we model all of the frames arriving within each interval as a virtual frame whose decoding and playback deadline is the end of the interval.
Mohamed Hefeeda 26
Effect of Frame Rate
Advantages of using virtual frame rate:
-
it allows us to design off line a universal feedback gain, which is independent of the actual frame rate of the stream or streams.
-
it allows us to reduce the rate of feedback from the client to the server.
-
since the interval between virtual frames is typically safely larger than a round trip time (RTT), a one-frame delay in the error space model is sufficient to model the feedback delay.
In the sequel we therefore use a virtual frame rate f = 1 fps, and we refer to this simply as the frame rate. With f = 1 and σ = 50 (chosen empirically for good damping), we can solve for G= [0.6307,−0.5225, 0.5225]. Mohamed Hefeeda 27
Controller Interpretation
we are now able to give an intuitive explanation of the source coding rate control (14.57): Mohamed Hefeeda 28
Controller Interpretation
-
Focusing on the first term (14.63), it can be seen that the source coding rate Rc tends to decrease if the current error e(n) = tb(n)−tT (n) is positive, and it tends to increase if e(n) is negative
-
from the second term (14.64), it can be seen that the source coding rate tends to decrease if the current error e(n) is numerically greater than the previous error e(n − 1), whether e(n) is positive or negative
-
Finally, from the third term (14.65), it can be seen that the source coding rate tends to decrease if it had previously increased 29 Mohamed Hefeeda
Controller Interpretation
Mohamed Hefeeda 30
Conclusion
Packet networks are lossy and shared
Users expect good quality, given a an average bandwidth
For solving network problems, we have buffer at client
Good streaming depends on proper buffer duration
We can control buffer duration by regulating source coding rate 31 Mohamed Hefeeda
Mohamed Hefeeda 32
Mohamed Hefeeda 33