Mohamed Hefeeda 1 School of Computing Science Simon Fraser University CMPT 820: Multimedia Systems Streaming Media Over Packet Networks Hamed S. Neshat

Mohamed Hefeeda Outline  Introduction  Source coding, channel coding, sending, transmission and arrival rates  Source coding rate control -Control theoretical model -Control objective -Target schedule design -Controller design -Effect of frame rate -Controller interpretation 2

Mohamed Hefeeda Introduction (1)  Packet networks are -Neither isochronous nor lossless -Are shared by multiple communication  Networks have time-varying behaviour  Regardless of network conditions, with an average bandwidth, users expect -Low start-up delay -Continues play back -High quality, as much as possible  using buffer 3

Mohamed Hefeeda Introduction (2)  Buffering serves several distinct but simultaneous purposes: -It allows the media to be coded with a variable instantaneous bit rate -It allows the client to compensate for short term variation in packet transmission delay(i.e. jitter) -It allows the client to continue playing back the media during lapses in network bandwidth -It gives t he client time to perform pocket loss recovery if needed  This section is about how to keep the client buffer sufficiently full, on average, to serve all purposes. 4

Mohamed Hefeeda -R C : source coding rate -R S : sending rate - rate at which data injected into transport layer Measured in bits/s of client time -R X : transmission rate - rate which data injected into network layer (TCP or UDP) -R X - R S = error control overhead -R S / R X = channel coding rate is determined by the retransmission mechanism in TCP or by any other error control layer -R a : arrival rate the sending rate R S is over the long run equal to the arrival rate R a a 5 Source coding, channel coding, sending, transmission and arrival rates

Mohamed Hefeeda Source coding, channel coding, sending, transmission and arrival rates 6  Many streaming media practitioners assume: -except during the initial buffering period, during which v = 0 and R a > 0  Unlocking R c and R a cam lead to important advantages -makes possible continuous control of the number of seconds of content in the client buffer (client buffer duration) -makes efficient streaming possible with arbitrary transports, such as TCP, DCCP, or other proprietary transport protocols.

Mohamed Hefeeda  client buffer duration -That is, if R a > R c ν then the buffer duration increases; otherwise it decreases -in the long run, ν cannot remain very much different from the user’s preferred playback speed -arrival rate R a is essentially limited by the network capacity -R c is the only appropriate way to maintain the buffer duration  source coding rate control -Adjusting the source coding rate in the face of time-varying network conditions 7 Source coding, channel coding, sending, transmission and arrival rates

Mohamed Hefeeda 8  if compatibility with TCP is not required, then it is possible for the source coding rate control and the error control modules illustrated in Figure a to be combined into a single module, as illustrated in Figure b. -This is a joint source-channel coding problem sometimes known as the scheduling problem and is treated in detail in Chapter 10, using the RaDiO framework. Source coding, channel coding, sending, transmission and arrival rates

Mohamed Hefeeda Source coding rate control  The goal is to control the client buffer duration to a target by choosing R c as a function of -R a -the client buffer duration and its history  We assume there are multiple media streams with various bit rate available at the server 9

Mohamed Hefeeda Control Theoretic Model -frame n arrives at the client b(n)/Ra seconds after frame n − 1 -t a (n) :arrival time of frame n at the client. -t b (n) : upper bound, the time that frame n is guaranteed to arrive. -t d (n): the time at which frame n is scheduled to be instantaneously decoded and played 10

Mohamed Hefeeda Figure

Mohamed Hefeeda Control Theoretic Model  In reality the arrival rate is not constant -frame would take longer to be transmitted -frame arrival times would become increasingly delayed  Adjusting buffer duration by selecting different source coding rate R c. How?! - let t a (n − 1) and t a (n) be the arrival times of frames n and n−1, respectively. instantaneous arrival rate at frame n is : 12 R a > R c ν

Mohamed Hefeeda Control Theoretic Model -the average arrival rate at frame n will be estimated by a moving average ˜R a (n) of previous values of R a (n), as detailed in the Appendix. v(n) :an error term that captures the effect of using ˜R a (n) instead of R a (n). -How does the source coding rate R c fit into all this? 13

Mohamed Hefeeda Control Theoretic Model -From Figure 14.5, it is clear that the decoder buffer fullness F d = B −F e (n) can also be expressed as : 14

Mohamed Hefeeda Control Theoretic Model 15

Mohamed Hefeeda Control Theoretic Model -Finally, we can find this equation: -With (14.34), the client can compute t b (n−1). -Then using (14.36), the client can control the coding rate R c (n) so that t b (n) reaches a desired value, assuming the frame rate and arrival rate remain roughly constant. 16

Mohamed Hefeeda Control Theoretic Model 17

Mohamed Hefeeda Control Objective -uninterrupted playback can be achieved by regulating the coding rate so that the client buffer does not underflow. -target schedule: a margin of safety that increases over time -By regulating the coding rate, we attempt to control the buffer tube upper bound so that it tracks the target schedule. -If the buffer tube upper bound is close to the target schedule, then the arrival times of all frames will certainly be earlier than their playback deadlines and thus uninterrupted playback will be ensured 18

Mohamed Hefeeda Control Objective -We try to control the upper bound to the target schedule minimize quality variations due to large or frequent changes to the coding rate  the first term penalizes the deviation of the buffer tube upper bound from the target schedule  the second term penalizes the relative coding rate difference between successive frames.  N is the control window size and σ is a Lagrange multiplier or weighting parameter to balance the two terms. 19

Mohamed Hefeeda Target Schedule Design -The gap between the playback deadline and the target schedule is the desired minimum client buffer duration. -Small gap at the beginning : allows a small start-up delay -Larger gap over time: increases the client’s ability to counter jitter, delays, and throughput changes. 20

Mohamed Hefeeda Controller Design -We wish to control the upper bound by adjusting the source coding rate: -As each frame arrives at the client, a feedback loop can send a message to the server to adjust the source coding rate. -by the time frame n arrives completely at the client, frame n+1 has already started streaming from the server  at time t a (n), frame n + 2 is the earliest frame for which the controller can determine the coding rate 21

Mohamed Hefeeda Controller Design -By defining error rate: -We obtain: -So, -We next define the control input and disturbance: 22

Mohamed Hefeeda Controller Design -Then (14.50) can be rewritten -Therefore, defining the state vector: -Thus, 23

Mohamed Hefeeda Controller Design -Assuming, the disturbance d(n) is a pure white noise we can measure all components of e(n) without using an estimator(perfect state measurement )  Then, the disturbance d(n) does not affect the controller design. -Thus we can use a linear controller represented by: where G is a vector feedback gain 24

Mohamed Hefeeda Controller Design -By the time frame n is completely received, all elements of e(n) are available at the client and u(n) can thus be computed. The ideal coding rate for frame n+2 can then be computed as 25

Mohamed Hefeeda Effect of Frame Rate -Until now, we assumed that the frame rate is constant, but this assumption is not always true: usually video and audio are streamed together, and their merged coding schedule may have no fixed frame rate Even if there is a fixed frame rate f we may wish to operate the controller at a rate lower than f to reduce the feedback rate -To address these issues, in practice we use the notion of a virtual frame rate (f) -We partition media time into intervals of size 1/f ; -and we model all of the frames arriving within each interval as a virtual frame whose decoding and playback deadline is the end of the interval. 26

Mohamed Hefeeda Effect of Frame Rate  Advantages of using virtual frame rate: -it allows us to design off line a universal feedback gain, which is independent of the actual frame rate of the stream or streams. -it allows us to reduce the rate of feedback from the client to the server. -since the interval between virtual frames is typically safely larger than a round trip time (RTT), a one-frame delay in the error space model is sufficient to model the feedback delay.  In the sequel we therefore use a virtual frame rate f = 1 fps, and we refer to this simply as the frame rate. With f = 1 and σ = 50 (chosen empirically for good damping), we can solve for G = [0.6307,−0.5225, ]. 27

Mohamed Hefeeda Controller Interpretation  we are now able to give an intuitive explanation of the source coding rate control (14.57): 28

Mohamed Hefeeda Controller Interpretation -Focusing on the first term (14.63), it can be seen that the source coding rate Rc tends to decrease if the current error e(n) = tb(n)−tT (n) is positive, and it tends to increase if e(n) is negative -from the second term (14.64), it can be seen that the source coding rate tends to decrease if the current error e(n) is numerically greater than the previous error e(n − 1), whether e(n) is positive or negative -Finally, from the third term (14.65), it can be seen that the source coding rate tends to decrease if it had previously increased 29

Mohamed Hefeeda Controller Interpretation 30

Mohamed Hefeeda Conclusion 31  Packet networks are lossy and shared  Users expect good quality, given a an average bandwidth  For solving network problems, we have buffer at client  Good streaming depends on proper buffer duration  We can control buffer duration by regulating source coding rate

Mohamed Hefeeda 32

Mohamed Hefeeda 33