1. Multimedia & Comm. Lab Video Streaming over the Internet 98/11/25 정승훈 shjeong@mmlab.snu.ac.kr

2. Multimedia & Comm. Lab 2 Contents  Introduction  Architecture of Adaptive Video Streaming  Congestion Detection  Rate Adjustment Schemes  Examples  Conclusion

3. Multimedia & Comm. Lab 3 Introduction  Video Streaming Problems Network  Protocol Issues  Congestion control Receiver capability  receiver’s video playback capability Server capability  contention for CPU and I/O resources QoS  SNR  Displayed Frame Rate

4. Multimedia & Comm. Lab 4 Introduction  Assumptions Clients have heterogeneous network capacity and processing power Large numbers of clients may access the server simultaneously Low startup playback latency The server maintains a large number of different video streams The server and clients are connected through the Internet where the dominant competing traffic is TCP-based.

5. Multimedia & Comm. Lab 5 Introduction  Goals well-behaved and TCP-friendly streaming utilize a fair share of bandwidth maximize the overall quality of delivered stream minimize the storage requirement at the server and the client minimize the playback delay minimize processing requirements at the server

6. Multimedia & Comm. Lab 6 General Architecture Internet Client Encoder rate controller Video source Buffer Server Feedback Information Q factor

7. Multimedia & Comm. Lab 7 System Model Internet Video Source rate controller Feedback collection IP UDP buffer Synch. control IP UDP buffer DeMux / Monitor V bufferA buffer V decoderA decoder Server Client TCP

8. Multimedia & Comm. Lab 8 Congestion Detection  Congestion Factors Packet receiving rate Packet Loss Ratio Packet End-to-end Delay Packet Interarrival Jitter  ACK / NACK based  Using packet receiving rate  Receiver-oriented  Server-oriented

9. Multimedia & Comm. Lab 9 Congestion Detection  ACK / NACK based Columbia Univ. - Dynamic Rate Shaping using TCP congestion control client only sends ACK or NACK server detects network congestion from ACK/NACK  Using Receiving Rate Bamba - IBM ‘97 Receiver monitors packet receiving buffer calculates packet receiving rate sends receiving rate to the server

10. Multimedia & Comm. Lab 10 Congestion Detection  Receiver-oriented Receiver monitors receiving buffer collecting congestion factors determines network congestion degree sends congestion degree to the server server regulate sending rate in accordance with congestion degree

11. Multimedia & Comm. Lab 11 Congestion Detection  Server-oriented ATC, INRIA RTP/RTCP based receiver collects Feedback information  highest sequence number received  the number of packet lost  packet interarrival jitter  timestamps sends Feedback info. by using RTCP packet. server determines network congestion.

12. Multimedia & Comm. Lab 12 Rate Adjustment  Encoder-level rate shaping regulates quantization factor  Frame dropping  Macroblock filtering  Quantization filtering discarding DCT coefficients  Layered Coding  Playback dilation

13. Multimedia & Comm. Lab 13 Encoder-level rate shaping  Model INRIA (Bolot ‘98) real-time encoder adjusts the maximum output rate of the encoder linear increase / multiplicative decrease Congestion if loss rate > 5% NoCongestion if loss rate < 2% if Congestion max_rate = max(max_rate/GAIN, MIN_RATE) else if NoCongestion max_rate = min(max_rate+INC, MAX_RATE)

14. Multimedia & Comm. Lab 14 Frame Dropping  Model StreamWorks, OGIST server determines congestion degree server adjusts frame rate by frame dropping frame ratesend pattern 2.5I I 5.0I P I P 10.0I P P P I P P P 15.0I PB P PB I PB P PB 20.0I BP BP BP BI BP BP BP B 30.0IBBPBBPBBPBBIBBPBBPBBPBB

15. Multimedia & Comm. Lab 15 Macroblock Filtering  block dropping  Macroblock 중 일부를 주기적으로 제거  Client 는 인접 Macroblock 으로 대치.  Feature-oriented block dropping

16. Multimedia & Comm. Lab 16 Quantization Filtering  SNR adaptation video quality 를 떨어뜨려 Data 크기를 줄인다.  DCT coefficient 를 인위적으로 제거  DCT coefficient 를 표현하는 비트 수를 줄임.

17. Multimedia & Comm. Lab 17 Layered Coding  Off-line transcoding Video source 를 video quality 에 따라 여러 계층으로 나누어 Encoding. Frame dropping, Macroblock filtering, Quantization filtering 모두 적용.  서버는 congestion degree 또는 요구되는 QoS 에 맞는 Layer 를 동적으로 선택하여 전송.  Architecture Tech. Co. Adaptive Video Streaming - LCN ‘97  University of Southern California ‘98

18. Multimedia & Comm. Lab 18 Playback Dilation  Client 의 Playback rate 를 줄이는 기법 Receive buffer 가 underflow threshold 를 넘는 경우 client 의 system capability 가 떨어질 경우.  Packet buffering delay 를 인위적으로 늘려서 Client 의 버퍼를 일정한 크기로 유지.

19. Multimedia & Comm. Lab 19 Examples  Berkeley Continuous Media Player ‘92 software feedback mechanism playback dilation  OGIST ‘97 user specification of presentation quality  frame rate, simple QoS Frame dropping  StreamWorks Session 설정시 frame rate 결정 Frame dropping

20. Multimedia & Comm. Lab 20 Examples  Architecture Tech. Co. Adaptive Video Streaming QoS Adaptation  Server, Network, Client Status Report (Feedback Information) Layered Coding  Off-line Video Transcoding  Adaptation table

21. Multimedia & Comm. Lab 21 Adaptation Table

22. Multimedia & Comm. Lab 22 Further Issues  Congestion Prediction ?  Multicast Issues Feedback Explosion problem heterogeneity problem  Video Streaming over DiffServ  Error propagation of lost packet Early packet discarding

23. Multimedia & Comm. Lab 23 Conclusion  Video Streaming over the Internet Congestion Detection Rate Adjustment Related Works  Further Issues