Pre-fetching based on video analysis for interactive region-of- interest streaming of soccer sequences Authors: Aditya Mavlankar and Bernd Girod Information Systems Laboratory, Department of Electrical Engineering Stanford University, Stanford, CA 94305, USA {maditya, Speaker : 童耀民 MA1G

Outline 1.INTRODUCTION 2.ROI PREDICTION AND PRE-FETCHING  Trajectory Prediction  Prediction Using H.264/AVC Motion Vectors  Prediction Tracking Soccer Ball  Prediction Tracking Soccer Ball and Players 3.EXPERIMENTAL RESULTS 4.CONCLUSIONS 2

INTRODUCTION  We consider a video streaming system in which the user can interactively watch an arbitrary region of a high-spatial-resolution scene.  Region-of-interest (RoI) prediction helps pre-fetch select slices of encoded video. 3

INTRODUCTION  Despite the availability of high- resolution video, challenges in delivering this high-resolution content to the client are posed by the limited resolution of the display and/or limited data rate for communications. 4

INTRODUCTION  The goal of the paper is to find out whether domain-specific techniques can predict the client’s RoI more accurately.  The more accurate the RoI prediction the lower is the percentage of missing pixels. 5

INTRODUCTION  In this paper, we focus on interactive viewing of soccer and investigate whether domain- specific RoI prediction based on semantic video analysis is more accurate than RoI prediction based on general techniques that apply to any type of content. 6

INTRODUCTION 7

ROI PREDICTION AND PRE- FETCHING  As part of earlier work, we have developed a graphical user interface [2,3] to allow the user to select an RoI while watching the video.  The application supports continuous zoom to provide smooth control of the zoom factor. 8

ROI PREDICTION AND PRE- FETCHING  The high-resolution layers are encoded using independent slices.  We choose the high-resolution layer that corresponds closest to the user’s zoom factor. 9

ROI PREDICTION AND PRE- FETCHING  If some required high-resolution slices are unavailable, we conceal the error by upsampling portions of the thumbnail video.  We compare the performance of four RoI predictors in this paper.  10

ROI PREDICTION AND PRE- FETCHING  The goal of each predictor is to predict the RoI in frame n + d when frame n is rendered on screen.  The zoom factor for frame n + d is predicted to be the same as the zoom factor observed for frame n. 11

ROI PREDICTION AND PRE- FETCHING 2.1. Trajectory Prediction  We adapt the autoregressive moving average (ARMA) prediction algorithm of [13] to extrapolate the coordinates of the RoI center.ARMA 12

ROI PREDICTION AND PRE- FETCHING 2.2. Prediction Using H.264/AVC Motion Vectors  This algorithm, proposed in our earlier work [12], exploits the motion vectors (MVs) contained within the encoded bitstream of the thumbnail video frames that are buffered at the client. 13

ROI PREDICTION AND PRE- FETCHING 2.2. Prediction Using H.264/AVC Motion Vectors  The MVs are used to find a plausible propagation of the RoI center pixel in every subsequent frame up to frame n+d. 14

ROI PREDICTION AND PRE- FETCHING 2.3. Prediction Tracking Soccer Ball  The RoI is simply predicted to be centered around the ball. 15

ROI PREDICTION AND PRE- FETCHING 2.4. Prediction Tracking Soccer Ball and Players  We have developed our own algorithm for player tracking using background subtraction and blob tracking based on MVs. 16

EXPERIMENTAL RESULTS  We use the Soccer1 sequence having 2560 × 704 pixels and 25 frames/sec.  The RoI display is 480 × 240 pixels. 17

EXPERIMENTAL RESULTS 18

EXPERIMENTAL RESULTS 19

EXPERIMENTAL RESULTS  PSNR (Peak Signal to Noise Ratio) ： 也是訊雜比 ， 只是訊號部分的 值 通通改用該訊號度量的最大 值。 以訊號度量範圍為 0 到 255 當作例子來計算 PSNR 時 ， 訊 號部分均當成是其能 夠 度量的最大 值， 也就是 255 ， 而不是 原來的訊號 20

CONCLUSIONS  For long look-ahead, RoI prediction is very challenging for both kinds of techniques and incurs a large percentage of missing pixels.  Nevertheless, we found that the domain- specific technique performs better though only by about 1 dB, while the drop in PSNR with respect to perfect RoI prediction is more than 3 dB. 21

22

23

24

25

26

27