Video object segmentation and its salient motion detection using adaptive background generation Kim, T.K.; Im, J.H.; Paik, J.K.;  Electronics Letters  Volume: 45 , Issue: 11  Digital Object Identifier: 10.1049/el.2009.0663  Publication Year: 2009 , Page(s): 542 - 543 IET JOURNALS Adviser ： Ming-Yuan Shieh Student ID： M9820202 Student ： Chung-Chieh Lien 2010/10/6

Outline Abstract Introduction Background modelling and updating Video object segmentation using proposed algorithm Experimental results Conclusion References 2010/10/6

Abstract Video object segmentation often fails when the background and foreground contain a similar distribution of colours. Proposed is a novel image segmentation algorithm to detect salient motion under a complex environment by combining temporal difference and background generation. Experimental results show that the proposed algorithm provides a twice higher matching ratio than the conventional Gaussian mixture-based approaches under various conditions. 2010/10/6

Introduction-1 Background generation [1] has been an early criterion for video object segmentation, while foreground modelling has recently been used in conjunction with background modelling for more accurate movement detection. Recently, the mixture of the Gaussian method is becoming popular because it can deal with slow lighting changes, periodical motions from the cluttered background, slow moving objects, long-term scene changes, and camera noise. 2010/10/6

Introduction-2 In spite of the above-mentioned advantages, it cannot adapt to the quick lighting changes and cannot successfully handle shadows. In this Letter, we present a real-time robust method that provides a realistic foreground segmentation to detect salient motions in complex environments by combining temporal difference and background generation. The proposed method, shown Fig. 1, aims at performing real-time background generation and salient motion detection of moving objects. 2010/10/6

Introduction-3 Fig. 1 Proposed video object segmentation algorithm using adaptive background generation 2010/10/6

Background modelling and updating-1 As the first step, we estimate the optical flow between two images and by minimising the Euclidean distance defined as: 2010/10/6

Background modelling and updating-2 For each pixel in , where represents the displacement of the pixel at and is initially set to be zero as: where w represents the neighbouring displacement. 2010/10/6

Background modelling and updating-3 If E is smaller than a pre-specified threshold, the background is updated at the corresponding w. In the experiment we have used 0.35 for the threshold value. For a w with high E value the background is generated by minimising E, while the median filter is used for the remaining w. 2010/10/6

Background modelling and updating-4 To overcome the drawbacks of a median filter under dynamic conditions, it is necessary to keep updating the background expressed as where represents the background at time t, the input image at time t, and the mixing ratio in the range [0, 1]. To detect an object’s salient motion in the background, we use the initial background from the previous frame . 2010/10/6

Background modelling and updating-5 Fig. 2 Results of background generation using proposed algorithm 2010/10/6

Video object segmentation using proposed algorithm-1 Temporally adjacent images and are subtracted and a threshold is applied to the difference image for extracting the entire region of change. To detect the slow motion or static objects, a fixed weighted accumulation is used to compute the temporal difference image as: 2010/10/6

Video object segmentation using proposed algorithm-2 Where is the weighting parameter which describes the temporal range for accumulating difference images. is initialised to an empty image. In this Letter, we set T = 20 and = 0.5 for all experiments. We assume that the foreground with salient motion shows consistency over a period of time in both temporal difference and background subtraction. 2010/10/6

Video object segmentation using proposed algorithm-3 It means that the optical flow of the region with salient motion in the given time period should be in the same direction. The salient motion is detected using the temporal difference with background subtraction, along with the change in illumination. On the other hand, simple background subtraction exhibits inaccurate results. The output of salient motion detection is obtained as: 2010/10/6

Video object segmentation using proposed algorithm-4 Where represents the generated background image by using the proposed algorithm. In this Letter, the difference between and is computed, and the difference image is then the threshold for obtaining the change in motion. Fig. 3 represents the temporal differences, subtracted background images, and the detected salient motion regions. 2010/10/6

Fig. 3 Results of foreground segmentation using proposed algorithm a–d: Temporal difference images at 425th, 500th, 551st, 651st frames e–h: Subtracted background images at 425th, 500th, 551st, 651st frames i – l: Detected salient motion regions at 425th, 500th, 551st, 651st frames Fig. 3 2010/10/6

Experimental results-1 The proposed algorithm was tested and compared to conventional methods using both simulated and real video sequences. For evaluating the performance of the proposed algorithm, we compared the segmentation matching ratios of the proposed and Gaussian mixturebased algorithms [7]. The matching ratio pixel is defined as: 2010/10/6

Experimental results-2 where represent the sum of pixels inside the boundaries for manually specified foreground segmentation in the original image, and represent the sum of pixels given by the proposed method. Fig. 4 shows the graph of matching ratios. 2010/10/6

Experimental results-3 Fig. 4 Comparison of segmentation matching ratio to ground truth 2010/10/6

Conclusion We propose a novel video object segmentation and salient motion detection algorithms using background generation to cope with problems in a complicated, unstable background. Experimental results show that the proposed background generation with the salient motion detection approach works twice as accurately as the conventional Gaussian mixture-based method. 2010/10/6

References Mittal, A., and Paragios, N.: ‘Motion-based background subtraction using adaptive kernel density estimation’. IEEE Conf. Computer Vision and Pattern Recognition, Washington, DC, USA, July 2004, pp. 302–309 Wildes, R.P.: ‘A measure of motion salience for surveillance application’. IEEE Conf. Image Processing, Chicago, IL USA, October 1998, pp. 183–187 Wixson, L.: ‘Detecting salient motion by accumulating directionally flow’, IEEE Trans. Pattern Anal. Mach. Intell, 2000, 22, (8), pp. 774–779 Monnet, A., Mittal, A., Paragios, M., and Ramesh, V.: ‘Background modeling and subtraction of dynamic scenes’. IEEE Proc. Computer Vision, Beijing, China, October 2003, pp. 1305–1312 Velastin, S., and Davies, A.: ‘Intelligent CCTV surveillance: advances and limitations’. Proc. Methods, Techniques, Behavioral Research, Wageningen, The Netherlands, 2006 Ren, Y., Chua, C., and Ho, Y.: ‘Motion detection with non-stationary background’. Proc. Image Analysis and Processing, Palermo, Italy,September 2001, pp. 78–83 Stauffer, C., and Grimson, W.E.L.: ‘Adaptive background mixture models for real-time tracking’. IEEE Proc. Computer Vision and Pattern Recognition, Fort Collins, CO, USA, June 1999, pp. 246–252 2010/10/6